NEW! Now supports FreeBSD! Maildircmd has been updated to work on FreeBSD as well as Linux. It hasn't been tested on other *BSD versions, but ought to work there as well.
Contents:
- Introduction: What is Maildircmd?
- Why Use Maildircmd?
- How do I Install Maildircmd?
- How do I Use Maildircmd?
- Who the Heck are You?
Other Links:
- Read the maildircmd man page.
- Read the serialcmd man page.
- Look at an example filter.
What is Maildircmd?
Maildircmd is an addition to the serialmail package written by Professor Dan Bernstein. Serialmail provides utilities for taking a maildir full of email messages, and mailing them out using either SMTP or QMTP. This provides the ideal mechanism for a store-and-forward email queue: just store your outgoing emails in a maildir, and periodically forward the messages through your email relay using serialmail.
Maildircmd adds two commands to the serialmail package: maildircmd and rewind. Maildircmd takes the contents of a maildir and pumps the messages one-by-one through a shell command of your choosing. The shell command can do anything: forward the messages by email; throw away duplicate messages; refile or index the messages; compile statistics; etc..
Maildircmd runs your shell command once for each email message in the maildir. The message is on the shell command's standard input. Several environment variables are initialized which describe the message, almost identically to the environment provided for commands in your .qmail file. So any filter in your .qmail file can be run instead from maildircmd, by making only minor adjustments.
Why Use Maildircmd?
There are several reasons to spool email in a maildir, and process it periodically with maildircmd. First, filters run under maildircmd are 100% safe, even under NFS. This is because the messages can be spooled in a maildir without any locks, and maildircmd supplies messages to a filter one-by-one--eliminating any need for locking.
Traditional email filters run whenever a message is received. But two messages might arrive simultaneously; what happens then? Without a locking mechanism, both messages might run through the same filter at nearly the same time, causing a disaster:
- If you use standard UNIX mail, your mail folder might be corrupted, destroying one of the messages--or merging both messages into one nasty mishmash.
- If you use MH, your mh-sequences might be corrupted, causing your mailer to forget what you've read and what you haven't.
- If you record information about emails (like senders, or subject lines) then your log file might become corrupted.
- If you filter out duplicate messages, the filtering might fail when the duplicates arrive together. Since duplicates often DO arrive together, that kinda defeats the purpose.
Filtering tools like procmail and maildrop address this problem, fairly effectively, by using a locking mechanism when delivering messages. Locks are not perfect, however:
- Stale locks can jam up your email delivery. (That's also why Netscape gives you those annoying "I can't save your bookmarks" and "Bookmarks have changed and are being reloaded" messages!)
- If you forget to use locks in the right places, then you're back where you started.
- Sometimes locks just don't work at all: when your files are shared using NFS, most locking mechanisms don't work correctly.
With maildircmd, the need for locks is completely eliminated. Mail is processed sequentially, so collisions cannot occur.
The second benefit of maildircmd is simplicity. A simple shell script usually suffices to filter email; often you really don't need complicated mail-filtering software. When you do, though, you can easily run procmail, maildrop, etc. with maildircmd. That gives you the power of procmail, without any of the risks--thanks to maildircmd, you still don't need to worry about locks.
How do I Install Maildircmd?
First, get the source code for Professor Bernstein's serialmail package, version 0.75, and unpack it in the usual way (there is a local copy at this site for your convenience). Maildircmd is distributed as a patch against DJB's sources. And don't worry: maildircmd does not modify any of the programs which come with serialmail; it simply adds the maildircmd programs and documentation.
Next, download the maildircmd patch. Apply the patch from within the serialmail source directory, using the command:
patch < maildircmd-1.0.patch
Finally, build and install serialmail as usual. That's it!
How do I Use Maildircmd?
That's easy. First, you should arrange for incoming mail to end up in a maildir. There are several ways to do that:
- If email is delivered on your machine using qmail, then just
type the following command:
rm -f .qmail*;echo ./Maildir/ > .qmail; touch .qmail-default
- If email is delivered by sendmail, procmail, etc., on your
machine, then you need my safecat package, available
through http://www.pobox.com/~lbudney/linux/software/safecat.html. The
included script maildir allows you to spool your data
to a maildir, even when your delivery agent doesn't know about
maildirs. Just put the following in your .forward file, or into
a procmail recipe. (The USERNAME part should be replaced with
your login name. It's a workaround for a very old sendmail
bug--don't ask.)
|exec /usr/local/bin/maildir ~/Maildir || exit 75 # USERNAME
If you fetch your email from a POP server with fetchmail, you can cut out the middleman and put the following into your .fetchmailrc. Be careful! This only works if your POP server uses qmail! Otherwise, the headers will be all wrong, and serialmail won't even be able to see the messages.
poll SERVER, user USER, password PASSWORD, no rewrite, mda "/usr/bin/env - /usr/local/bin/maildir ~/Maildir"
(Using safecat and maildircmd gives an added benefit to fetchmail users. Fetchmail has reliability problems, and it messes around with email headers. Using this trick, you can make fetchmail leave your email the heck alone!)
Note that you don't have to divert all of your email into a maildir. You can adapt the above instructions to your needs, and spool only certain emails into the maildir. For example, you can use procmail to divert emails matching a certain pattern. Or you can tell fetchmail to use safecat only for certain POP servers. Or you can put the ./Maildir/ instruction into .qmail-EXT, for some EXT, instead of .qmail.
Okay, so now you've got a maildir full of incoming mail. How do you use maildircmd to have some fun with it? First, create a script which does something useful. Here's a good minimal script to try:
#!/bin/sh
#
# Delete duplicate emails in ~/Maildir, and then deliver to my
# inbox. Thanks to maildircmd, no duplicates will be missed, and
# my inbox will never be corrupted.
# First, eliminate duplicates with formail (part of procmail).
if formail -D 65536 "$HOME"/.msgid.cache
then
echo duplicate chucked
exit 99 ;# Message in bit bucket.
fi
# Now, go back to the beginning of the message!
rewind
# Finally, deliver the email. Uncomment the appropriate line.
#/usr/lib/nmh/rcvstore +inbox ;# MH users
#{ echo; echo "$SENDER" `date`; cat } >> $HOME/Mail/inbox ;# mbox users
This simple script eliminates duplicates, and then delivers the message to your inbox. It's not very complicated, but even this simple script gives you the benefit of serialization with maildircmd: no duplicate message will ever make it through; not even when the duplicates arrive simultaneously.
For a more complicated script, look at this example. As a final suggestion, read the manpage for qmail-local(8). Nothing stops you from invoking qmail-local directly, which brings your .qmail files into play (with one important difference: emails are delivered one at a time. Again, you don't need locks and nothing can clobber anything else).
