Email system part #2, Configuring and updating.
Configuration.
Please read the first part of this article first, it explains the process that email goes through as it wends it way through the system and gives some background to what we’ll discuss here.
The instructions on this page are only for those who are comfortable in working in a Unix environment, who understand how to copy, move, backup and edit files. If you are not sure what you are doing, then don’t. Ask someone who is familiar with the environment.
Configuring sendmail
The first rule of configuring sendmail is don’t! If there are other ways of achieving what you want, then use them. Sendmail is notoriously tricky to set up correctly and if you get it wrong you will probably lose email, or worse, end up with an open relay. I’ll give a brief indication of what is possible here and refer you to the online manuals for the gory details.
There are two files that we can use to influence the rules of delivery with sendmail: /etc/mail/virtusertable and /etc/mail/access. Well, there are more, but these are the two that are the easiest to deal with here. The access file is used to specify which hosts and accounts can make use of sendmail, the virtusertable specifies how to deliver mail to a virtual domain recipient and can also rewrite the outgoing addresses of emails.
The access file is a simple file of key/value pairs separated by white space. The key specifies a domain name, IP address or email account and the action is to accept, reject or deny the email. For example:
spammer@spamking.com REJECT
will reject mail from that email address.
spamisus.com REJECT
will reject any email from that domain.
As we also have blacklist_recipients turned on in the sendmail configuration we can also blacklist recipient addresses:
sales@mydomain.com
will refuse mail that is sent to that email address.
All this gives us quite a bit of flexibility in controlling who can send and receive emails and for further details I’ll refer you sendmail.org’s anti-spam pages (about half-way down the page).
Probably the most important thing you can do here is to disable the ‘catchall’ email address (do it through Site Manager) and only receive emails to specific email addresses. As most of the recent virii/worms create a fictitious email address to send email to, this will cut down on many of the worms you receive.
The virtusertable is a similar format file and is used to tell sendmail which virtual domain users map to which local account. For example:
joe@mydomain.com joe
is a normal mail mapping that would be set up if you created a user through
Site Manager. It is also possible to map the email to an off site domain:
joe@mydomain.com joe@another.com
which has the effect of forwarding the email on to another.com.
This means that we can use virtusertable to route email sent to virtual domains on our system to system accounts, like so:
joe@domain1.com joe
joe@domain2.com joe
jane@domain1.com jane
It’s important to note from the above (which should be self explanatory) that we cannot have joe@domain1.com and joe@domain2.com as two different users, unless we create two different accounts (say joe1 and joe2). This is a common source of confusion. A Unix system deals with user accounts, each of which must have a unique name. The system knows nothing about virtual users and domains and such. So we have to map these somewhere else (e.g. in virtusertable for sendmail, in httpd.conf for Apache).
There are many further possibilities for using the virtusertable file, and for more details you can check out the virtual hosting pages at sendmail.org.
Important: after making changes to either the access or virtusertable you need to recreate the hashed maps, to do this type:
$ make && newaliases
in the shell while in the /etc/mail directory.
In addition to the above, sendmail also allows you to look up the senders address of an email to check if it is on a Spam List (Realtime Block List) and to do Domain Masquerading, neither of which I will cover here as I don’t use them, but for further details, read AntiSpam (look for ‘dnsbl’) and Masquerading.
Configuring procmail
procmail receives the email from sendmail and then matches the email against a set of rules specified in, firstly, /etc/procmailrc and then looks for a file $HOME/.procmailrc. This means that we can have a system wide procmail file and one for each user. On a Westhost server this means /etc/procmailrc and possibly /.procmailrc (for the default user) and possibly /ftp/pub/usernames/.procmailrc if there are other users.
Procmail has an excellent set of man pages, that you can read on a man server is you don’t have them installed on your own machine (e.g. http://www.rt.com/man). The default /etc/procmailrc file contains a couple of lines of headers and then a small recipe to handle mail that excedes the mail quota for the account. If spamassassin has been installed on your server then there will be an additional rule to send the email off to spamassassin for checking.
In the procmailrc (or ~/.procmailrc) file we specify a set of tests and actions to take if those tests succeed. Procmail allows us to write very powerful tests using regular expression syntax and it would be easy to write a book about the various possibilities. For now, let’s examine the rules that appear in a default Westhost installation of sendmail/procmail/spamassassin.
In the /etc/procmailrc file, you will find these lines:
# SPAMASSASSIN BLOCK
# The condition line ensures that only messages smaller than 250 kB
# (250 * 1024 = 256000 bytes) are processed by SpamAssassin. Most spam
# isn't bigger than a few k and working with big messages can bring
# SpamAssassin to its knees.
:0
* ! ^FROM_DAEMON
* < 256000
{
:0 fw: /var/lock/spamassassin.lock
| /spamassassin
}
# END SPAMASSASSIN BLOCK
The lines that start with :0 are the start of rule, the lines that start with is the rule itself and the line(s) after the rule is the action to be taken. The above section, in English, would say:
"For all emails that are not sent by our local mail program and that are less than 256kb in size, do what is inside the ‘{‘,’}’."
Inside the braces we have:
"Filter this email, wait for it to return and lock the file. The filter is located at /spamassassin"
So, small emails are sent off to spamassassin and returned to procmail after processing. Spamassassin will have written some new headers into the email to indicate its status and the likelihood of it being spam. Also Spamassassin will change the subject of the email to include the phrase "* POSSIBLE SPAM **" if the score is above a certain value. (All of the above is configurable). So now we can setup a filter in procmail to act on those new headers. Assuming that we consider a spam score of over 10 to be definite spam, between 5-10 to be possible spam and under 5 to be legitimate emails, then we want to take the following actions:
over 10 — delete the email
5 – 10 — move the email to spam mailbox
under 5 — deliver as normal
The following set of rules should take care of the situation (assuming we’ve set the spam trigger level at 5 in spamassassin, see below):
# 10 stars or more… :0: * ^X-Spam-Level: ********** /dev/null # remainder with a spam flag # delivered to spammail mailbox :0 * ^X-Spam-Flag: YES /var/spool/mail/spammail # anything else should be delivered normally.
The possibilities of what you can do with procmail are almost endless and I don’t have the space to enumerate them all. The procmail man pages are well written and thorough (see procmail, procmailrc, procmailsc and procmailex) and there are many resources on the web, possibilites are procmail.org, some excellent sample rules or Timo’s Tips or the procmail.faq.
Configuring spamassassin
SpamAssassin may not be installed on your server, in which case you will need to install it through the Site Manager.
SpamAssassin works by analysing each email and comparing the various headers and message bodies against a sample set of strings. So spam senders are constantly changing the strings in their emails to stay one step ahead of SpamAssassin’s rules. This makes it essential to keep the rules database up to date. Another tool that SpamAssassin can use is Bayesian analysis, where the text of emails is analyzed and, effectively, learned from.
SpamAssassin looks for its rules in three places: /usr/local/perl/share/spamassassin/, /etc/mail/spamassassin and /home/user/.spamassassin (or sometimes /.spamassassin). In the first are the global set of rules, the second, in /etc/mail/spamassassin/local.cf, holds the site wide rules that apply to all emails and the third set of rules, in /home/user/.spamassassin/user_prefs, apply only to that user.
The first of the above, default rules, we can leave alone. They work fine as they are.
The local.cf and the userprefs have the same format and in these files we can place configuration directives and rules. The local.cf file already has some directives in it (with comments) and generally they can be left as is. The users file (userprefs) is the best place to add additional rules and directives.
Most of the directives in local.cf are self-explanatory and further information on them will be found in the man pages for SpamAssassin or on the sites listed below.
The user configuration file, userprefs is where we add extra rules. For example, to flag all mail containing the URL ” we can set this rule:
body PHARMAWHAREHOUSE /pharmawarehouse.biz/ describe PHARMAWHAREHOUSE Link to pharmawharehouse.biz score PHARMAWHAREHOUSE 6.0
which will have the effect of checking the body of each email for that url and flagging it as spam. Reading through the default rules at /usr/local/perl/share/spamassassin/ is a good source of ideas for other rules of your own making.
We can also set blacklists of domains or email addresses, such as:
blacklistfrom *@choiceandvalue.com *@bestvaluenow.com *@myquickdeals.com
and it may be a good idea to whitelist emails that you know are safe, such as:
whitelistfrom girlfriend@fastmail.fm
which will immediately allow through those very important emails.
Note that the default path for the site-wide configuration mentioned above will change if you have upgraded SpamAssassin to a later version, they will be somewhere else. See the section on upgrading SpamAssassin.
As with sendmail and procmail, I’ll refer you to other sites for far more information than I can present here. Good sources are the spamassassin.org web site and the spamassassin wiki. For a set of rules that cover almost every contingency, check out RulesEmporium.
The rulessets mentioned above change quite rapidly as folks redesign them for the latest spam, in fact some of them change daily. In this case it may be worth setting up RulesDuJour, which is a script available from RulesEmporium that can be run as a cron job (using myrulesdujour. I’m running this on site for a while now without problem.
For information on upgrading Spamassassin to the latest version, take a look at the Perl pages.
SpamAssassin and your users
As a default, I don’t install SpamAssassin for my clients unless they specifically ask for it. Many people are not troubled by spam and so I don’t add it to the account. If they do want spam filtering, then I do a default installation where the spam is simply tagged and explain how they can filter it on their end, with their email client.
Some users insist that they want the spam deleted on the server, and in this case I make it very clear to them that it is not spam that will be deleted, but emails that appear to be spam. This is an important difference that the client needs to understand and I need to be sure that I am not held legally responsible for any loss of emails. Be very clear with your clients on this matter! And, even then, I usually raise the spam threshold to 8, so that some will get through and the ones that get deleted are pretty certainly spam.
Trouble shooting
For setting up and testing an email system, especially for the anti-spam measures its worth collecting a few sample emails for testing purposes. Have some obvious spam, some borderline spam and so on (and one from your girlfriend to make sure it arrives), upload them onto the server and save them in a directory (e.g. ~/spam-tests). These can then be used for testing rules in procmail and spamassassin.
While setting up the email it is a good idea to turn on logging for sendmail and procmail, details on this on the logging page.
To check what procmail is doing:
$ procmail testrc < testemail.txt
then check the logs to see what happens
To check what spamassassin is doing:
$ spamassassin -D –lint < testemail.txt
and check the output.