Slamming Spam with SpamAssassin

This chapter is from the book 

Installing SpamAssassin

Before getting to the specifics of installing the supporting software for each MTA, we first cover the installation of SpamAssassin. If SpamAssassin is installed first, several of the packages will automatically determine that SpamAssassin has been installed and adjust their configuration.

The first step is to download the sources, which are available from the SpamAssassin site: http://useast.spamassassin.org/released/Mail-SpamAssassin-2.63.tar.gz . After you have downloaded them to a location such as /usr/local/src, uncompress and extract the files as follows:

bash$ gzip -d Mail-SpamAssassin-2.63.tar.gz
bash$ tar xf Mail-SpamAssassin-2.63.tar.gz
bash$ cd Mail-SpamAssassin-2.63

To install SpamAssassin, perform the following:

bash$ perl ./Makefile.PL
What email address or URL should be used in the suspected-spam report
text for users who want more information on your filter installation?
(In particular, ISPs should change this to a local Postmaster contact)
default text: [the administrator of that system] user@mydomain.com

Checking if your kit is complete...
Looks good
Writing Makefile for Mail::SpamAssassin
Makefile written by ExtUtils::MakeMaker 6.03

bash$ make
bash$ sudo su
# make install

SpamAssassin is now installed. Configuration of SpamAssassin is covered later in this chapter.

SpamAssassin and Sendmail

To give you an idea of how SpamAssassin is integrated into Sendmail, Figure 3.3 illustrates the flow of email through a Sendmail system filtered with SpamAssassin and MIMEDefang on a system-wide basis.

The first step is to download and install SpamAssassin. After that is accomplished, milter and MIMEDefang must be installed to complete the installation.

Installing milter

The next task is to ensure that milter is installed. Under most Linux/Unix distributions, milter is not enabled, so Sendmail must be recompiled with milter support. To check whether your version of Sendmail was compiled with milter support, run this command: path-to-sendmail/sendmail -bp -d0.4 | grep MILTER . If the command returns a blank, then you need to install milter. Otherwise, you can skip this section.

We start off by installing milter. In order to enable Sendmail milter functionality, Sendmail must be recompiled. Let's assume the Sendmail sources are located in the /usr/local/src/sendmail-8.12.10 directory. Create a file in the devtools/Site directory of the Sendmail sources called site.config.m4 with the following contents:

dnl Milter
APPENDDEF(`conf_sendmail_ENVDEF', `-DMILTER')
APPENDDEF(`conf_libmilter_ENVDEF', `-D_FFR_MILTER_ROOT_UNSAFE')

These lines tell Sendmail to run milter and the environment for its execution. After making the changes to the configuration file, rebuild Sendmail by executing ./Build.sh from the root of the Sendmail source directory. Install Sendmail by executing the ./Build.sh install command, and you now have a Sendmail binary that supports milter.

Installing MIMEDefang

The final step is to install MIMEDefang. It is important to note that the MIMEDefang installation automatically detects the SpamAssassin installation and will configure itself appropriately. So, installing SpamAssassin prior to MIMEDefang is a time-saver.

There are two aspects to installing MIMEDefang. MIMEDefang depends upon a number of Perl modules. The developers of MIMEDefang have made a nice package of all the required Perl modules, which can be installed in one shot. Point your browser to http://www.mimedefang.org/node.php?id=1 and download the MIME-tools-5.411a-RP-Patched-02.tar.gz sources. Then install them like this:

bash$ tar xzvf MIME-tools-5.411a-RP-Patched-02.tar.gz

bash$ cd MIME-tools-5.411a-RP-Patched-02

bash$ perl Makefile.PL
Checking if your kit is complete...
Looks good
Writing Makefile for MIME-tools

bash$ make
bash$ sudo su
# make install

The second step is to download the MIMEDefang sources from a repository. A good choice would be the MIMEDefang page at http://www.mimedefang.org/node.php?id=1 . After downloading, the package is installed as follows:

bash$ tar xzf mimedefang-2.39.tar.gz
bash$ cd mimedefang-2.39
bash$ ./configure

bash$ sudo groupadd defang

bash$ sudo useradd -c 'MIMEDefang user' -d /var/empty -s /bin/false defang

bash$ make
bash$ sudo su
# make install

# mkdir /var/spool/MIMEDefang
# chmod 700 /var/spool/MIMEDefang

# cp -p examples/init-script /usr/local/bin/mimedefang.sh

# chown defang.defang /usr/local/bin/mimedefang.sh

The configuration file mimedefang-filter that the installation package includes is reasonable. However, you might need to change some of the parameters, including the following.

To change the email address and name to where notifications go:

$AdminAddress = 'postmaster@mydomain.com';
$AdminName = "MIMEDefang Administrator's Full Name";

To change the email address MIMEDefang uses to send email, change this:

$DaemonAddress = 'mimedefang@mydomain.com';

If you want warnings as part of the message instead of an attachment (default is 0), set this variable to 1:

$AddWarningsInline = 1;

The default action is to send logs via email:

md_graphdefang_log_enable(mail,1);

Activating MIMEDefang/SpamAssassin

To activate SpamAssassin, edit sendmail.mc from your Sendmail source directory to include the following line:

INPUT_MAIL_FILTER(`mimedefang', `S=unix:/var/spool/MIMEDefang/mimedefang.sock,F=T, T=S:1m;R:1m')

This line tells Sendmail to invoke MIMEDefang, which will in turn call SpamAssassin. The Sendmail configuration file is built by running the following command while in the $SRC/cf/cf directory:

# m4 ../m4/cf.m4 sendmail.mc > sendmail.cf

The resulting sendmail.cf file can be installed in /etc/mail and the server restarted by issuing /etc/init.d/sendmail restart.

SpamAssassin and Postfix

The flow of mail under Postfix integrated with SpamAssassin is diagrammed in Figure 3.4 for a system-wide basis setup.

SpamAssassin is activated under Postfix by using the amavisd-new package. More information is available online for amavisd-new at http://www.ijs.si/software/amavisd/ . amavisd-new calls the SpamAssassin libraries directly, making the installation a bit simpler than qmail and Sendmail, which both require additional pieces of software to activate SpamAssassin. Calling the SpamAssassin libraries directly also saves some overhead because additional system resources are not required if spamc and/or spamd are not invoked.

Installing amavisd-new

amavisd-new requires a number of Perl modules to be installed on the target. The INSTALL file notes regarding prerequisites from amavisd-new are as follows:

Archive::Tar   (Archive-Tar-x.xx)
Archive::Zip   (Archive-Zip-x.xx) (1.09 or later is recommended!)
Compress::Zlib (Compress-Zlib-x.xx)
Convert::TNEF  (Convert-TNEF-x.xx)
Convert::UUlib (Convert-UUlib-x.xxx)
MIME::Base64   (MIME-Base64-x.xx)
MIME::Parser   (MIME-Tools-x.xxxx)
(the patched MIME-tools by David F. Skoll is recommended over 5.411,
  as it better handles broken/bad MIME syntax:
    http://www.mimedefang.org/ -> Download section.
  The new 6.2xx from http://search.cpan.org/dist/MIME-tools/
  also includes these patches, and more.
Mail::Internet (MailTools-1.58 or later have workarounds for Perl 5.8.0 bugs)
Net::Server    (Net-Server-x.xx)
Net::SMTP      (libnet-x.xx)     (use libnet-1.16 or later for performance)
Digest::MD5    (Digest-MD5-x.xx)
IO::Stringy    (IO-stringy-x.xxx)
Time::HiRes    (Time-HiRes-x.xx) (use 1.49 or later; some older cause problems)
Unix::Syslog   (Unix-Syslog-x.xxx)

Make sure all of these Perl modules are installed on the target system. If any are missing, download and install them from http://www.cpan.org .

Unfortunately, there is no install script, so the software must be installed and configured manually. To begin, download the amavisd-new sources from http://www.ijs.si/software/amavisd/amavisd-new-20030616-p6.tar.gz in a directory, such as /usr/local/src. Extract it and change directory into the directory by running:

# gzip -d amavisd-new-20030615-p6.tar.gz
# tar xvf amavisd-new-20030615-p6.tar.gz
# cd amavisd-new-20030616

Then create a directory under /var called amavis as the amavisd home directory:

# mkdir /var/amavis

Create the group amavis and user amavis:

# groupadd amavis
# useradd -c 'Amavis Daemon' -d /var/amavis -g amavis -s /bin/false amavis

Make the permissions and ownership correct on the directory /var/amavis:

# chown amavis:amavis /var/amavis
# chmod 750 /var/amavis

Copy the amavisd executable to /usr/local/sbin and change the permissions appropriately:

# cp amavisd /usr/local/sbin/
# chown root /usr/local/sbin/amavisd
# chmod 755  /usr/local/sbin/amavisd

Copy the amavisd.conf configuration file to its default location, /etc, and make the permissions correct:

# cp amavisd.conf /etc/
# chown root /etc/amavisd.conf
# chmod 644  /etc/amavisd.conf

(If you change the location, you must start up amavisd with the -c option to tell it where to read its configuration from.)

Next, you must create the quarantine directory (where amavisd stores viruses that are caught) and set the permissions and ownership:

# mkdir /var/quarantine
# chown amavis:amavis /var/quarantine
# chmod 750 /var/quarantine

Finally, you need to adjust the amavisd.conf configuration file to reflect the appropriate settings. If you followed the preceding recommendations, then set the following values as follows:

$mydomain = 'example.com';
$daemon_user  = 'amavis';
$daemon_group = 'amavis';
$TEMPBASE = "$MYHOME/tmp";
$forward_method = 'smtp:127.0.0.1:10025'; # for postfix
$notify_method = $forward_method;         # for postfix
$inet_socket_bind = '127.0.0.1';          # improves security
$QUARANTINEDIR = '/var/quarantine';

You will want to change example.com to the name of the domain you are receiving email for. $daemon_user and $daemon_group are set to the name of the amavisd-new user—in our case, amavis. $TEMPBASE is set to the amavisd-new variable $MYHOME appended with /tmp. You may want to set this to /var/tmp or /tmp, depending upon your setup. The $forward_method setting tells amavisd-new what to do with the message after processing it. In our case, Postfix expects to receive the message on port 10025 of the local machine. $notify_method tells amavisd what to do with notify messages—in our case, treat them the same as the $forward_method. $inet_socket_bind is set to loopback in order to restrict the IP addresses that are allowed to connect to amavisd. Finally, the $QUARANTINEDIR keyword tells amavisd-new what to do with messages if they are identified as a problem and need to be set aside.

If you are not running virus checks, you will want to enable this line:

@bypass_virus_checks_acl = qw( . );

This will disable virus checking, if necessary. The log level can be set anywhere from 0 (no logging) to 5 (everything is logged). For debugging purposes, start with 5 and then reduce it down to 2 after everything is running smoothly.

$log_level = 2;

After all of the settings have been changed, start amavisd with the debug option to check for any missing Perl libraries or other misconfigurations:

bash$ sudo su
# /usr/local/sbin/amavisd debug

After it starts cleanly, enable amavisd-new to start on bootup by executing the following, assuming you are running a recent version of Linux:

# cp amavisd_init.sh /etc/init.d/
# ln -s /etc/rc.d/init.d/amavisd_init.sh /etc/rc.d/init.d/rc2.d/amavisd

Configuring Postfix

The Postfix configuration required to activate SpamAssassin and amavisd-new is relatively straightforward. Only a few lines need to be added to your main.cf and master.cf located by default in /etc/postfix.

In main.cf, add the following line:

content_filter = smtp-amavis:[127.0.0.1]:10024

The above line tells Postfix to invoke the amavisd-new content filter by connecting to the loopback interface on port 10024. In master.cf, add the following lines:

#
# The amavis interface
#
smtp-amavis unix - - y - 2 smtp
      -o smtp_data_done_timeout=1200
      -o disable_dns_lookups=yes

127.0.0.1:10025 inet n - y - - smtpd
      -o content_filter=
      -o local_recipient_maps=
      -o relay_recipient_maps=
      -o smtpd_restriction_classes=
      -o smtpd_client_restrictions=
      -o smtpd_helo_restrictions=
      -o smtpd_sender_restrictions=
      -o smtpd_recipient_restrictions=permit_mynetworks, reject
      -o mynetworks=127.0.0.0/8

The first configuration entry beginning with smtp-amavis here tells smtp (Postfix's delivery agent) to run in a chroot'ed environment with a maximum of two instances. It invokes smtpd, sets the smtp done timeout to 1200 seconds, and disables DNS lookups to improve performance. The second configuration entry starting with 127.0.0.1 tells amavisd-new to reinject the filtered results into a chroot'ed instance of Postfix's smtpd on port 10025 configured with the listed restrictions.

The next step is to tell Postfix to re-read its configuration files:

bash$ sudo postfix reload

You should now be up and running with SpamAssassin/amavisd-new support in Postfix. You may skip ahead to the "Verifying SpamAssassin Operation" section now.

SpamAssassin and Qmail

The flow of mail when utilizing SpamAssassin and qmail on a system-wide basis is shown in Figure 3.5. For the purposes of this book, it is assumed that qmail (and required associated programs) have been previously installed by the administrator.

In order to integrate SpamAssassin into qmail, a number of additional packages are required. These include

• Qmail-Scanner ( http://qmail-scanner.sourceforge.net/ )
• qmail-queue patch ( http://www.qmail.org/qmailqueue-patch )
• maildrop 1.3.8 or higher ( http://www.courier-mta.org/maildrop/ )
• Perl module Time::HiRes ( http://search.cpan.org/dist/Time-HiRes/ )
• Perl module DB_File (may be included in your Perl distribution; if not, it's available at http://search.cpan.org/dist/DB_File/ )
• Perl module Sys::Syslog (most distributions come with it preinstalled; if not, it's available at http://search.cpan.org/~nwclark/perl-5.8.2/ext/Sys/Syslog/Syslog.pm )

Qmail-Scanner allows anti-virus and anti-spam tools such as SpamAssassin to be invoked by qmail. More information on Qmail-Scanner is available at http://qmail-scanner.sourceforge.net/ . The qmail sources need to have the qmail-queue patch, which is available at http://www.qmail.org/qmailqueue-patch , installed. The qmail-queue patch allows filtering, header rewriting, and other functionality required for programs like SpamAssassin. If you are running Linux, a qmail-queue patched version of qmail-1.03 pre-built RPM prepackaged binary is available at http://untroubled.org/qmail+patches/ . The instructions here assume that you are building qmail from scratch and not using the RPM.

To enable qmail to invoke SpamAssassin, you must apply a simple patch to qmail-1.03. From the http://www.qmail.org/qmailqueue-patch site, copy the lines beginning with diff to the last line from the qmail-queue patch page to a file called patchfile in your qmail-1.03 directory. Apply the patch from the directory containing the root of qmail-1.03 like this:

bash$ patch -p0 < patchfile

Become root and reconfigure, recompile, and reinstall qmail:

bash$ sudo su
# ./config

Next, install maildrop. Please note that you need to have GNU make installed in order to build maildrop. GNU make is installed on most free operating systems such as Linux and BSD variants. To install maildrop, download the sources from http://www.couriermta.org/maildrop/ to /usr/local/src and unpack them:

bash$ bzip2 -d maildrop-1.6.3.tar.bz2
bash$ tar xf maildrop-1.6.3.tar
bash$ cd maildrop-1.6.3

Then, compile and install the package as follows:

bash$ ./configure
bash$ make
bash$ sudo su
# make install

Next, we must prepare to install Qmail-Scanner. The Qmail-Scanner package has a number of prerequisites, namely:

• Perl module Time::HiRes
• Perl module DB_File
• Perl module Sys::Syslog

Be sure they are downloaded and installed prior to continuing the installation. After the environment is set, download qmail-scanner 1.20 from http://prdownloads.sourceforge.net/qmail-scanner/qmail-scanner-1.20.tgz?download to a location on your system, such as /usr/local/src. Unpack the archive:

bash$ gzip -d qmail-scanner-1.20.tgz
bash$ tar xf qmail-scanner-1.20.tar
bash$ cd qmail-scanner-1.20

After that is accomplished, compile and install Qmail-Scanner:

bash$ ./configure
bash$ sudo su
# ./configure --install

After Qmail-Scanner has been installed, you need to update the tcprules to have qmail run the qmail-scanner-queue.pl executable instead of the default qmail-queue binary. This is accomplished by copying the following lines into a file in your local directory called tcprules.tmp:

127.:allow,RELAYCLIENT="",RBLSMTPD="",QMAILQUEUE="/var/qmail/bin/qmail-queue"
10.:allow,RELAYCLIENT="",RBLSMTPD="",QMAILQUEUE="/var/qmail/bin/qmail-scanner-queue.pl"
:allow,QMAILQUEUE="/var/qmail/bin/qmail-scanner-queue.pl"

When loaded into the local tcprules.tmp file, update your tcprules by executing the following command as root:

# tcprules /etc/tcp.smtp.cdb ./tcprules.tmp < /etc/tcp.smtp

This command builds a new tcp.smtp.cdb file with the new configuration from your ./tcprules.tmp file. After the tcprules have been updated and qmail restarted, qmail will process incoming messages through SpamAssassin.