Like this article? We recommend
Like this article? We recommend
Like this article? We recommend
A number of tools are available for the harried mail administrator. Thinking of E-mail in terms of postal mail, it's straightforward to classify these filtering tools according to whether they "open the envelope." In other words, some tools care only about what's written on the outside (information about the sending mail server and the SMTP mail delivery instructions), while others perform a thorough examination of the mail's actual contents (the mail headers and body).
Envelope Tests
When one mail server sends mail to another, a few vital pieces of information are exchanged:
IP address of the host connecting to the receiving mail server
Sender's (supposed) hostname
Sender's (supposed) E-mail address
List of recipient E-mail addresses
Before the receiving mail server agrees to accept the rest of the mail, it has a chance to use this bit of information from the "outside of the envelope" to decide whether it should reject the mail outright. A few rudimentary tests can be performed:
Is the connecting host on a DNS block list?
Are the mail's recipients actually valid users?
Is the connecting host "authorized" to send mail from its domain?
Let's take a look at each of these types of tests, and what they accomplish.
DNS Block Lists (DNSBLs)
The IP address of the connecting client can be looked up against various DNS-based lists of spammers, spam sources, and spam-related sites. These lists serve as an informal "reputation rating" for Internet hosts (from individual IP addresses to blocks of IP addresses to entire ISPs or domains), in much the same way that credit bureaus let creditors get a feeling for whether a potential customer is likely to make his payments. IP addresses that become known for sending out a lot of spam end up on these lists, which are generally available for any mail administrator to consult.
The purpose of block lists is twofold:
Protecting one's own network from receiving mail from sites that are known to send spam
Punishing those sites in the hope of encouraging their administrators to reform their ways
The power of these lists, for better or worse, is achieved when large numbers of sites use them; as more sites use a given list, more parts of the Internet refuse mail from listed sites. This strategy can be crippling to a listed site, isolating it from much of the Internet. When a listed site meets certain "cleaned up" criteria laid out by the operator of the particular block list, it can be removed from the list.
Block lists are not without controversy. The fact that each block list operates by its own rules and sets its own criteria for listing and delisting a site means that mail administrators need to be careful about which ones they choose to use and trust. Some are more aggressive than others, more willing to list large domains—for example, all of AOL.com. You might imagine that blocking all mail from so many millions of users can have a crippling effect on your own users who need to talk to people who are AOL subscribers. Since a certain amount of human judgment is involved in the listing process, innocent sites can also end up on these lists, and some block lists are slower than others at removing sites.
That said, most DNS block lists are quite effective; the more reputable lists, such as the Spamhaus Block List, are in widespread use these days. Their effectiveness is not lost on spammers, who have targeted several DNSBLs with distributed denial-of-service (DDoS) attacks over the past year. By their very nature as DNS-based lists, DNSBLs are vulnerable to such attacks, and two of them—Osirusoft and monkeys.com—were forced to shut down last year as a result of prolonged DDoS attacks.
Tests for Valid Recipients
It seems like common sense to suggest that your mail server should only accept mail addressed to actual users it knows about, but an amazing number of mail servers out there aren't doing this relatively simple check before accepting mail. Instead, they accept the mail and only later in the process discover that the recipient doesn't exist, forcing the mail server to then bounce an error message to the (supposed) sender.
Where this causes serious problems is with "dictionary attacks," used by spammers to try to find out which E-mail addresses actually exist at your site. When a dictionary attack occurs, the spammer sends mail to a large number of addresses that start with common names (jack@..., jackie@..., jacklyn@..., jackson@..., and so on) in the hope that some of these won't result in a bounce-back message. The ones that don't bounce, of course, are valid addresses that the spammer can add to his list. In the meantime, forcing your server to accept this mail in the first place can seriously slow that server during a dictionary attack, when it might well be faced with thousands of such "tests" an hour.
Fortunately, most modern mail servers have built-in features that allow you to explicitly list the E-mail addresses for which you want to accept mail, and automatically reject mail destined for other addresses (for example, Sendmail's "virtusertable" feature).
SPF: Sender Policy Framework
In theory, mail servers should be receiving mail only from local mail clients, and from other mail servers that are "authorized" mail exchangers (MX'es) for their domains. This is how the Internet E-mail system was designed to work, but this intention is not strictly enforced. Knowing this, spammers often take advantage by connecting directly to mail servers to send their spam from dial-up accounts and broadband-connected hosts—hosts that are not listed as "official" mail servers for their domains.
Letting just anyone connect to your mail server generally leads to bad things. If your mail server is poorly configured, it might well agree to deliver mail to anyone from anyone, whether the recipients are local or not. That's called an "open relay," one of the spammers' favorite tools for broadcasting their mailings at your expense. Even if your server is secured against relaying, though, spammers connecting directly to your mail server can use that connection to deliver spam to your local users. Ideally, you should accept mail only from clients and servers that are properly authorized to send your server mail.
Sender Policy Framework (SPF) is an attempt to finally start enforcing this longstanding convention. It works by having administrators add a bit of extra information to the DNS records for their domains, listing the hosts that are authorized to send mail from that domain. When your mail server receives a connection from a client, you can check its IP address against the SPF information in the DNS record for the sender's supposed domain. If the client's IP address is not in that list, it's not an "authorized" mail server for that domain, and should therefore be refused a connection. If there's no SPF information in the domain's DNS record, however, no conclusion can be drawn.
Before SPF will be effective, it needs to be adopted on a reasonably large scale. Administrators need to add SPF information to their DNS records, and mail server software needs to be updated to incorporate SPF support. On the other hand, AOL has been testing SPF, and has plans to implement this technique in 2004. When that happens, AOL will reserve the right to refuse E-mail from sites that don't have SPF records, and that may well be the incentive that administrators across the Internet need to get onboard, given the sheer number of AOL subscribers.
And That's It for Envelope Testing
So now you're familiar with the many hazards that lurk in the average user's E-mail inbox, as well as the technique of the "envelope test," where a number of quick checks can help to cut down the amount of spam your mail server bothers to process. These techniques alone can save a significant amount of bandwidth and CPU cycles—not to mention user and administrator sanity—in the battle against spam. As this series continues, you'll learn about more of the tools available when fighting against this mail administrator's nemesis, and then we'll get into the meat of the thing: how to put all of this knowledge to use.