1.2. The Good, The Bad, The Ugly
Some filtering techniques are more suitable for use during the SMTP transaction than others. Some are simply better than others. Nearly all have their proponents and opponents.
Needless to say, these controversies extend to the methods described here as well. For instance:
Some argue that DNS checks penalize individual mail senders purely based on their Internet Service Provider (ISP), not on the merits of their particular message.
Some point out that ratware traps like SMTP transaction delays and Greylisting are easily overcome and will be less effective over time, while continuing to degrade the Quality of Service for legitimate mail.
Some find that Sender Authorization Schemes like the Sender Policy Framework give ISPs a way to lock their customers in, and do not adequately address users who roam between different networks or who forward their e-mail from one host to another.
I will steer away from most of these controversies. Instead, I will try to provide a functional description of the various techniques available, including their possible side effects, and then talk a little about my own experiences using some of them.
That said, there are some filtering methods in use today that I deliberately omit from this document:
Challenge/response systems (like TMDA). These are not suitable for SMTP time filtering, as they rely on first accepting the mail, then returning a confirmation request to the Envelope Sender. This technique is therefore outside the scope of this document. [1]
Bayesian Filters. These require training specific to a particular user, and/or a particular language. As such, these too are not normally suitable for use during the SMTP transaction (But see User Settings and Data).
Micropayment Schemes are not really suitable for weeding out junk mail until all the world's legitimate mail is sent with a virtual postage stamp. (Though in the mean time, they can be used for the opposite purpose - that is, to accept mail carrying the stamp that would otherwise be rejected).
Generally, I have attempted to offer techniques that are as precise as possible, and to go to great lengths to avoid False Positives. People's e-mail is important to them, and they spend time and effort writing it. In my view, willfully using techniques or tools that reject large amounts of legitimate mail is a show of disrespect, both to the people that are directly affected and to the Internet as a whole. [2] This is especially true for SMTP-time system wide filtering, because end recipients usually have little or no control over the criteria being used to filter their mail.
Notes
| [1] | Personally I do not think challenge/response systems are a good idea in any case. They generate Collateral Spam, they require special attention for mail sent from automated sources such as monthly bank statements, and they degrade the usability of e-mail as people need to jump through hoops to get in touch with each other. Many times, senders of legitimate mail will not bother to or know that they need to follow up to the confirmation request, and the mail is lost. |
| [2] | My view stands in sharp contrast to that of a large number of "spam hacktivists", such as the maintainers of the SPEWS blacklist. One of the stated aims of this list is precisely to inflict Collateral Damage as a means of putting pressure on ISPs to react on abuse complaints. Listing complaints are typically met with knee-jerk responses such as "bother your ISP, not us", or "get another ISP". Often, these are not viable options. Consider developing countries. For that matter, consider the fact that nearly everywhere, broadband providers are regulated monopolies. I believe that these attitudes illustrate the exact crux of the problem with trusting these groups. Put plainly, there are much better and more accurate ways available to filter junk mail. |