SPAM Filtering Technology

SPAM
We all get it, we all hate it—Spam. Few of us know where it comes from or how to prevent it. For all the clever ways junk mail peddlers have for finding us, email administrators try to be one step ahead—and there are lots of solutions out there to help them. It is not just email, however. Usenets and Newgroups have it, instant messaging providers, web search engines, even some phones.

HISTORY
Other than the trademarked name for a canned product made of pork shoulder and ham, the term "Spam" was made famous by a ridiculous Monty Python sketch in 1970 Spam is also known as Unsolicited Commercial Email (UCE) or Unsolicited Bulk Email (UBE).

SOURCES OF SPAM
We don’t like Spam, but where does it come from? The US is the single largest producer of Spam, with output exceeding that of South Korea, Russia, China, and Turkey (the other leaders) combined. Unsolicited mass mailings have been around since the days of the telegraph. Arpanet had Spam in 1978 in all but name. Since it is profitable, easy to produce, hard to stop and the Internet Service Providers have to handle the cost of constant bandwidth increase, why not Spam?

DEALING WITH THE SPAM CHALLENGE
The U.S. spends nearly as much as the rest of the world combined trying to stop, cleanup, and just deal with Spam. Spam distribution violates ISP user agreements, and there are costs from fraud, identity theft, and other crimes (e.g.,, spoofing) to hide its source. So is it free speech? Many countries have laws against it, but when and what is blocked is considered censorship by some.

Not unexpectedly, the technology industry has mobilized to mitigate and try to block Spam. A number of organizations have sprung up to create Real-Time BlackLists (RBLs) or Spam-Block Lists (SBLs) to create up-to-the minute databases of known spammer email domains, IP ranges, and content types. One such organization is the non-profit Spamhaus Project. Based in Geneva and London, this organization has presence in ten different countries and maintains the Register of Known Spam Operations (ROKSO), SBLs, and a massive DNS infrastructure to go with it. As of October 2009 they are protecting over 1,400,000,000 email boxes worldwide.

RBL sites collect information from the internet looking for open relay and open proxy ports on servers that can often be exploited to send Spam. Organizations like Spamcop provide a feedback loop to allow a community of users to report Spam and subsequently block these messages.

One of the biggest challenges in identifying sources of Spam is that spammers use multiple accounts, sometimes only one time, to stay a step ahead of ISP enforcement agents. Even more frightening is that a lot of Spam is now generated from botnets, which can be activated at any time by their operators to send Spam or complete other nefarious tasks.

Regrettably, Spam has some more evil cousins that ride in on the Spam bus – viruses often arrive through attachments inside unsolicited emails, either intentionally sent by a miscreant or unintentionally sent by a virus-laden computer. And cleverly-written phishing emails pose an even more serious threat by duping the recipient into voluntarily forfeiting personal information such as bank account numbers and passwords.

PROTECTING YOURSELF
By now, most organizations have recognized the challenge of Spam and are using some Spam mitigation strategy, including some or all of these approaches: Sender blocking, keyword blocking, RBL's, document filtering, "safe sender" whitelists, and a scientific filtering process based on Bayesian recursive analysis. There are even extensions to DNS such as SPF, designed to help mitigate Spam. Spam mitigation systems can operate at all levels of the email handling process from the ISP, through specialized companies, in premise-based appliances, on servers, or at the user level.

Commercial options abound. But be aware of any solution that relies on only one technology or method. The face of Spam is constantly changing, so a solution that incorporates dynamic updates will be better suited. Dedicated Spam- and virus-filtering managed services have significant human and technical resources to devote to filtering for just a few dollars per mailbox. Many of these services offer additional features, like attachment blocking, in- and out-bound scanning, and content filtering for inappropriate words or subjects.

Other important mitigation strategies include not directly posting or publishing email addresses on the web, since spammers routinely rake the web looking for addresses. Company websites can hide email addresses by using secure forms. Generic addresses are to be avoided (e.g., sales@ or info@) as well. Due to the risk of an infected machine spamming the world, ensure that your workstations have antivirus and antispyware software installed and updated. Lastly, make sure users delete messages where they don’t recognize the senders, and certainly not to reply.

CONCLUSION
With the amount of legitimate and unsolicited email so high and its low barrier to entry, Spam is an unnecessary evil that is not going away anytime soon. The face of Spam is constantly changing, so an aggressive posture against it is a reality in today’s business climate.


Spammers running your day? Click here to find out more about InfoStructures' FilterPoint content filtering service.