Fighting Spam, Part II: Webmaster Strategies
Fighting spam — like countering any illegitimate activity — is a battle fought best on multiple fronts. There are simple steps end-users can take to minimize the impact of junk mail (outlined in Part I), but the webmaster or e-mail provider has even more tools available.
If spammers had to gather e-mails individually the effort would outweigh the reward. Their enterprise takes advantage of automation, in the form of spambots, programs that browse websites looking for e-mail addresses, which they then 'harvest' and collect into large lists.
Foiling them by raising the spammer's cost, without turning your efforts into a full-time job, is the most effective way to shift the equation in your favor. Some of the end-user techniques discussed in Part I are even more useful to the webmaster.
CAMOUFLAGE
Spambots can only do what they're programmed to do. Often, disguising a publicly visible e-mail address is enough to cause the spambot to bypass you. They're frequently programmed to look for character strings like John_Example@somecleverdomainname.com. A change to John_Example_at_NOSPAMsomecleverdomainname.com is enough to fool them.
Even if your disguised e-mail address is still harvested, at minimum the address has to be 'scrubbed' in order to be used. Scrubbing routines are even harder to write than spambots, because there are so many possible variations. (NO_SPAM, NOSPAM, no*spam, and many that are much more clever. Be creative!)
The method has a drawback: users have to strip out the extra letters and insert the @-sign (in the above example) — something they sometimes fail to do.
E-mail addresses can be made un-harvestable by embedding them in a graphic, rather than using mailto: or other plain-text options. Very few bots are sophisticated enough to read a graphic and translate the pixel pattern into usable text — particularly since the graphic can have an infinite variety of shapes. Here again, users can't simply copy-and-paste or reply to, so there's some inconvenience for them.
An alternative option is to eliminate visible and accessible e-mail addresses entirely. Instead, provide a feedback form that stores user information in areas unsearchable by spambots. Or, store the information in easy to encode but difficult to crack encrypted formats by using for example javascript.
BARRIERS
Banning visible e-mail addresses, or hiding them in graphical form, makes communication between trusted parties more difficult. Put the burden back on the spammer by blocking known spambots.
They often have an easily spotted signature, in the form of a known IP address or process name (or both), or by looking for non-browser User-Agents.
IP address blocking is a simple matter for any webmaster, but blocking unwanted processes isn't difficult either. Just start a cron job that periodically scans for a process name and uses kill to terminate the associated process ID.
The more sophisticated webmaster can have a daemon that sleeps until a process name is instantiated, wakes up instantly and kills the process before it can do any harvesting. Only slightly more difficult to implement, sample programs are available by searching your favorite engine.
It's possible to set a spambot trap that blocks incoming requests based on excessive search behavior or other pattern. The technique is a little more difficult to implement and administer since it requires defining patterns and altering them for different bots. Again, sample perl scripts and how-to guides are available by a brief search.
RAISE THE PRICE
Eventually, even determined spammers get tired of programming variations to bypass the hurdles thrown in their way, deciding the effort isn't worth the reward. The trick is to make the cost of their effort much higher than the reward, while making the cost to you low and the reward high.
Spammers won't surrender until the profit is taken out of their efforts. Even legislation, such as CAN-SPAM in the U.S., seems to have deterred mostly legitimate businesses who were not the guilty parties.
But junk mail filters are getting more sophisticated, penalties for sending spam are having some effect and there are new proposals being discussed (such as "mailer id") that will eventually reduce the problem to a negligible annoyance.
That's bad news for spammers — which is welcome information for the rest of us.
If spammers had to gather e-mails individually the effort would outweigh the reward. Their enterprise takes advantage of automation, in the form of spambots, programs that browse websites looking for e-mail addresses, which they then 'harvest' and collect into large lists.
Foiling them by raising the spammer's cost, without turning your efforts into a full-time job, is the most effective way to shift the equation in your favor. Some of the end-user techniques discussed in Part I are even more useful to the webmaster.
CAMOUFLAGE
Spambots can only do what they're programmed to do. Often, disguising a publicly visible e-mail address is enough to cause the spambot to bypass you. They're frequently programmed to look for character strings like John_Example@somecleverdomainname.com. A change to John_Example_at_NOSPAMsomecleverdomainname.com is enough to fool them.
Even if your disguised e-mail address is still harvested, at minimum the address has to be 'scrubbed' in order to be used. Scrubbing routines are even harder to write than spambots, because there are so many possible variations. (NO_SPAM, NOSPAM, no*spam, and many that are much more clever. Be creative!)
The method has a drawback: users have to strip out the extra letters and insert the @-sign (in the above example) — something they sometimes fail to do.
E-mail addresses can be made un-harvestable by embedding them in a graphic, rather than using mailto: or other plain-text options. Very few bots are sophisticated enough to read a graphic and translate the pixel pattern into usable text — particularly since the graphic can have an infinite variety of shapes. Here again, users can't simply copy-and-paste or reply to, so there's some inconvenience for them.
An alternative option is to eliminate visible and accessible e-mail addresses entirely. Instead, provide a feedback form that stores user information in areas unsearchable by spambots. Or, store the information in easy to encode but difficult to crack encrypted formats by using for example javascript.
BARRIERS
Banning visible e-mail addresses, or hiding them in graphical form, makes communication between trusted parties more difficult. Put the burden back on the spammer by blocking known spambots.
They often have an easily spotted signature, in the form of a known IP address or process name (or both), or by looking for non-browser User-Agents.
IP address blocking is a simple matter for any webmaster, but blocking unwanted processes isn't difficult either. Just start a cron job that periodically scans for a process name and uses kill to terminate the associated process ID.
The more sophisticated webmaster can have a daemon that sleeps until a process name is instantiated, wakes up instantly and kills the process before it can do any harvesting. Only slightly more difficult to implement, sample programs are available by searching your favorite engine.
It's possible to set a spambot trap that blocks incoming requests based on excessive search behavior or other pattern. The technique is a little more difficult to implement and administer since it requires defining patterns and altering them for different bots. Again, sample perl scripts and how-to guides are available by a brief search.
RAISE THE PRICE
Eventually, even determined spammers get tired of programming variations to bypass the hurdles thrown in their way, deciding the effort isn't worth the reward. The trick is to make the cost of their effort much higher than the reward, while making the cost to you low and the reward high.
Spammers won't surrender until the profit is taken out of their efforts. Even legislation, such as CAN-SPAM in the U.S., seems to have deterred mostly legitimate businesses who were not the guilty parties.
But junk mail filters are getting more sophisticated, penalties for sending spam are having some effect and there are new proposals being discussed (such as "mailer id") that will eventually reduce the problem to a negligible annoyance.
That's bad news for spammers — which is welcome information for the rest of us.
No comments:
Post a Comment