Home
Archive
Columnists
Video
Blogs
Discuss
About
Search
Donate
Advertise

An inside look at the "arms race" between spammers and anti-spam technology.

Advertisement
Advertisement
Advertisement
Advertisement
Register to Vote: Rock the Vote, powered by Working Assets Wireless
Advertisement
  • AlterNetYour turn

Support AlterNet
Do you value the information you're getting from AlterNet? Please show your support with a tax-deductible donation.


Feedback
Tell us how we're doing.

The Meaning of Spam

By Annalee Newitz, AlterNet. Posted December 12, 2006.


An inside look at the "arms race" between spammers and anti-spam technology.
Advertisement

I spend an inordinate amount of time wondering why my spam looks the way it does. Until quite recently, I received about 20,000 spam e-mails every day. The poor little Bayesean filter in my Thunderbird e-mail program couldn't keep up and would routinely barf when confronted with such huge piles of crap from "Nuclear R. Accomplishment" with the subject line "$subject" and a message body full of random quotes from Beowulf.

Before I finally fixed my spam problem -- oh blissfully small inbox! -- I developed a few vaguely paranoid theories. Briefly, I imagined spammers were spying on my inbox and culling sender names from it that matched those of my friends. In my saner moments, I would wonder why exactly spam evolved to look the way it does. Why do spammers keep sending me pictures of pink, bouncy letters that spell "mortgage," followed by text from a random Web site? And why, oh why, do they send me e-mails containing nothing but the cryptic line, "he said from the doorway, where she"? How can that be good business sense?

So I called expert Daniel Quinlan, who is an antispam architect at Ironport Systems as well as a contributor to open-source antispam system Spam Assassin. He patiently listened to me rant about my e-mail problems -- I think antispam experts are sort of like geek therapists -- then explained why I receive spam from random dictionary words strung together into a name like Elephant Q.

Thermodynamic. It's done to fool any spam filter that refuses to receive e-mail from somebody who has already sent you spam in the past. "They want to create a name that your spam filter has never seen before," Quinlan said. It turns out every weirdness in my spam is "probably there for a good reason," he said. In the arms race between spammers and antispammers, spammers try every trick they can to circumvent filtering software.

Often, the spam you get is the result of months or years of this arms race. For example, spammers of yesteryear started sending images instead of text, so that spam filters looking for text like "viagra" would be fooled. Instead, the image would contain the word "viagra," but filters would see only an image and let it through. In response, antispam software began tossing e-mails that contained only an image, since spam containing an image typically has some text with it like "check out my pictures from Hawaii" or whatever. Rarely does a real person send just an image.

Quinlan said spammers figured out their pictures were being chucked, so they started adding a few random words to their mail and got through the filters again. Then antispammers started chucking e-mails with images that also contained random words that didn't make sentences. And that's why, today, you get images with chunks of text taken from random books and Web sites. As long as the text fits into sentences and isn't random words strung together, spam filters have a harder time figuring out if the mail is spam or ham. Spammers also send slightly different images every time, so that spam filters can't identify the image itself as spam. And they fill the images with bouncy, pink letters advertising their crap because character recognition software can't read bouncy letters. So any spam filter that uses character recognition software to look at text in images to find spam will be fooled.

OK, so there is a reason behind the madness. But how could Quinlan explain the spam I get that contains no advertisement for anything, no links nor images, and instead merely quotes some random passage from Dostoyevsky? Quinlan said there's no way to know for sure, but the reigning theory among antispam experts is that it's part of what's called a "directory harvest attack" in which the spammer tries to figure out if there's a real person behind a randomly chosen e-mail address. The spammer sends out millions of innocuous e-mails and may get a slightly different response from the mail server if the mail has reached an actual person. Once the spammer has established that certain addresses are valid, he can send his real spam and be sure that he's reaching an inbox.

All of this sounds perfectly reasonable. Spammers are doing bizarro things to get their messages out. But why do I sometimes get a spam with the subject line "$subject"? Why would I ever be fooled into thinking that was a piece of legitimate e-mail? "That's just some spammer who doesn't know how to use his spamware," Quinlan said. "Sometimes spammers do things that are -- for lack of a better word -- dumb."

Digg!

See more stories tagged with: spam, technology, anti-spam filters

Annalee Newitz (annalee@techsploitation.com) is a surly media nerd who is in recovery from receiving spam.

Liked this story? Get top stories in your inbox each week from AlterNet! Sign up now »


Advertisement

 

Comments Turn comments off sitewide Give us feedback »
Comments closed.
The comments for this story have been closed. Thank you to everyone who participated.
View:
My Aunt Came Up With The Word For Junk Mail-Spam
Posted by: hole11 on Dec 12, 2006 1:23 PM   
Current rating: Not yet rated    [1 = poor; 5 = excellent]
She sent me e-mails wanting me to send other e-mails out so that she could win some diamond necklace. Well she won the contest by sending out the most e-mails, though I didn't help her at all because I thought it was a gimmick and very bogus. She won and was on CNN.

I don't understand why people go to jail for spam when we have freedom of speech and press.

[« Reply to this comment] [Post a new comment »] [Rate this comment: 1 - 2 - 3 - 4 - 5]

» Why not? Posted by: kepstein7777
» RE: Why not? Posted by: hole11
» Freedom of Speech Posted by: zipper696
» RE: Freedom of Speech Posted by: hole11
» RE: Why not? Posted by: RevRick
» RE: Butt Of Course Posted by: hole11
» RE: Butt Of Course Posted by: RevRick
» RE: Butt Of Course Posted by: hole11
Informative and readable
Posted by: kepstein7777 on Dec 13, 2006 3:33 AM   
Current rating: Not yet rated    [1 = poor; 5 = excellent]
Your last few articles have been good: Light yet interesting and substantive. No preaching or spitting in the wind.

[« Reply to this comment] [Post a new comment »] [Rate this comment: 1 - 2 - 3 - 4 - 5]

We should just take action.
Posted by: colinmeister on Dec 13, 2006 6:18 AM   
Current rating: Not yet rated    [1 = poor; 5 = excellent]
I have managed my spam down to a reasonable level, but am still annoyed at the stuff I do receive.

Some are just obnoxious ads, like offers to re-finance my non-existant home loan.

Some could be offensive to some people, like offering me contacts with local slappers who want to have sex with me.

Some are downright criminal, like Nigerian 419ers and Chinamen looking for agents to forward money to them.

Some are laughable, like notifications of my winning a German lottery which I never bought a ticket for, sent from an email address in Mexico.

Maybe the solution is to find out where the mail originates from, and crash the computer system sending it? ISPs will take more notice of what their customers are emailing if their servers keep getting successfully crashed!

[« Reply to this comment] [Post a new comment »] [Rate this comment: 1 - 2 - 3 - 4 - 5]

» what about innocent bystanders? Posted by: thistleblower
» Nigerian ? but.... Posted by: zipper696
» RE: Nigerian ? but....not you too? Posted by: Johnny Hempseed
Don't they know?
Posted by: lamar on Dec 13, 2006 6:30 AM   
Current rating: Not yet rated    [1 = poor; 5 = excellent]
Yeah, but don't they know that I've already had my penis enlarged homeopathically? Seriously, does anybody have figures on how many legitimate transactions are initiated by spam? I suspect if the number was significant, spam would still be getting through.

[« Reply to this comment] [Post a new comment »] [Rate this comment: 1 - 2 - 3 - 4 - 5]

» Forgot to Add: Posted by: lamar
Al Gore, who invented the internet BTW...(-;
Posted by: mn on Dec 13, 2006 7:52 AM   
Current rating: Not yet rated    [1 = poor; 5 = excellent]
If Al Gore had been able to assume his rightful presidency spam would no longer be a problem. Of course he didn't invent the internet (he *was* in on the early legislation that helped move it along) but he's a smart technical guy, and he has an interest in technical innovation, especially when it comes to technology and business (unlike the current butt monkey in the White House who probably thinks that anything that hinders communication--like spam--is a good thing). The reason there is spam is due to a design flaw when ethernet and TCP/IP were first being developed--the designers knew it was a problem, and you can find their early comments in RFC papers from the late 1970s. The internet (known as Darpanet back then) was originally intended as a way for battlefield commanders to be able to communicate after a nuclear war; all of the communicators on the wire were considered by default to be "friendlies" and the main issue was making the transport technology extremely fault tolerant. And that worked just fine when the internet was only being used by academics and gov't researchers, with a sprinkling of Fortune 500 users. But then came the advent of the browser--and commercial interests seeing the great business potential--around 1994-96, and the rest is history. Everything that travels on the telephone network is authenticated, meaning we always know where it comes from. That's not true on the internet. It *could* be true, if there was someone in the White House who saw that businesses were being robbed of billions of dollars annually due to a silly technical problem. The solution to spam is leadership: lock the vendors up in a room and tell them they will get only bread and water until they agree on a standard for authenticated packets. There, I just solved the spam problem. BTW, this solution has been on the shelf for years. The only thing that prevents it from happening is the fact that we all live in Potterville instead of Bedford Falls. And for that you have to thank our glorious military industrial complex and its idiot monkey savants like George W. Bush.
Michael P. Anderson
mikea@clientworks.com

[« Reply to this comment] [Post a new comment »] [Rate this comment: 1 - 2 - 3 - 4 - 5]

found a great service-nothing else like it
Posted by: studiosus on Dec 13, 2006 10:24 AM   
Current rating: Not yet rated    [1 = poor; 5 = excellent]
Try www.egismail.com --web-based email, completely secure, private, no datamining, no advertising, even better--no spam

[« Reply to this comment] [Post a new comment »] [Rate this comment: 1 - 2 - 3 - 4 - 5]

Am I the only one?
Posted by: sprachenlehrer on Dec 13, 2006 10:23 AM   
Current rating: Not yet rated    [1 = poor; 5 = excellent]
Am I the only one who reports SPAMMERS to their ISP? Yes, it's time-consuming, but by electing to receive "Full Headers" one is able to determine where SPAM originates and to report the SPAMMER to his service provider. Yahoo.com is especially vigilant and will shut down the SPAMMER's account. Most, if not all ISP's will do the same thing. One must track the SPAMMER using services such as ARIN, APNIC, AfriNIC, JPNIC, KRNIC, LACNIC, and RIPE -- but I get great satisfaction in knowing that my reports are the catalyst that shuts down a few of them every day. Hopefully, it will result in NO SPAM coming to my account. Want to learn more? Start here: http://spam.abuse.net/userhelp/howtocomplain.shtml

[« Reply to this comment] [Post a new comment »] [Rate this comment: 1 - 2 - 3 - 4 - 5]

» RE: Am I the only one? Posted by: fitzjohn
catching raindrops
Posted by: mn on Dec 13, 2006 11:44 AM   
Current rating: Not yet rated    [1 = poor; 5 = excellent]
However personally satisfying your approach might be, you are trying to catch raindrops here. You are having zero effect on stopping the larger picture of spam. Many, if not most, spamming machines are bots that the users don't even know are being used to spam. The spam problem [and all other network-sourced malware, for that matter] is enabled by a flaw in the internet's architecture, and fixing that is the only thing that will ever stop spam. But it's probably too late, since there is now a huge anti-spam industry that would suffer immensely if spam were eliminated. What a waste of human time and money. Spencer Tracy is probably throwing up in his grave (this is an allusion to "Desk Set" where efficiency expert Tracy helps to keep the humans as productive as possible)...mikea@clientworks.com

[« Reply to this comment] [Post a new comment »] [Rate this comment: 1 - 2 - 3 - 4 - 5]

how did you fix the spam problem?
Posted by: oakbog on Dec 13, 2006 2:15 PM   
Current rating: Not yet rated    [1 = poor; 5 = excellent]
A good timely article, and a much discussed topic these days with the exponential rise of spam. I too have been inundated, and particularly puzzled by those totally random spams containing no links, ads or offers to buy anything. I assumed they were 'scouts' testing for weaknesses in antispam filtering, which seems to be the industry guess of the moment also.

One intriguing item was mentioned and left dangling; you write:

I received about 20,000 spam e-mails every day ... Before I finally fixed my spam problem -- oh blissfully small inbox!

What did you do to fix the spam problem? Please share!!!

[« Reply to this comment] [Post a new comment »] [Rate this comment: 1 - 2 - 3 - 4 - 5]

» RE: mn--no squirting here Posted by: studiosus
Apocalpso B. Smithering
Posted by: kelt65 on Dec 15, 2006 6:49 AM   
Current rating: Not yet rated    [1 = poor; 5 = excellent]
These bizarre emails from the ether are signs of an awakening conciousness.

In 2012, Spam will become aware.

God help us all.

[« Reply to this comment] [Post a new comment »] [Rate this comment: 1 - 2 - 3 - 4 - 5]

» RE: Apocalpso B. Smithering Posted by: eringhorm
spam takes about 50% of all data-traffic on the internet!
Posted by: han on Dec 16, 2006 5:23 AM   
Current rating: Not yet rated    [1 = poor; 5 = excellent]
Spam takes about 50% of all data-traffic on the internet! And I don't think they make a real profit of it. I think something else is going on here which goes along very familiar lines: Create a problem, Manipulate reaction, Offer solution.
In other words, some people are fucking up our lovely free internet just to take it into control again. And the only solution in my opinion is not accepting the spam-traffic. Let me explain the basic idea: I run my own mailserver and it's set up with a tarpit, as soon as someone tries to send mail he is tested against a blacklist and if he matches his connection is being forwarded to the tarpit, which sends him a bogus message at the speed of one character per second (0.001kb/s). The spammer now has to waste time and resources and the rest of internet is unloaded. This is the only real solution. Spamfilters may detect mail, but for them to work you still have to receive the message. With the tarpit method you stop the message at the source.

[« Reply to this comment] [Post a new comment »] [Rate this comment: 1 - 2 - 3 - 4 - 5]

Even worse than gun control, there are too many regulations on the net to let spammers get away !
Posted by: maxpayne on Dec 18, 2006 10:27 AM   
Current rating: Not yet rated    [1 = poor; 5 = excellent]
Maybe the author should go back and review all those BULLSHITTY regulations and filters along with the OBSCENE costs ISPs and our fucked up government along with Corporate America place on the customers. Remember the CAN SPAM ACT and those silly filters ISPs promise will reduce spam but in the end block more legitimate email all the while come nowhere close to filtering out spam given that spammers can and will easily circumvent the ISPs filters? Currently, even worse than gun control which does nothing to deter gun thieves, our current "free" market not only does nothing to hold spammers accountable, the fucked up system actually rewards them for it ! That should be something the author may want to consider the next time she posts another BULLSHITTY article to try to make us feel powerless ! STOP overregulating customers and HOLD SPAMMERS ACCOUNTABLE, STUPID !

[« Reply to this comment] [Post a new comment »] [Rate this comment: 1 - 2 - 3 - 4 - 5]