Index Invaders

People do funny things when they need attention, like jumping up and down or piercing important body parts. On the Web, where attention is measured in "hits" -- individual visits to a site, more or less -- getting extra attention often involves technical sneakiness.High hit counts can be particularly appealing if your site carries ads, as many do, that put a penny or so in your pocket for every hit. In this case, obviously, more is better. What's a cyber-entrepreneur to do? You could create a high-quality, award-winning site that becomes a sensation through word-of-mouth, but that requires hard work, intelligence, and patience. Isn't there a lazier and less ethical way to bring thousands of unsuspecting people to your site?Yes. It's called "spamdexing," an umbrella term for tricks designed to make a site more likely to pop up in the search results of automated indexes like Infoseek, WebCrawler, Lycos, AltaVista, and HotBot.If you ran a site that hawked a line of pet care products, say, wouldn't you want to be at the top of the list of 100,000 pages that results when somebody innocently searches for the word "dog"? You would. And spamdexing might get you there.I first ran across this technique almost two years ago, when a page labeled "Red Herring" showed up in a non-herring-related search I had just done with WebCrawler. The page turned out to be an experiment by a clever fellow named Tom White, who had discovered a way to make his page show up in any WebCrawler search regardless of the search keywords used. Unlike today's spamdexers, White did not have a financial motive. The page's only purpose was to show that hijacking was possible. "I could have led people wherever I wanted, to propaganda I was preaching or a product I was pushing, but I thought I'd just do something silly," he told me.In less than two weeks the previously unknown page registered over 50,000 hits. Some of the visitors were less than amused; "I've even gotten a couple of threats," White said. Eventually, he took the page down. He kept his technique secret in order to let WebCrawler staff quietly patch the hole.Today, the standard spamdexing techniques are crude and unexciting: fill your page with words people are likely to search for; repeat the words over and over; put the words in the title line of each page (most search engines weigh the title heavily in ranking); make the words invisible to people, but visible to the search programs, by coloring them black-on-black or white-on-white.Signs of trouble for spamdexers are beginning to appear, though.Sites that use other people's trademarks in an attempt to lure visitors, for example, are getting served with lawsuits. This is gratifying to see, if only narrowly applicable.Relief is more likely to come from search engine policies that make these tactics counterproductive. Two or three of the major search engines relegate pages with lots of repeated words to the bottom of the results list. Similar penalties are assessed for the misuse of standard indexing labels.If search technology were the only variable involved, these countermeasures might eventually lead to a Web where only the most relevant documents appeared in a search. To believe that would be to forget the trump card, however: cash. Some search engine sites already sell advertising that will appear only for certain searches (compare the ads accompanying the results for a search on "sports" versus one on "sex"); the idea of selling top billing in the results lists was inevitable. So even as the spamdexers drop to the bottom, we may watch the corporate biggies rise to the top.Another deflation of the myth of the Net as a great communications equalizer. Hopefully not all the search engines will sell out and sacrifice accuracy to a marketing mentality. If they do, I'll ... I'll ... I'll jump up and down and go get pierced.***Sites in My SightsFor more information about how the various Web indexes rank their results, or almost any other bit of trivia about their operation, pay a visit to Search engine Watch (

Enjoy this piece?

… then let us make a small request. AlterNet’s journalists work tirelessly to counter the traditional corporate media narrative. We’re here seven days a week, 365 days a year. And we’re proud to say that we’ve been bringing you the real, unfiltered news for 20 years—longer than any other progressive news site on the Internet.

It’s through the generosity of our supporters that we’re able to share with you all the underreported news you need to know. Independent journalism is increasingly imperiled; ads alone can’t pay our bills. AlterNet counts on readers like you to support our coverage. Did you enjoy content from David Cay Johnston, Common Dreams, Raw Story and Robert Reich? Opinion from Salon and Jim Hightower? Analysis by The Conversation? Then join the hundreds of readers who have supported AlterNet this year.

Every reader contribution, whatever the amount, makes a tremendous difference. Help ensure AlterNet remains independent long into the future. Support progressive journalism with a one-time contribution to AlterNet, or click here to become a subscriber. Thank you. Click here to donate by check.