Home
Archive
Columnists
Video
Blogs
Discuss
About
Search
Donate
Advertise
Advertisement
Advertisement
Advertisement
Advertisement
Register to Vote: Rock the Vote, powered by Working Assets Wireless
Advertisement
  • AlterNetYour turn

Support AlterNet
Do you value the information you're getting from AlterNet? Please show your support with a tax-deductible donation.


Feedback
Tell us how we're doing.

The Government's Massive Data Sweep

By Mark Clayton, Christian Science Monitor. Posted May 15, 2006.


A 'dataveillance' program within the Department of Homeland Security is sifting through millions of online records to note user patterns and possible links to terrorism.
051506_story
The Government's Massive Data Sweep

Share and save this post:
Digg iconDelicious iconReddit iconFark iconYahoo! iconNewsvine! iconFacebook iconNewsTrust icon

More stories by Mark Clayton

Get AlterNet in
your mailbox!

 
Advertisement

Editor's Note: The following article was published in the Christian Science Monitor on Feb. 9, 2006. We think it is even more relevant today, amid revelations that the NSA has been routinely collecting the telephone call records of millions of Americans.

The US government is developing a massive computer system that can collect huge amounts of data and, by linking far-flung information from blogs and e-mail to government records and intelligence reports, search for patterns of terrorist activity.

The system -- parts of which are operational, parts of which are still under development -- is already credited with helping to foil some plots. It is the federal government's latest attempt to use broad data-collection and powerful analysis in the fight against terrorism. But by delving deeply into the digital minutiae of American life, the program is also raising concerns that the government is intruding too deeply into citizens' privacy.

"We don't realize that, as we live our lives and make little choices, like buying groceries, buying on Amazon, Googling, we're leaving traces everywhere," says Lee Tien, a staff attorney with the Electronic Frontier Foundation. "We have an attitude that no one will connect all those dots. But these programs are about connecting those dots -- analyzing and aggregating them -- in a way that we haven't thought about. It's one of the underlying fundamental issues we have yet to come to grips with."

The core of this effort is a little-known system called Analysis, Dissemination, Visualization, Insight, and Semantic Enhancement (ADVISE). Only a few public documents mention it. ADVISE is a research and development program within the Department of Homeland Security (DHS), part of its three-year-old "Threat and Vulnerability, Testing and Assessment" portfolio. The TVTA received nearly $50 million in federal funding this year.

DHS officials are circumspect when talking about ADVISE. "I've heard of it," says Peter Sand, director of privacy technology. "I don't know the actual status right now. But if it's a system that's been discussed, then it's something we're involved in at some level."

Data-mining is a key technology

A major part of ADVISE involves data-mining -- or "dataveillance," as some call it. It means sifting through data to look for patterns. If a supermarket finds that customers who buy cider also tend to buy fresh-baked bread, it might group the two together. To prevent fraud, credit-card issuers use data-mining to look for patterns of suspicious activity.

What sets ADVISE apart is its scope. It would collect a vast array of corporate and public online information -- from financial records to CNN news stories -- and cross-reference it against US intelligence and law-enforcement records. The system would then store it as "entities" -- linked data about people, places, things, organizations, and events, according to a report summarizing a 2004 DHS conference in Alexandria, Va. The storage requirements alone are huge -- enough to retain information about 1 quadrillion entities, the report estimated. If each entity were a penny, they would collectively form a cube a half-mile high -- roughly double the height of the Empire State Building.

But ADVISE and related DHS technologies aim to do much more, according to Joseph Kielman, manager of the TVTA portfolio. The key is not merely to identify terrorists, or sift for key words, but to identify critical patterns in data that illumine their motives and intentions, he wrote in a presentation at a November conference in Richland, Wash.

For example: Is a burst of Internet traffic between a few people the plotting of terrorists, or just bloggers arguing? ADVISE algorithms would try to determine that before flagging the data pattern for a human analyst's review.

At least a few pieces of ADVISE are already operational. Consider Starlight, which along with other "visualization" software tools can give human analysts a graphical view of data. Viewing data in this way could reveal patterns not obvious in text or number form. Understanding the relationships among people, organizations, places, and things -- using social-behavior analysis and other techniques -- is essential to going beyond mere data-mining to comprehensive "knowledge discovery in databases," Dr. Kielman wrote in his November report. He declined to be interviewed for this article.


Digg!

Mark Clayton is a staff writer for the Christian Science Monitor.

Liked this story? Get top stories in your inbox each week from AlterNet! Sign up now »


Advertisement

 

Comments Turn comments off sitewide Give us feedback »
Comments closed.
The comments for this story have been closed. Thank you to everyone who participated.
View:
How do we really know which purpose they have in mind?
Posted by: dainin on May 15, 2006 12:35 PM   
Current rating: Not yet rated    [1 = poor; 5 = excellent]
Ever notice how the activities the government would engage in to protect us from terrorism are the same activities the government would engage in to protect themselves from us?

[« Reply to this comment] [Post a new comment »] [Rate this comment: 1 - 2 - 3 - 4 - 5]

another scam
Posted by: rsaxto on May 16, 2006 5:09 AM   
Current rating: Not yet rated    [1 = poor; 5 = excellent]
This whole data mining area is just another scam by corporations to get big bucks out of ordinary folks pockets while making a few exectutives rich in the process. The real terrorists are sitting in the White House, etc. Data tap them and we will have everything we need to impeach their asses.

[« Reply to this comment] [Post a new comment »] [Rate this comment: 1 - 2 - 3 - 4 - 5]

» RE: another scam Posted by: Jimbo
It's already helped stop terrorism, we just can't cite examples cuz it's classified...
Posted by: WarHippy on May 16, 2006 4:00 PM   
Current rating: Not yet rated    [1 = poor; 5 = excellent]
Yeah, Right, and you're a member of a government agency sorta like.........NSA?? So we should believe you, right? If every phone is hooked up to that system, that just means they don't need a wire-tap, all they gotta do is hook up a recorder to it and they can listen to the latest gossip in whatever household they care to. Give the government an inch, they'll take......IRAQ

[« Reply to this comment] [Post a new comment »] [Rate this comment: 1 - 2 - 3 - 4 - 5]

The government is more dangerous than the al Qaada
Posted by: robchapman on May 16, 2006 5:45 PM   
Current rating: Not yet rated    [1 = poor; 5 = excellent]
The Bush Administration's "security" efforts present a greater long term threat than al Qaada. The Bush Administration keep bringing up the specter of an attack leaving thousands dead in a major American city.
That has already happened on 9/11/01. It happened despite the FBI security check for flight school clearance linking at least six of the nineteen hi-jackers to groups engaged in hi-jacking aircraft. It happened despite warnings from at least a half dozen foreign security agencies that an assault was imminent. It happened despite an investigation from the FBI field office in Minneapolis and their warning to higher ups that the operation was in the works.
The Bush Administration simply failed to use the tools available to them to stop the attack. This failure should not used as an excuse for them to take more power and violate our rights.

Robert Chapman
Lansing, New York

[« Reply to this comment] [Post a new comment »] [Rate this comment: 1 - 2 - 3 - 4 - 5]

What is real liberty?
Posted by: monkeywrench on May 16, 2006 6:13 PM   
Current rating: Not yet rated    [1 = poor; 5 = excellent]
Tune in, get smart, drop out – and go back to old technology, or no technology. Deny them your information as much as possible: use cash, buy local, avoid the WalMart trap, buy only what you need. And learn, learn, learn how your government has sold you (and everyone else) out. "The price of liberty is eternal vigilance." That's us, folks. . .

[« Reply to this comment] [Post a new comment »] [Rate this comment: 1 - 2 - 3 - 4 - 5]

» RE: What is real liberty? Posted by: peacefulaim