Monitoring Social Media

by Noah Shachtman – Wired Magazine

America’s spy agencies want to read your blog posts, keep track of your Twitter updates — even check out your book reviews on Amazon.

In-Q-Tel, the investment arm of the CIA and the wider intelligence community, is putting cash into Visible Technologies, a software firm that specializes in monitoring social media. It’s part of a larger movement within the spy services to get better at using ”open source intelligence” — information that’s publicly available, but often hidden in the flood of TV shows, newspaper articles, blog posts, online videos and radio reports generated every day.

Visible crawls over half a million web 2.0 sites a day, scraping more than a million posts and conversations taking place on blogs, online forums, Flickr, YouTube, Twitter and Amazon. (It doesn’t touch closed social networks, like Facebook, at the moment.) Customers get customized, real-time feeds of what’s being said on these sites, based on a series of keywords.

“That’s kind of the basic step — get in and monitor,” says company senior vice president Blake Cahill.

Then Visible “scores” each post, labeling it as positive or negative, mixed or neutral. It examines how influential a conversation or an author is. (”Trying to determine who really matters,” as Cahill puts it.) Finally, Visible gives users a chance to tag posts, forward them to colleagues and allow them to response through a web interface.

In-Q-Tel says it wants Visible to keep track of foreign social media, and give spooks “early-warning detection on how issues are playing internationally,” spokesperson Donald Tighe tells Danger Room.

Of course, such a tool can also be pointed inward, at domestic bloggers or tweeters. Visible already keeps tabs on web 2.0 sites for Dell, AT&T and Verizon. For Microsoft, the company is monitoring the buzz on its Windows 7 rollout. For Spam-maker Hormel, Visible is tracking animal-right activists’ online campaigns against the company.

“Anything that is out in the open is fair game for collection,” says Steven Aftergood, who tracks intelligence issues at the Federation of American Scientists. But “even if information is openly gathered by intelligence agencies it would still be problematic if it were used for unauthorized domestic investigations or operations. Intelligence agencies or employees might be tempted to use the tools at their disposal to compile information on political figures, critics, journalists or others, and to exploit such information for political advantage. That is not permissible even if all of the information in question is technically ‘open source.’”