Ex British intel chief offers lessons in mining social media

The government needs a systematic approach to mining social media for intelligence, instead of allowing erratic practices to emerge, said Sir David Omand, former chief of the U.K. Government Communications Headquarters, a British intelligence agency. Law enforcement and security agencies that work within a framework would have more legitimacy to effectively tap social networks, he added.

His arguments, written for a U.K. audience in a paper published April 24 by a London-based think-tank, offers lessons for the U.S. government. Federal agencies such as the Intelligence Advanced Research Projects Activity and the Defense Advanced Research Projects Agency are funding dozens of tools to interact with social media networks. The problem now is that the nascent technology may be advancing faster than legal frameworks being crafted to prevent abuse.

The analysis of social media can help governments study the spread of radical ideas and the intersections between criminals’ online and offline personas, the report noted. But a lack of legal and conceptual clarity around the use of data will trigger a backlash from privacy advocates. Agencies need to draw distinctions between collecting open-source, publicly accessible data and the interception of private personal information, the report notes.

So far, technology to turn large volumes of social media data into timely, decisive intelligence has fallen short, Omand said. Scotland Yard missed the opportunity to use social media to get a grasp on London’s August 2011 riots because “the surging social media output did not fit into the police’s formal process to evaluate reliability, did not therefore count as intelligence, and consequently was not acted on,” the report noted. See the full memo here.