Intelligence community wrestles with Web 2.0 tools for information sharing

Where others see the colors orange and red when it comes to homeland security threats, Chris Rasmussen sees purple.

Rasmussen is a social-software knowledge manager at the National Geospatial-Intelligence Agency. But he prefers to call himself a purple intelligence and mashup evangelist, pointing to the fact that purple is the color that results from mixing multiple points of the spectrum.

Purple is an apt symbol for combining the expertise of organizations working to help prevent future attacks, he said.

Rasmussen has seen purple power in action through countless little success stories accomplished via Intellipedia, the information-sharing wiki that serves intelligence agencies, the military and the State Department. “All the time, people are connecting with others [who] they didn’t know worked on the same issue six feet down the hall,” he said.

Connecting the dots, more formally known as information discoverability, is gaining increasing attention from homeland security officials and experts in their ongoing attempt to corral anti-terrorism information that resides across federal, state and local jurisdictions.

In January, the departing director of national intelligence issued Intelligence Community Directive 501, which gave intelligence personnel a “responsibility to discover” information believed to be relevant to their work, along with a corresponding “responsibility to request” information they have discovered.

The directive defined discovery as the act of obtaining knowledge of the existence, but not necessarily the content, of information collected or analysis produced by any intelligence community element.

Two months later, the bipartisan Markle Foundation published a report that reaffirmed “discoverability” as the first step in any effective information-sharing system.

“Solving discoverability simplifies solving information sharing,” said Jeff Jonas, an IBM distinguished engineer and a member of the Markle Task Force on National Security in the Information Age.

But despite these high-profile mandates, challenges call into question the feasibility of discovery tools and techniques for solving data-sharing problems that span agencies, jurisdictions and cultural boundaries. Some say the technology isn’t even the hard part.

“It’s difficult to make information discoverable, useful, and, at the same time, make sure it complies with all of the other requirements around privacy and security,” said Andre Etherly, chief solutions partner at systems integrator Keane.

Managing information

One of the biggest discovery roadblocks is the mountain of data that federal, state and local authorities collect.

“When we speak about discoverability, we’re speaking about the ability to determine who to ask for a certain piece of data,” Jonas said.

That’s why some information systems architects want to abandon what has been the traditional answer: a giant data warehouse. Officials create these massive databases by merging copies of the central databases maintained by intelligence and law enforcement agencies such as the CIA and FBI.

“That model isn’t productive because you have so much data to move and the more copies, the more data to protect and keep current,” Jonas said.

Data warehousing is also problematic because it requires officials to know before setting up the system what information they might want that others have and vice versa.

Instead, the Markle Foundation and others advocate indexes built with metadata, the IT industry’s electronic equivalent of a library card catalog. In its report, the foundation said indexes not only help people find information among individual databases but also protect privacy and security because only the data’s location in the database is revealed.

The actual information stays safely in its local database until someone allows its release to an authenticated requester. In addition, anonymization techniques can hide personal information from the indexing technology to guard against identity disclosures.

The federal government’s Information Sharing Environment is one of the first to use a metadata index technique for its Suspicious Activity Reports systems, which taps into fusion center databases across the country.

“There’s no big database in the sky,” said Paul Wormeli, executive director of the IJIS Institute, a nonprofit services and training group that specializes in information-sharing systems. He said the Suspicious Activity Reports system allows investigators to query certain databases about particular topics. Moreover, the database owners can control what gets discovered and disclosed.

Although technology exists for easily creating metadata subject labels, implementing the tags is difficult. “You have to find all this data wherever it resides and then begin tagging it,” Etherly said. “That’s not an easy process.”

Complicating the task, people who create the tags often disagree on what the metadata should include. “Whoever makes up the taxonomy has tremendous power,” Rasmussen said. “Whether Pluto is a planet or not is a big deal.”

Communications breakdowns could plague homeland security efforts even for seemingly fundamental definitions, such as who is a suspect.

For example, within the FBI, the term “suspect” has a well-defined legal meaning. In other parts of the intelligence community, finding a name written on the bottom of a rock in a cave in Afghanistan might be enough to say that person is a suspected terrorist. “If one of the three-letter agencies shared this information with the FBI, they could both come away with a false conclusion,” Etherly said.

And it’s not enough for authorities to use pointers, indexes and portals to discover information if they can’t quickly obtain cross-agency clearances to view the data.

Wormeli called for federated identity management to streamline such clearances. For example, a New York City police detective who has been vetted by his own department could use his credentials as verification to allow access to certain information held by some of the 16 federal intelligence agencies. Such a system would require a nationwide effort to create these federated identity management capabilities.

“We’re nowhere near doing that,” Wormeli said.

Nevertheless, he noted reasons for optimism. Federal agencies and a majority of states now use the National Information Exchange Model for justice-related purposes. NIEM architects created a methodology for Information Exchange Package Documentation that makes it easier for two agencies to exchange files, such as arrest reports.

“With that specification, we can respect the business needs of the two agencies and much greater standardization for how computers can talk to each other,” Wormeli said.

Social-networking innovations

Some government 2.0 proponents said public agencies don’t need to spend so much time developing methodologies and standards for information discoverability — the innovations are happening independently within Web communities. Discoverability challenges, in turn, can be solved with secure versions of wikis, blogs and social-networking tools such as Intellipedia.

“That’s so clearly the ideal world in my mind that I don’t understand why there should be any alternative,” said Matthew Burton, a technology consultant in intelligence. “Culture and money [are] probably two issues, but from a technical level, there is no reason not to do it.”

Social networking can be a driving force in making homeland security information discoverable based on Web 2.0 tools' track record of bringing together public communities separated by geographic locations or intellectual concepts, said Andrea Baker, director of Enterprise 2.0 at Navstar, an IT and telecommunications contractor. Such tools create a central platform where the community can connect.

“From there people can say, ‘I have this [piece of] information,’” Baker said.

Jack Holt, senior strategist of emerging media at the Defense Media Activity, the Defense Department’s consolidated public affairs organization, said social networks showed their discoverability potential in 2006 when bloggers raised questions that forced the Reuters news service to retract enhanced photos that exaggerated the destruction caused by an Israeli bombing raid in Beirut.

“That information came forward from the social-network society" and the technology that allows that level of collaboration, Holt said.

Intellipedia illustrates the potential and challenges of Web 2.0 for homeland security discovery efforts. It has become active enough that, on some days, it might experience 5,000 edits, said Rasmussen, an early and regular contributor to the wiki since it launched three years ago.

But the wiki also has experienced some flameouts, such as when the Office of the Director of National Intelligence tried but failed to create a national intelligence estimate about Nigeria. The problem wasn’t that users didn’t discover relevant information. Rasmussen said they didn’t fully trust what they found.

“We post a caveat on every social tool we have saying that this is not ‘finished’ intelligence,” he said. “That says it’s not" as good as what’s in a formal report.

He explained that contributors think out loud in Intellipedia, but when it’s time to get serious, analysts vet the information through a traditional system run by one of the agencies, resulting in an official report.

“And that’s the problem: Which one do you pick — the CIA’s, the DIA’s, the NGA’s, the FBI’s?” Rasmussen said.

The result is more information overload, which creates more discoverability roadblocks. There might be bits of valuable information spread among multiple reports from different agencies, but it can be difficult to find amid all the redundant information. A wiki can reduce the number of those reports and be the single, authoritative tool that everyone uses.

There is also no reason why a wiki can’t also incorporate a system that vets information as people collaborate, rather than relying on the traditional hierarchical review processes, Rasmussen said.

Even so, some skeptics said wikis alone aren’t enough. Analysts need other ways to judge the reliability of the information appearing on the sites.

“If the pedigree of that information is not conveyed as the information goes forward and gets shared, then people using the information don’t have a good way to assess the quality and the risk of using that information,” Etherly said.

One possible answer is the Pedigree Management and Assessment Framework, originally created for the Air Force. It allows an agency to assess over time the reliability and quality of information. It has a self-learning quality, so that if nine out of 10 things a particular informant says turn out to be false, then officials can conclude that a high risk comes with using information from that person.

Nevertheless, social-networking tools have a big advantage over custom-built information-sharing systems for agencies looking to simplify discoverability: They can be very easy to use.

“People want to use things like Google that are just that simple and tell you what you need to know and don’t require a 200-page manual to understand," Burton said.