Intell agencies adapt data architecture model

Analysts would need years to sort through every piece of data collected clandestinely in a single day, intelligence officials say. But to prevent terrorist attacks, even waiting a few days can be too long.

"The problem is finding the stuff that's relevant in a sea of data," said Bryan Aucoin, lead enterprise architect for the U.S. intelligence community. "You have to get the dots, you have to connect the dots, and then you have to tell people about the dots."

Aucoin is leading an effort to create an enterprise architecture so that the 15 federal agencies that collect and analyze intelligence can exchange data. "Putting every piece of information we have in one database is not going to work," he said at a Nov. 9 meeting of data architects in Washington, D.C.

Spy agency officials must create a meaningful way of exchanging information, and the recently released data reference model portion of the federal enterprise architecture can help, Aucoin said. "What the [model] can do is provide the context" for data, he said.

Working in phases, intelligence agency data architects will modify the data reference model to meet their needs. Their version should be complete by December 2005, they said.

Some standards for Extensible Markup Language counterterrorism metadata will be easier to implement than others, said Bill Dawson, deputy CIO for the intelligence community. As a first step, officials can adopt the Justice Department's Global Justice XML Data Model as a law enforcement metadata standard, Dawson said at a Nov. 1 breakfast briefing of the Bethesda chapter of AFCEA International. But other standards will be harder to assign, he added.

President Bush approved an executive order Aug. 27 that requires the creation of metadata standards for the widest possible dissemination of intelligence information. Officials must reach consensus on seven standards by Nov. 24, including an exchange standard for terrorist watch list data.

Intelligence community architects may change elements of their data model by expanding users' ability to make associations between the data content and context, said Michael Daconta, the Homeland Security Department's metadata program manager. "We will never be able to connect the dots until we get serious about standardizing the associations," he said.

The federal model also overlooks the exchange of unstructured data, such as images, text and other pieces of information that cannot be entered into predefined fields, Daconta said.

Additionally, the federal model doesn't address security concerns well enough, he said. "Clearly, we have to add a security context," he said. "If we're talking about terrorism, we have to be talking about security."

Setting up an architecture is not easy, but the results are worthwhile, Aucoin said. "You've got single sources of record data for any given subject," he said. "You've got integrated data from multiple sources; you've got data of known quality."