Unlocking history at State

The State Department's archives include more than 25 million records of telegram e-mail messages to and from the department's field offices around the world since 1973. The content ranges from the mundane — a 1975 record of the embassy in Suriname receiving a new Dodge truck was an example shown during a recent demonstration — to exotic, top-secret, we'd-tell-you-but-we'd-have-to-kill-you stuff.

The department's challenge is separating the wheat from the chaff when looking for specific items in the huge database. The department's Office of Information Resources Management Programs and Services (IPS) is charged with providing information sought under the Freedom of Information Act and with determining when documents can be declassified and released to the public.

The professional searchers who look for data in the State Archiving System use an advanced records-searching capability that helps them quickly find relevant information.

The State searchers use Verity Inc.'s Information Server software, with a sophisticated interface to perform complex searches. "Anything you read in the Washington Post or New York Times or see on CNN that is attributed to the State Department, chances are, somewhere in this department, we are researching that data," said an IPS team member, who spoke on condition of anonymity.

The department also maintains a publicly accessible database (foia.state.gov) that uses the same Verity search technology, but through a simplified interface to accommodate computer novices and people who speak little English.

Before the adoption of the Verity search software, one could search the telegram records only by their assigned number, the subject line, and the Traffic Analysis Geography or Subject information. Now, the entire database is searchable by text, date, sender or recipient, classification ("top secret," for example) and many other parameters.

The system returns results quickly. During the demonstration, a search for information about Suriname from 1975 was quickly narrowed in scope to a half- million records. A few seconds later, the search was narrowed further and the results were in: It seems that as pleased as the embassy workers were to have a new truck, they really wished Dodge would hurry up and ship an air conditioner for it as soon as possible.