FirstGov portal to soon offer better search organization

FirstGov.gov users will get search results clustered into groups of related hits through new technologies that the General Services Administration is implementing early next year.

Microsoft MSN Search and Vivisimo’s Clusty.com will power the official government portal under a $1.8 million contract announced today. Vivisimo and MSN will offer FirstGov users more organized and comprehensive results from an assortment of media outlets, image libraries and government Web pages.

The move indicates a shift in government acquisition practices toward mirroring industry trends, rather than playing catch-up. FirstGov had been using search supplier Fast Search and Transfer since 2002. When government officials decided they wanted FirstGov to adopt more advanced, user-friendly search capabilities, they awarded a new contract in fewer than three months.

“Since we first awarded that [Fast Search] contract in 2002, search has changed a lot," said MJ Pizzella, associate administrator of GSA's Office of Citizen Services and Communications, which oversees FirstGov. "We try to stay on the cutting edge, as best we can, considering the restraints on government with budgets."

The contract is part of an $18 million, five-year blanket purchase agreement that FirstGov set up to acquire future search capabilities. The BPA enables officials to award a new contract every year to a host of vendors, including Vivisimo, Fast Search and Gigablast. Well-known search companies Google and Yahoo did not bid, government officials said.

“Next year, when the renewal date comes up, we could have the option to look at any other company that is under the BPA," Pizzella said. "This technology is moving so fast that if you lock yourself in, you can’t move as fast as technology.”

Pizzella said that the government "is often woefully behind technology. Hopefully, this will allow the government to move in real time with the pace of technology.”

The new search engine will be three times as large and half as costly as FirstGov's current search capabilities. Now, FirstGov indexes about 8 million federal government Web pages. Vivisimo and MSN will comb through about 24 million pages from federal, state, local, tribal and territorial government Internet sources. For the first time, FirstGov will retrieve images and news articles from the mainstream press. In the same search, the engine will also scour government images and official agency news releases.

FirstGov will query the multiple databases and Web pages, then list results grouped by subject matter. This feature is possible because of two next-generation search enhancements commercial Web sites, metasearch and clustering use. Metasearch digs through multiple databases, search engines and the Web to answer each query. Clustering clumps search results together based on textual and linguistic similarities. The users see subject-matter hyperlinks, which they can click on to get subsets of results. Clustering lets users see results that would otherwise be buried at the end of ranked lists.

For instance, a search of “nursing jobs” might return one cluster of job opportunities listed on USAJobs, the federal government’s official job database, and a separate group of links to articles on nursing culled from other government sites.

"We’re going to build the best government search portal in the world," said Raul Valdes-Perez, co-founder of Vivisimo and a computer science adjunct associate professor at Carnegie Mellon University.

Clusty was made possible by a government grant from the National Science Foundation. FirstGov marks Valdes-Perez’s most public thank you to the federal government, after working with the defense and intelligence organizations, the Energy Information Administration, and the Social Security Administration.

“We think it’s a great payback to the taxpayer,” he said.