Lawmakers: Make agency Web sites more open to search engines

Many federal agency Web sites including the Federal Business Opportunities portal, the Federal Procurement Data System, the Library of Congress’ online catalog and about 2,000 others cannot be indexed by commercial search engines, and therefore information is not easily accessible. Lawmakers and open government advocates want to fix this problem. Joseph Lieberman (I-Conn.), chairman of the Senate Homeland Security and Governmental Affairs Committee, said earlier this week that his e-government reauthorization bill is the first piece of the solution. It requires agencies to review, report and test accessibility to search engines.“Our intention is for everything to the maximum extent possible to be easily available except for personal information and classified data,” Lieberman said during a hearing Tuesday on the E-Government Reauthorization Act. He added that the expectation of openness should be for the executive branch and the legislative branch. To that end, Lieberman and others introduced a bill to require the Congressional Research Service (CRS) to post its reports online. The bill was referred to the Senate Rules and Administration Committee. Lieberman said because CRS reports are not widely available, there is a knowledge gap between those willing to pay for them and those who aren’t. Rep. Christopher Shay (R-Conn.) introduced a similar bill in May, and it was referred to the House Administration Committee. The larger issue, however, is why more federal agencies are not using a standard called the sitemap protocol that would make their information more easily indexed by commercial search engines, said John Needham, Google’s manager for public-sector content partnerships. “The government produces a lot of information and these databases cannot be navigated by Web crawlers,” he told the committee during the hearing. “Agencies are concerned more about how information is presented than if users are finding it.” Karen Evans, the Office of Management and Budget’s administrator for e-government and information technology, said agencies must ensure that the information is presented in the correct context when the search engine crawls the Web site. “We need to balance between streamlining back-end databases and ensuring we are providing data that will not frustrate users,” she testified. Other experts disagreed with Evans’ rationale that context is a reason not to make information available to commercial search engines. Ari Schwartz, deputy director of the Center for Democracy and Technology, said people would figure out the context. Needham added that Google worked with Arizona government officials on adding the sitemap protocol to eight databases that originally were not intended to be publicly searchable. “We did it and it worked,” he said about adding the protocol to federal databases in similar situations. Evans said there is no policy against using the sitemap protocol and that her office would ensure agencies are not interpreting any policy the wrong way. “I think our policy supports the sitemap protocol,” Evans said after the hearing. “One way to get it more in government would be for the National Institute of Standards and Technology to create either a guidance or special publication about how to prepare and make information more accessible.”