Government data sharing gets past the warm-up

The government’s open-data venture took some serious strides forward this year, but many thorny issues about how and what to share publicly remain.

Data.gov, the government’s flagship data accessibility site, surpassed the 300,000 mark in acquired datasets, having started with less than 50 holdings when it launched in 2009. State and local governments are also sharing a variety of information. For example, many municipalities now post data about the real-time location of subway trains and buses.

However, the data didn’t just sit around gathering dust. Practical and, in some cases, dramatic applications surfaced. In Haiti, emergency responders used tools such as OpenStreetMap to coordinate relief efforts. Closer to home, the National Oceanic and Atmospheric Administration used a variety of government data sources to populate its GeoPlatform site, an interactive map about the Deepwater Horizon oil spill and response efforts.

Despite those achievements, there’s a sense among open-data experts that the truly difficult challenges still lie ahead.

“We’ve seen this initial rush of information,” said Bill Allison, editorial director at the Sunlight Foundation, a nonprofit organization that pursues greater government openness. “But there is an awful lot of missing data.”

“I think that the low-hanging fruit has been picked,” added Tom Lee, director of the organization’s Sunlight Labs division. “Now it’s time for the hard part.”

Concerns about privacy and protecting the proprietary nature of some of the business data the government collects make releasing certain datasets more complicated. In addition, agencies often maintain data in formats that don’t lend themselves to rapid release, Allison said.

Other challenges include getting the word out on the available data and providing tools for ease of access.

“There’s still an issue of awareness,” said Ben Shneiderman, a computer science professor at the University of Maryland.

He said the number of people who tap the government's open data remains relatively small. That’s a view shared by Jane Fountain, a professor of political science and public policy and adjunct professor of computer science at the University of Massachusetts at Amherst.

Shneiderman, who focuses on information visualization and has worked on Recovery.gov and the Health and Human Services Department’s Community Health Data Initiative, said more work needs to be done to make journalists, scholars, researchers and politicians familiar with the data sources.

He also cited the need for appropriate tools to explore datasets in a comprehensible fashion. Once the province of academic research departments, data access tools are becoming more readily available, he added.

Along those lines, the Sunlight Foundation plans to create a series of National Data Apps that will let users more easily tap federal data. Allison said the foundation pursues apps in four areas: health care, banking/consumer finance, environment and education. The objective is to take government data that might exist in not the friendliest of formats and make it available on devices such as smart phones. The John S. and James L. Knight Foundation is funding the effort with a $1.2 million grant.

Meanwhile, Shneiderman said he believes open data might be one of the more enduring contributions of the Obama administration.

In 2011, making the most of the available data and pinpointing information that still needs unearthing will rank among the top open-data priorities. In addition to raising awareness of the data's availability, efforts will include building communities around specific datasets and user groups.

Read more of 6 ideas with some 2011 bounce or return to the 2011 Federal List.