How Data Can Aid Farmers' Markets and Education Innovations

Two things that came across my desk this week show the potential of President Obama’s new open data policy but also its potential pitfalls.

That policy requires agencies to make all future data they collect open and machine readable by default. The hope is that entrepreneurs outside government will use those data to build nonprofit and for-profit businesses based on information the government is collecting anyway.

First, the Agriculture Department and the federal Chief Information Officers Council are highlighting a new application programming interface, or API, that will stream automatically updated USDA data about farmers markets straight to anyone who wants it.

The Farmers Market Directory is one of USDA’s most popular datasets, receiving nearly 2 million views per year, the agency said. The API built off those data will allow local organizations to easily build mobile apps and other tools for people seeking fresh produce in their communities.

The second item is a sources sought notice from the Education Department, which is looking for a vendor that can establish a common framework for data from the president’s Gaining Early Awareness and Readiness for Undergraduate Programs. The GEAR UP initiative “provides six-year grants to states and partnerships to provide services at high-poverty middle and high schools.”

Here’s a key nugget of the notice:

GEAR UP restricted-use data files: These files shall provide accurate and consistent data on at least 50,000 GEAR UP students representing at least 12 GEAR UP state grantees. In addition, the data files shall include appropriate outcome data on non-GEAR UP comparison students. The files shall allow approved researchers (with appropriate licenses to address privacy concerns) to perform analyses, such as rigorous evaluations, in order to better understand GEAR UP implementation and effectiveness. The restricted-use data files shall contain grantee implementation data consistent with the data framework and academic progress indicator data including data on high school achievement and completion, achievement on standardized assessments comparable across states, and postsecondary application, enrollment, persistence, and completion. Compiled data shall allow researchers to address study questions such as: o What are the relationships of specific GEAR UP activities relative to key academic progress indicators? o What is the relationship of GEAR UP students' academic progress relative to non-GEAR UP students of similar demographic characteristics?

In other words, even in the era of “big data” emanating from satellites, sensors and Google searches, collecting the small data that can be used to evaluate a government program is still an arduous job. And interpreting those data in a meaningful way requires a great deal of expertise.

That’s not to say the GEAR UP data -- in a sanitized form that’s scrubbed of any information identifying specific students -- won’t be a boon one day for teachers or parents researching classroom innovations. But opening the data up is only one part of what gets us from here to there.

Bonus: Government IT watchers should look closely at Google’s push for intuitive searching or what the company is calling “Star Trek search.” It will be interesting to see how this next generation of search might aid government, which has struggled mightily with getting information to the people looking for it without requiring them to navigate or understand the government’s complex bureaucracy.