Charting agency big data progress

Although much of the federal government has lagged behind local governments and the private sector in pursuing big data solutions in earnest, a number of agencies have made headway. Here are some examples.

1. Internal Revenue Service

Using big data analytics, the IRS was able to fix errors early in the tax-filing process for 2012, saving $100 million in erroneous claims. The agency also has a research lab developing data models for detecting fraud.

2. Bureau of Indian Affairs

Until recently, the agency’s law enforcement efforts on Indian reservations involved responding to crimes and other incidents on an individual basis, with the result that an agencywide picture was murky at best and only available from data given to the FBI. But a recently introduced pilot program connected four reservations and allowed them to share crime data, including historical and current reports. The entire agency has since adopted a similar model.

3. NASA: Big Data Challenge

In October, NASA launched something besides rockets: an online competition that asks participants to come up with novel ideas to make use of the large, unwieldy heterogeneous datasets kept by various federal agencies in the areas of health, energy and earth science. The submission period closed Oct. 13, and NASA is due to award three grand prizes of $750 each for the best ideas, with a 30 percent bonus for ideas that make use of live streaming data.

4. NASA: Johnson Space Center

NASA’s Johnson Space Center in Houston is the storehouse for one of the world’s largest still imagery and audiovisual collections. It consists of 4 million still shots and more than 81,000 hours of video, and the collection is growing at an exponential rate. The agency is able to keep track of it all through analog-to-digital migration and file compression and storage processes.

Main story: Managing big data

Related: How big data fights crime

5. National Institute of Biomedical Imaging and Bioengineering

Big data aided in an innovative risk management approach at the institute that involves using analytics software to assess entire research portfolios rather than individual grants. The approach helps determine the success of projects as measured by such salient features as number of publications and licensing agreements, to name a few, giving the institute an idea of where best to allocate research dollars.

6. Department of Veterans Affairs

VA maintains one of the largest datasets in the federal government; as of mid-2012, the agency had more than 175 data centers. The Veterans Health Administration’s Office of Analytics and Business Intelligence keeps track in real time of homeless veterans nationwide and shares that information with the Department of Housing and Urban Development in its efforts to combat homelessness.

7. Centers for Medicare and Medicaid Services

CMS has increasingly been using big data to help keep track of trends in its patients’ health status and to improve care.

8. National Archives and Records Administration

As of the beginning of 2012, NARA was in charge of 142 terabytes of data and 4 billion cubic feet of analog archives, including 400 million pages of classified information. To keep track of it all, the agency has constructed its Electronic Records Archives to be able to quickly adapt to users’ changing needs, resulting in a more rapid capability to archive and categorize new records.