How Open Data and Open-Source Approaches Can Help Federal Agencies Save Time and Lives

The COVID-19 pandemic has shed light on the urgent need for federal agencies to enhance data access to save time, money and lives. The past year and a half have also served as a reminder that data is ever-changing—whether it’s coming from a state’s centralized repository of health records or a military jet that produces one terabyte of data per hour—and needs to be readily available and up-to-date to enable key decision-makers to analyze it accordingly.

The rise and importance of artificial intelligence and machine learning in the public sector will enable agencies to achieve better, faster and more accurate decisions. Open-data and open-source approaches are also the most efficient way to ensure data is up to date and freely shared in today’s world.

Data and AI Is a Top Public Sector Priority with Challenges to Address

From the Biden administration launching the National AI Research Resource Task Force to the Department of Defense announcing a data strategy, it’s clear that the federal government is focused on modernizing its data analytics and warehousing capabilities. The government last year also announced $1 billion in new funding to invest in AI research and development. The increased focus on cloud, analytics and AI could lead to improved operational efficiencies. A Deloitte report estimates an annual $41 billion in savings for federal agencies from data-driven automation. 

However, the AI technology space is large, complex and constantly evolving, making it challenging to keep up with all possible solutions to determine what’s best for a specific application. The primary hurdle is within the data itself: Data silos often hinder the process of gleaning actionable insights and getting a unified view of company data. It can also be challenging to obtain and scale talent with the skills required to realize robust AI solutions. 

Approaches that Can Save Time and Lives

Open data or open-source approaches are paths federal agencies can take to address some of these innovation problems with much-needed agility and the most efficient way to ensure data is up to date.

A recent report shows that governments at all levels are seeing the benefits of embracing open solutions, including lowering costs, improving trust, increasing transparency and reducing vendor lock-in. An open-data approach also encourages collaboration, enabling federal agencies with similar problems to not have to reinvent the wheel to solve them. 

Barriers to Open-Source Adoption

While open-source solutions can benefit agencies in a variety of ways, a few barriers to successful adoption remain. Many agencies have to deal with outdated legacy systems—which are often disjointed, creating data siloes—or may lack the technical ability to implement open-source tools or run the software.  

Data security is another concern when evaluating open solutions. To reduce the risk of data exposure, it’s important only to publish the code, not its associated data. Implementing “supported” open-source software that has been scrutinized and supported by a vendor can also provide peace of mind when it comes to data security, as the government will be notified when a vulnerability is exposed and it will get patched in short order. 

Open-source software solutions enable federal agencies to uncover efficient and sustainable solutions—boosting collaboration, reducing costs and ultimately saving lives. Looking ahead, it's hard to imagine how federal agencies that need to share data with other government agencies, nations and the public could use closed source solutions where data is typically locked in a proprietary format. Prioritizing openness is the missing puzzle piece to the gridlocked information-sharing riddle.   

Howard Levenson is the general manager and area vice president of federal for Databricks.