How federal agencies can integrate artificial intelligence into records management

The way people communicate has evolved and diversified dramatically over the last two years. Gone are the days when email and phone calls were the primary forms of communication for many. In both personal and professional settings, people increasingly rely on messaging apps, social media and video conferencing.

Meanwhile, many federal agencies are just beginning to address digitizing their paper records. M-19-21 compels all federal agencies to make their records fully electronic by December 2022. Federal agencies’ difficulty managing their paper document stores paired with these new emerging data sources has resulted in major repercussions. For instance, many agencies are backlogged in responding to Freedom of Information Act requests, and Congress will sanction some of these agencies for their inability to respond to these basic requests.

My data has data!

While many agencies are only now starting to resolve their paper records challenges, the data landscape is rapidly shifting to newer mediums—from instant messages and Zoom transcripts to surveillance, closed circuit TV and body-cam videos—further complicating the digital data frontier. Federal agencies must capture, manage and glean insights from these new sources while digging out from under decades of paper records. This is a daunting task, but achieving it is essential to keep up with FOIA requests, maintain compliance with Congressional oversight and stay on top of litigation.

Cloud solutions provide the agility and scalability needed to do so while ensuring that agencies can take full advantage of tools like artificial intelligence that can help make sense of their data and use taxpayer resources efficiently. Here’s how agencies can think about tackling extracting valuable information from both their paper and digital resources to stay on top of their obligations.

How do we even start?

Artificial intelligence can streamline paper record management and provide valuable insights— including identifying what must be retained and what can be discarded—while also helping to make sense of the data in digital records. However, AI technology is only as good as it's been trained to be. It requires human input to be effective. Agencies must identify where relevant data resides, identify an appropriate AI technology that can work with diverse data types, train the tool with the correct data and then verify that the results are valid. Even the most powerful AI tools developed to date need to be carefully managed to work correctly.

This process may involve reconsidering agency collections requirements and strategies to account for new data types. For example, agencies should create a data retention plan for digital records based on information policies and individual device policies as well as regulations. This helps ensure the appropriate records are kept for technologies like AI to extract insights from.

Lean into other agencies and organizations

One solution for tackling this daunting challenge can be identifying a technology or consulting partner (or even collaborating with another agency) that has experience implementing effective AI data management solutions. While the AI procurement and implementation process can be daunting, many federal agencies are facing this challenge together. They can look to each other for support and learning

Partners must be able to keep pace with the agency’s needs and provide future-proof solutions so that agencies can avoid repeatedly having to go through the procurement, transition and onboarding cycles. One helpful step will be developing evaluation criteria during the market research phase to identify vendors who prioritize innovation and integration, and have strong past performance with other vendors. Because the procurement process tends to be long in the federal space, it’s important to think long term and find a partner that can scale with the agency’s needs for years to come.

Bells and whistles should come last

Once the agency works through these first two steps, they can begin migrating paper and digital-native records into a data lake in which they can be stored, sorted and searched in a uniform way. At this step, they should begin evaluating tools and processes that will make the act of searching and retrieval easy and accurate. Metadata is a critical aspect that makes this information searchable. For instance, scanned PDFs don’t have metadata that is easily sortable, so searching them is similarly slow and unhelpful to slogging through paper records.

Once the data has been roadmapped and stored, agencies can begin applying AI tools to glean insights from it and respond to FOIA requests, Congressional oversight and litigation. Here’s how a few key AI tools could come into play:

• Data visualization is a bird's eye view of an entire data lake. Tools with interactive visuals can help quickly sort through massive amounts of data by quickly providing insights into things like file type, custodians, conversation parties and more.
• Predictive coding sorts and tags documents much faster than a human can. Predictive coding technology amplifies an agency’s resources by learning from the decisions people have made to surface related data without the continued resource drain of manual review.
• Clustering groups documents by conceptual similarity and generates insights without user input. For instance, FOIA requests submitted to Immigration and Customs Enforcement tend to be on similar topics (border crossings, border retention, etc.). Identifying data clusters about those keywords decreases the number of documents ICE would need to review when responding to a request, while also ensuring that they don’t miss any relevant information.

The new data frontier may seem daunting, but if agencies take it step by step, future-proofed data management is possible. The benefits are bountiful: helping them remain compliant, be more efficient with taxpayer money, and fulfill their duties to serve citizens.

Angela Kovach is director of Government Solutions and Operations at Everlaw.