Experts Question Value of Federal Cybersecurity Data Capture Mandate

New federal requirements to log every data packet that crosses agency networks—called packet capture or PCAP—have raised concerns among cybersecurity experts in and out of government, who say the new rule is unclear, resource intensive and of little value during a real-world breach investigation.

The PCAP requirements were a direct result of a series of breaches detected in late 2020, including the SUNBURST incident that triggered a governmentwide remediation effort and a series of new mandates for agencies.

By August 2021, the Office of Management and Budget issued a memo on “Improving the Federal Government’s Investigative and Remediation Capabilities Related to Cybersecurity Incidents,” which included new requirements for agencies to create and maintain certain data for use in post-breach forensics. Those data elements included “full packet capture,” as well as more simple cybersecurity logs.

That requirement was followed up by new guidance last month from the National Archives and Records Administration—its first update to cybersecurity records rules since 2014—that formally established the retention rules: 30 months for cybersecurity logs and 72 hours for PCAP data.

But the broad nature of the PCAP requirement has proven confusing for some federal cybersecurity officials who have questioned the value of doing PCAP on modern networks and the cost associated with keeping the large data sets. The specifics of the policy—particularly the short 72-hour window—further reduce what little value packet capture offers for forensic purposes.

For instance, one federal cybersecurity official, who asked to remain anonymous as they were not cleared by their agency to speak publicly, said the 2021 memo and NARA update aren’t enough to help agencies define their PCAP procedures and don’t seem to offer leeway for those trying to build a similar cybersecurity posture without PCAPs.

Robert Graham, a cybersecurity consultant and PCAP expert—and prolific “random guy on the internet”—agreed the biggest issue with the government’s PCAP policy is the ambiguity.

“I don’t understand it enough to make a judgment,” he told Nextgov. “One interpretation seems OK; another interpretation is on the wacky side.”

Asked for clarification, an OMB official, speaking on background, would only confirm the government’s official policy: “Storing PCAPs for 72 hours is required by M-21-31; CISA and OMB work with agencies to provide technical assistance on log retention and management.”

A spokesperson for the Cybersecurity and Infrastructure Security Agency declined to comment on how agencies should implement the requirement and referred Nextgov back to NARA on the records retention rules.

But the implementation will only be effective if agencies set the PCAP up on the right parts of the network, Graham said.

“There’s no spot in the network where you can tap in and see all the traffic. The network is sort of a mesh—it’s why we call it a network—and so you have to tap into lots of points. Some of those points will see lots of traffic—more than you want to capture. Other parts will see just the traffic that you want to capture,” he said. “Where do they actually mean that these PCAPs should happen?”

For instance, a link between main servers and a backup server collecting data purely for redundancy doesn’t need to have packet capture, as that PCAP data would be redundant and expensive to store.

“Does it mean network traffic going to the internet? Or does it mean all network traffic anywhere, including between two servers?” he said. “Two servers might exchange backups, for example, which would quickly overload things. I can't imagine that's what they intend.”

The problem gets exacerbated when agencies encrypt network traffic, as they are required to do under OMB’s January 2022 Zero Trust mandate. Cybersecurity incident investigators can glean some data from an encrypted packet—such as its size—but get little additional information without decrypting the packet, which is not possible under most security setups.

“Given that most traffic is encrypted without possibility of decryption, there's little added value over flow logs. Passive collection of the certificates would help, along with passive DNS, but beyond that, I can't imagine what benefit it would have,” Graham said.

With little added value and high storage expense, some have questioned whether PCAP is truly worthwhile.

Graham agreed that people worried about compliance—especially those in government—might tend toward the “wacky side” to be safe, either going overboard and spending far more money and resources than is needed or doing just enough packet capture to be in compliance with little to no effect on the agency’s cybersecurity posture.

Even if there are useful insights in the PCAP data, it would likely be a rare occurrence for PCAP data to be valuable within a 72-hour window.

Various sources offer different statistics on how long an average breach goes unnoticed.

According to the IBM Cost of a Data Breach Report 2022, the average time for an organization to detect a breach is more than 200 days, varying little over the last six years.

Data from cybersecurity firms Mandiant and Sophos show a far rosier picture, with mean dwell times—defined as the “number of days an attacker is present in a victim environment before they are detected”—of 24 days and 15 days, according to data in their respective 2021 reports.

The Sophos report notes the dwell time is much higher for incidents that did not include ransomware—for which attackers generally send ransom notes informing the victims of the breach—more than doubling to a mean of 34 days.

While those timelines are much shorter than 200 days, they are still well outside the 72-hour window prescribed by OMB and NARA.

“The breach happened long before 72 hours,” Graham agreed. However, he noted that, even if a breach were detected within the 72-hour window and the unique PCAP were potentially useful, it will still be difficult for agencies to make good forensic use of the PCAP data due to one of the most persistent issues in federal IT management: having enough of the right kind of skilled cybersecurity employees.

PCAPs can be useful in determining whether a potential incident is a true concern or just a false-positive, Graham said, but “such analysis of PCAPs is a specialized skill that the average [security operations center] worker doesn't have. If you’ve got people on your team that do it well, then you’ll probably want to do it. But it’s really hard to hire for it. So, most people usually end up taking these PCAPs and never doing anything with them.”

Editor's Note: This article has been updated to refer to a specific breach by the name of the attack instead of the company involved.