DHS: Here’s Why Agencies Should Not Try to Boot Hackers by Themselves

Agency IT administrators should leave hackers in their systems until outside investigators are called in. All federal data centers should be shuttered.

These might seem like drastic recommendations -- but they come from the mouths of a top Department of Homeland Security director and a recently departed DHS senior official, respectively.

The compromise of secrets on 21.5 million national security personnel and their families in the care of the Office of Personnel Management exposed cyber shortcomings governmentwide that cannot be repaired overnight. Those failures include maintaining sensitive data on outdated machines and throwing out key evidence of a hack. 

"A lot of times, the system administrators are trying to clean up the issue" and they wipe their machines, leaving "no artifacts” for us to “even try find the adversary that was in their network and that is still in their network,” said Ann Barron-DiCamillo, director of the DHS U.S. Computer Emergency Readiness Team. “They didn't get them out."

She was discussing the aftermath of the OPM breach Wednesday afternoon at Nextgov's annual Nextgov Prime conference in Washington.  

In an effort to identify the intruder, IT staff sometimes will ping the IP address, the network location, associated with the unauthorized activity, Barron-DiCamillo, said, adding, "Well, you've just let the adversary know that you've seen them.”

Human instinct is to clean up the security issue pronto, but when doing so without advice from an incident response group, agencies are “losing their really important data from memory," she said. 

In an attempt to root out a problem they observe, IT staff actually can cover up other network behavior.

“They are losing the visibility of the actor across their network,” Barron-DiCamillo said. “As an incident responder, we get there and we've lost all visibility; [the agency has] written over the log data."

U.S. CERT strives to encourage agency IT personnel to contact investigators earlier in the response process so staff know what kind of precautions are OK to take. 

Part of the reason intruders are able to break in to begin with is that federal data centers, oft-home to decades-old mainframes, are inherently insecure, former DHS Chief Information Officer Richard Spires said at the same event.

Manufacturers often no longer provide fixes for security holes in the government’s machines.

So, agencies should outsource information for safekeeping to cloud companies certified under FedRAMP, Spires said. The Federal Risk and Authorization Management Program, as the process is officially called, is a series of inspections and monitoring procedures vendors must undergo on a continuous basis to sell their goods to government. 

"Most government agencies would be safer, would be better off, to move all of their traffic right now to FedRAMP-compliant providers, that are cloud service providers, than to keep it in-house because they are being held to a much higher standard than the government agencies," said Spires, now CEO of Resilient Network Systems. 

When Spires was at DHS, he co-led a governmentwide effort to downsize server farms in government called the Federal Data Center Consolidation Initiative.

As of May 2014, agencies reported a total of 9,658 data centers -- approximately 6,500 more than reported by the White House in 2011, according to a September 2014 Government Accountability Office Report. The paring back of centers began in 2010. 

“Right now, what we need to do is rapidly, rapidly retire a lot of these legacy systems that frankly have significant vulnerabilities," Spires said. "You can't patch. You can't fix."