The National Institute of Standards and Technology released on Monday guidelines for agencies to test how well their computer systems fend off cyberattacks. Many analysts say the recommendations could be the first step in fixing one of the more serious flaws in government's approach to network security.

Comment on this article in The Forum.NIST's instructions detail how agencies can assess their procedures for testing security controls for information systems. The release is the latest addition to the NIST Special Publication 800 series, which offers research and guidelines to help agencies implement the 2002 Federal Information Security Management Act. SP 800-53A explains how to evaluate a network's security controls, risk management processes, and security strengths and weaknesses of information systems that support missions and applications.

The guidance notes the need for additional assessment for moderate-impact and high-impact information systems. The document is a companion to the revised version of SP 800-53, Recommended Security Controls for Federal Information Systems, which was released in December 2007.

"This is a huge step forward," said Tom Kellermann, vice president of security awareness at Core Security Technologies and former senior data risk management specialist for the World Bank treasury security team. "In essence, [the guidance] made Pinocchio come alive. The focus of SP 800-53 was just about compliance -- a checklist exercise. No one learned to truly assess effectiveness. But now, agencies have the methods to assess security controls and build a cohesive policy and security network."

FISMA, overseen by the Office of Management and Budget, requires agencies to define and inventory their IT systems, determine the sensitivity of information stored on them, find potential vulnerabilities in the systems, and deploy security controls. Agencies undergo annual audits to certify and accredit systems. But some industry analysts and members of Congress question whether certification and accreditation are true metrics by which to measure the strength of information security because agencies have no means to determine how well they can defend against cyberattacks.

The guidance helps fill the gaps left by FISMA, most notably through an appendix that details ways to conduct penetration testing to find weaknesses in systems, Kellermann said.

"Penetration testing has always been a dark magic, which some knew how to do and some didn't, and which some were afraid to do in fear that it might damage systems or eliminate plausible deniability -- the ability to say they didn't know better," Kellermann said. "Instead, agencies did glorified vulnerability scans. It's like comparing a physical exam from a doctor to an MRI -- one pokes and prods and maybe recognizes a problem, while the other can actually identify the [cause]."

Penetration tests are controlled attempts to breach security controls, using the latest, most sophisticated forms of attacks with appropriate hardware and software tools. According to SP 800-53A, the tests should offer proof of actual risks and the level of effort needed to harm an agency's operations and assets, reveal incorrect system configurations, assess the trusted relationships between organizations, look for architectural weaknesses, and produce a detailed log of activities performed. They should be conducted at a minimum on recently developed systems, after important changes are made to the systems (whether internally or as the result of a breach) and when a new cyberattack method is discovered by industry or government.

"Instead of being an appendix, penetration tests need to be actual security controls, because it's the nature of the beast that only proactive folks will conduct these assessments," Kellermann said. "But it's important we move ahead, and this is a huge step."