Unfair grading?

The federal government received poor grades from Rep. Tom Davis (R-Va.), chairman of the House Government Reform Committee, on security score cards he released in February. The score cards tell lawmakers how well agencies have secured their electronic information and information systems. On Davis’ grading curve, the government earned an average score of 68 points out of a possible 100.

The federal government’s D-plus was a slight improvement from the D it earned in 2003 and a whole letter grade better than the F it got in 2002. Such low scores could suggest that the committee’s grading standards are too tough, but several security analysts disagree.

“That’s the stick they’re using to push people along,” said Paul Proctor, vice president for security and risk strategies at META Group, now owned by Gartner.

Another security expert expressed similar views.

“I think they’re tough enough, but I’m not sure they’re asking the right questions,” said Lynn McNulty, director of government affairs for the International Information Systems Security Certification Consortium. He was computer security director at the National Institute of Standards and Technology from 1988 to 1995.

McNulty said he would overhaul the security questionnaire and emphasize “more narratives and less bean counting.” He suggested asking what each agency considers to be its top five accomplishments of the past year and their importance.

Lawmakers say the annual score cards reflect the extent to which federal agencies are complying with security requirements that became law when the president signed the Federal Information Security Management Act (FISMA) of 2002.

To enforce agencies’ compliance, Office of Management and Budget officials prepare a FISMA questionnaire each year that asks basic security questions. But even those questions, Proctor said, are immensely challenging for agencies that have neglected security.

“When you start literally with nothing, getting to the level that FISMA requires to get a good grade requires astronomical effort,” he said.

Some security experts get hung up on issues such as encryption keys’ strength, Proctor said. But when grading Cabinet-level agencies on how well they protect electronic information, the organizations’ sheer size and complexity become overriding factors, he said.

Items on the FISMA questionnaire look simple, but they are not, given the size of most federal agencies, Proctor said. “One of the simplest questions they’re asking is, ‘Do you have an inventory of all your systems?’ ” he said. But for an agency such as the Energy Department, where contractors perform 93 percent of the work, that question is one of the most difficult to answer. Agencies also are penalized 10 points on their federal security score cards if, for example, they lack an inventory of at least

96 percent of their major information technology systems and applications.

FISMA grades are not as objective as people think, Proctor said. “When somebody says they’re doing really well, I don’t believe it,” he said.

And when large departments with more than 106,000 employees, such as the Agriculture Department, receive a failing grade every year, he said, “I also don’t put a lot of faith in that because small gains at that scale are substantial and need to be rewarded.”

Proctor said federal agencies’ approach to managing information security focuses on vulnerabilities rather than managing risk, which is where he thinks the emphasis should be eventually.

“I like to think they’re tackling the problem one [step] at a time,” he said. “Let’s get an inventory of our systems and attack our vulnerabilities, but that’s not the end game.”

Security analysts are not the only experts with strong opinions about the federal security score cards. In a recent telephone survey conducted by Telos, an IT services company, federal chief information security officers (CISOs) said the annual score cards are useful for focusing attention on federal computer security. But they gave the score card itself a C.

A majority of the 30 CISOs who responded to the survey said they do not fully understand all the questions on which they are graded. They also questioned why OMB officials have not used the federal budgeting process to punish agencies for poor security grades or reward those with superior grades.

The answer is that punishment at the end of a budget stick was never part of

FISMA, said Melissa Wojciak, staff director for the House Government Reform Committee. Through the act, lawmakers intended to promote better management of information security, Wojciak said. “We have a lot of confidence that as OMB officials are looking at budgets overall, they are working hard to eliminate duplicative spending that’s due to poor management,” she said.

For their part, Energy officials say that raising the department’s grade will require a transformation of their decentralized culture, which is largely research-oriented and based on open networks. “We continue to get an F,” said Rose Parkes, the department’s CIO, citing a fragmented governance structure.

As a first step toward improving information security, Energy officials are struggling to identify the systems that the department and its contractors own and to define a security perimeter. “We do not have 100 percent visibility into our inventory,” Parkes said. “We need an asset management system.”

The Interior Department fared slightly better than Energy this year. Interior officials raised their security grade from an F to a C-plus.

“I can’t tell you how much work that was,” said W. Hord Tipton, CIO at Interior. Still, it was difficult trying to impress Interior Secretary Gale Norton with a C-plus, he said.

But, like Tipton, other security experts who work with federal agencies say they can’t help but see the government’s D-plus as a glass half full rather than half empty. Glenn Schoonover, security solution specialist at Microsoft Federal, is one optimist.

“I think the key takeaways are that this year’s grade is better than last year’s grade, and that a lot of agencies that fared poorly in 2003 did improve,” he said.