Cyber eye: Cybersecurity report cards don’t make the grade

Congressional report cards on the government’s computer security have succeeded in focusing attention on the challenge of securing the nation’s information infrastructure. It’s easy to grab headlines when the government gets an overall grade of D+ and seven agencies get an F, as in the latest report card.But it is difficult to believe these grades are an accurate reflection of the government’s cybersecurity posture.The report cards have been given out annually for five years now by the House Government Reform Committee or one of its subcommittees. The most recent, which prompted hearings on the government’s poor security performance, showed marginal overall improvement, although two agencies that had received Cs for fiscal 2003 dropped to F in 2004.How are we to take these grades? Are we really to believe that a department as complex as Justice was able to improve its security enough to go from an F to a B in one year? Or that the Defense Department, which has received a D for the last two years, really is unable to adequately defend its networks and information resources?Outgoing Homeland Security Department CIO Steve Cooper testified before the Government Reform Committee that unrealistically high thresholds in the grading system mask the progress the department has made, resulting in a failing grade. He predicted the department would work itself up to only a D next year.This is not to say that Justice should not get credit for its progress, or that improvements are not needed at DOD or DHS. But the grades should be taken with a grain of salt.The problem is twofold:First, security is a complex job that does not lend itself to a coarse five-grade scale. The committee’s grades are based on a possible 100 points, awarded on the percentage of compliance in seven categories. The points are the same whether a department has a dozen IT systems or 1,000 systems. It does not take into account whether an agency has a policy of securing its mission-critical systems first. A point is a point, and if you lose more than 40 of them, you fail.Second, what the report card measures is compliance with the Federal Information Security Management Act, and FISMA does not equal security.FISMA is an important tool in establishing a consistent program for managing IT security. But it is only a tool. FISMA compliance does not necessarily mean good security, and poor compliance does not necessarily mean poor security. As FISMA matures and departments get the initial baseline work out of the way, the act certainly should contribute to IT security. But it is not yet a one-to-one correlation.The Government Reform Committee should consider retooling its report card to give a more accurate picture of the government’s security posture. A more fine-grained scale, weighted to be more sensitive to improvements made to mission-critical systems, might not grab as many headlines, but it just might provide more useful information about how secure our IT systems really are.