Agency CIOs Undercount ‘High-Risk’ IT Projects, GAO Says

Too many federal agency chief information officers are viewing their major IT investments through rose-colored -- or make that green-colored -- glasses. 

That’s according to a new Government Accountability Office report, which found agency CIOs at more than a dozen agencies were too optimistic in assessing the risks associated with large-dollar IT projects.

The CIO “risk ratings” are collected on the IT Dashboard -- a public website launched by the Obama administration in 2009 designed to provide a real-time snapshot of cost and performance of the more than $80 billion spent each year on federal IT.

Based on CIOs' ratings, projects are coded green, yellow or red to denote “low-risk,” medium-risk” and “high-risk” IT investments.

But CIOs, according to GAO’s latest analysis, are not adequately assessing risk of major projects -- which diminishes the power of the IT Dashboard to operate as an early-warning system for potentially off-the-rails IT projects.

» Get the best federal technology news and ideas delivered right to your inbox. Sign up here.

Some agencies’ risk ratings “do not provide an accurate assessment of investment risk and thus reduce the value of this important tool for transparency and oversight,” GAO auditors concluded.

As part of its review, GAO specifically examined 95 IT investments across 15 different agencies, including the departments of Commerce, Defense, Homeland Security, Health and Human Services, and Veterans Affairs.

Using agencies’ own data, GAO reassessed the risks associated with their major IT projects -- winding up with some very different results in many cases.

Nearly two-thirds of the projects evaluated by GAO showed more risk than originally reported by CIOs, according to the report.

For example, of the 95 investments selected for review by GAO, agency CIOs had initially reported 61 of them as low-risk. When GAO conducted its own analysis, it turned up only 15 low-risk projects.

Overall, GAO’s assessments matched CIO ratings just 22 times. GAO uncovered more risk than originally reported in 60 investments -- and less risk in 13 investments.

Among the projects originally green or yellow that GAO says should’ve been flagged red:

• The Pentagon’s massive Defense Healthcare Management System Modernization, which aims to replace the department’s electronic health records system with commercial software.

• The Census Enterprise Data Collection and Processing initiative -- the data-collection effort that plays a central role in the tech-heavy plans for the 2020 Census.

• U.S. Citizenship and Immigration Service’s troubled multibillion-dollar “Transformation” program to modernize the mostly paper-based immigration process.

The report concedes the practice of evaluating risk is an inexact science.

“Such assessments of risk inherently involve a great deal of human judgment,” the GAO report stated. Auditors also said CIOs may have been privy to additional information about a particular program.

What else accounts for the discrepancy in ratings?

Contrary to Office of Management and Budget guidance, some agencies are only infrequently updating data on the dashboard. (Current OMB guidance requires at least monthly updates, although administration officials told GAO that requirement will be relaxed in future guidance).

The sporadic updates “raise concerns that those updates are not reflecting timely and accurate risk information,” GAO concluded, adding, “Such practices limit the transparency and oversight of the government’s billions of dollars in IT investments.”

In addition to infrequent updates, the report also found some agencies were only taking into account process-based risks, such as whether a process or policy existed, and not including so-called active risks, “such as funding cuts or staffing changes, which detail the probability and impact of pending threats to success.”

GAO recommended agencies update risk ratings more frequently and include active risks in their evaluations.

Most agencies agreed with GAO’s recommendations.