Metrics key to measuring success on IT projects

What: “Creating a Balanced Portfolio of Information Technology Metrics,” by Kevin C. Desouza of Arizona State University, for the IBM Center for the Business of Government.

Why: Information technology provides real-time data and the tools to display it, which is helping government decision makers meet their agencies’ needs. But according to the report, CIOs would benefit from more and better effective use of performance metrics.

Desouza interviewed 27 federal, state and local government CIOs and found a balanced portfolio of metrics can help agencies track the progress of IT initiatives. Metrics are transparent, identify agency priorities and track how goals are being met. And while that takes time and money, Desouza points out that without such measurements, the conversation in the public sector will be usually be about the times when things don’t work.

Among the report’s key findings.

• CIOs recognize the value of metrics. They said metrics help with project management, transparency, and seeing the bigger picture. For example, whether an IT failure was isolated or part of a larger pattern.

• CIOs are adept at managing through metrics for outsourced contracts. Agencies rely on contractors for many IT projects and the report found CIOs are adept at managing metrics from outsourced contracts.

• CIOs are participating in agency strategic planning processes. The CIOs Desouza interviewed said they are talking to stakeholders across the agency and matching their goals with overall agency goals. The report said CIOs had good results when they had these kinds of discussions.

• CIOs use metrics for benchmarking, but with caution. There is not always enough information about what other agencies are doing and access to resources differs among agencies. However, the CIOs surveyed are interested in what other IT departments are doing and the reasoning behind their metrics.

• CIOs need to carefully manage IT metrics around cost. Price point metrics measure cost in relation to the return on investment. Making a pitch around a unit price metric, like cost per megabyte, gives IT more leverage when negotiating with government officials and vendors.

Verbatim: “Given this growth in spending and the critical connection of IT to the operations and success of public agencies, we hear a limited number of success stories on how IT has helped transform agencies, deliver service more optimally, save taxpayer resources, and even lead innovation efforts to address vexing social challenges. We are more likely to hear about IT projects that have 'gone rogue' and failed to deliver on their promises. One possible cause for these problematic IT projects is that the chief information officer (CIO) community has not done enough to invest in the creation of metrics that capture the performance of IT assets and their contribution to organizational performance.”