Do metrics measure up?

"Information Technology: Leading Commercial Practices for Outsourcing of Services"

Performance-based services contracts, in which agencies hold contractors accountable for attaining specific goals, have been around for a while. The question now, federal officials said during a March 28 panel discussion, is whether the concept actually helps an agency carry out its work.

Most agencies with large information technology outsourcing contracts have spent the first few years of the contract getting a better understanding of how to manage the contract, including the expectations of users and the relationship with the vendor, officials said. They have also been struggling with finding and managing the metrics used to gauge performance.

So only now are agencies really getting to the point where they can start examining whether the outsourcing contract has really helped the greater mission of their agency, said David McClure, director of IT management issues at the General Accounting Office, speaking on the panel of federal officials at a breakfast hosted by the Armed Forces Communications and Electronics Association's Bethesda, Md., chapter.

GAO is starting a review of the reasons some agencies use for not awarding IT services contracts and whether those reasons have any merit, McClure said.

NASA put in place several performance measures as part of its agencywide IT outsourcing contract, the Outsourcing Desktop Initiative for NASA (ODIN), said Lee Holcomb, the agency's chief information officer. Those measures include the availability of service, the level of service delivery, the level of customer satisfaction and the reduction in cost for each desktop.

Judging the measures to be successful, NASA is looking at the "more challenging" issue of how ODIN is helping the agency do its business, Holcomb said.

Agency officials recently began looking at the various business needs across the agency to determine how ODIN can better support those needs, he said. They will likely change the contract to provide better support and bring in new metrics to measure the agency's performance improvements, Holcomb said.

Many agencies engaged in large-scale IT services contracts are following commercial best practices outlined in a GAO report that came out last year, McClure said. But some agencies still are not, and as part of its review, GAO wants to find out why.

Many times, officials cite existing policy or legislation that does not in fact prohibit using the best practices, and agencies and Congress need to determine how to counteract those claims, McClure said.