The Lectern: Defense Contract Audit Agency problems

There's been more publicity the past few days -- because of hearings at the Senate Government Affairs and Homeland Security Committee -- about suggestions, first raised in a Government Accountability Office report, that the Defense Contract Audit Agency (DCAA) pressured auditors to "go soft" in audits of contractors.

I don't know anything about this controversy other than what I have read in the media about it, but one new theme that has caught my attention is the allegation that the use of "performance metrics" at DCAA put pressure on auditors to finish audits quickly, at the expense of an appropriate thoroughness. The implication from these accounts, either stated or implicit, is that DCAA should be getting rid of metrics.

This is a bad idea. What gets measured gets done, and that's actually generally good news for friends of improving government performance, because it suggests that metrics are a powerful tool to influence behavior. But of course this means that care must be taken in choosing measures, lest an organization be sent scurrying in an inappropriate direction.

When the critics attack "performance metrics" at DCAA, they are criticizing the fact that there were, apparently, timeliness metrics for audit work. Timeliness is a legitimate dimension to look at for any organization's performance, including an audit organization, lest employees waste time (that otherwise could be available for other audit activity, for example). At the same time, the observation that if you only have a speed metric for an activity that involves both speed and quality you will run into problems, comes from Performance Measurement 101.

This is not an insoluable problem. If an organization is worried that a metric looking at the average time a call center employee spends on the phone with callers encourages poor customer service or low quality, the average call time metric can be paired with metrics for customer satisfaction or answering accuracy (using a "mystery shopper"). Similarly, in an audit activity, a timeliness metric can be paired with a metric measuring the results of third-party random checks of audit quality.

Some people don't like metrics -- those who don't appear to share the urgency that others of us feel about encouraging the best possible government performance. Life can be easier, perhaps, for some employees without any metrics, but the public purpose an organization serves must come first. It's unfortunate that some seem to want to use the issue around DCAA audits to launch an attack against one of the best tools we have to make government work better.

Steve Kelman