VA waiting lists and performance measurement

Accusations that a Department of Veterans Affairs hospital in Phoenix, and perhaps others, created artificially positive wait-time statistics by hiding the names of veterans who were waiting a long time for appointments on a secret list have been all over the print and television media. A Google search with the keywords "Veterans Affairs Phoenix" yielded over 19 million hits.

As a balanced Associated Press article in the Washington Post makes clear, at this point the accusations have not been proven and VA hospital officials in Phoenix have denied them. But let's assume for a moment that they are true, which is not impossible or even implausible. If so, it would not be the first time that people fudged or cheated when reporting performance information to overseers. Think of the various scandals in which teachers changed student test results or, to take a very dramatic corporate example, the accounting manipulation at Enron.

The VA's cheating, if it occurred, happened in the context of the department's performance measurement system, which collects data about various aspects of the performance of VA hospitals for the purpose of using it to improve how well the hospitals serve patients. What are the implications, then, of cheating for how we look at the increasingly widespread use of performance measurement to manage in government?

The strongest critics -- and they are not a tiny group -- would argue that situations like this show that government should not be using performance measures at all.

That argument does not make any more sense than it would to argue that Enron or any other accounting scandal meant profit should no longer be used as a performance measure for businesses. The ability of profit to drive the improved performance of companies means that a world without profit as a performance measure might not have Enron, but it would likely have lots of East Germanies.

Indeed, the allegation that VA patients died because of waiting-list falsification can be made only in the context of noting that a genuine, non-falsified system of performance measurement saves lives. In a system with no performance measures for wait times, those on the secret list would not have gotten treated any faster than they did. Indeed, in that world, everyone seeking treatment would in effect be part of a secret list.

Furthermore, the suggestion that people might have died because of the falsification makes sense only if we assume that correct performance information about wait times (as opposed to eliminating those waiting a long time from the numbers) would generate information about deficiencies that then would encourage corrective action.

So if we should be tracking the VA's performance measures, what should we do to reduce the risk of cheating? As Shelley Metzenbaum, president of the Volcker Alliance, said, the first line of defense is to integrate performance measurement into the organization's IT system for processing customers. If every patient who seeks an appointment at a VA hospital is immediately entered into an IT database and that database is used to develop the performance information on the hospital's wait times, it would not be possible to create a secret list that removes the patient from the system, especially if the system were configured to make it impossible to remove a name once entered. Frankly, I would be a little surprised if VA hospitals did not already have such a system, which makes me slightly skeptical of these allegations in the first place. (Full disclosure: Metzenbaum is a former Office of Management and Budget and Environmental Protection Agency official, a 2011 Federal 100 winner, and my wife.)

Beyond that, we should return to the Enron accounting example I mentioned earlier. The most widely used performance measure in the world is the profit measure for private firms. Given the large amounts of money at stake for financial markets and individual company executives, that measure creates enormous incentives for cheating and gaming. That is why more than a hundred years ago an accounting industry developed to audit the accuracy of companies' financial performance information. That system is not perfect, but it is hard to imagine that capitalism could function without auditors keeping the level of fraud low for performance measures.

As government becomes more serious about performance measurement, the danger of falsification will increase. In some sense, this is a good sign that measurement is being taken seriously. And falsification is not bad just in itself; it is bad because it hurts the ability of those measures to improve performance. It would be good, in my view, if inspectors general diverted energy from some low value-added, grandstanding activities in which they are currently engaged to taking on the role of an agency's auditors of performance measures.