Can we trust evaluations?

As the administration presses for more transparency, one emerging proposal is causing a stir in the acquisition world: publishing contractor performance evaluations. But although advocates consider it a no-brainer, it can’t pass the feasibility test. The major hurdle is the government version of organizational conflict of interest.

At first glance, because public funds are spent, it seems reasonable to reveal the evaluation reports as public information. But we have to ask: Can these reports be objective, fair and accurate? The government prepares them to keep on file and help selection panels grasp a company’s past performance — not for public review.

It’s common for agencies and contractors to say they are “partnering.” Although contract documents usually don’t call for much in the way of shared risks, the partners do tend to lock arms when pesky auditors, congressional committees or probing media people show up. That helps explain why unfavorable reports of any kind rarely hold a company back from the next contract award.

And it’s important to note that only a minuscule percentage of the millions of contract actions performed yearly are ever found to be problematic. That might comfort taxpayers but also might make them wonder if the reports are comprehensive and accurate, not to mention fair and consistent.

And it does takes two to tango. Contractor performance issues that reach daylight are almost always inextricably bound with government program management problems, such as defective requirements or inept contract oversight.

Take Project 28, the virtual border fence pilot project conducted by Boeing for the Homeland Security Department. The department forgot to get the users, the Border Patrol, involved with Project 28. In the end, DHS tagged Boeing’s technical deliveries as egregious, delayed acceptance of the product for eight months, demanded and got a $2 million refund, and said in so many words that it could not use the Boeing solution. Boeing reportedly spent twice as much as it earned through the $20 million fixed-price contract. Like the government, it changed over its program management before Project 28 was done.

It was an ugly performance all around — though a powerful generator of lessons learned in technology integration and acquisition that both parties should have been expected to know already. Still, then-Secretary Michael Chertoff insisted Project 28 was successful. Would you trust in the accuracy and objectivity of DHS-prepared evaluation of Boeing’s performance in Project 28?

Somebody might have, because DHS said May 7 that it was picking up where it left off with Project 28 and tasking Boeing to proceed with implementation of the virtual border fence. We infer that either DHS gave little weight to an objective evaluation report that documented Boeing’s problems or the report was not reliable enough in the first place. Either kind of problem is common, according to frontline acquisition managers.

And not surprisingly, reviews have shown repeatedly that the government awards an improbably high percentage of maximum or very high award fees almost all the time. Those who determine these fees would be the people who prepare performance evaluations.

And if the use of public funds fuels the drive to reveal contractor performance reports, it might not be long before citizens demand that government employees’ individual performance evaluations be published.