Correct use of past performance key

I recently received a phone call from a friend who works for a vendor that had just lost a big award. Company executives learned during the debriefing with the agency that a major reason why the company had not won was that it did not score as well as others on past performance.

My friend was somewhat angry because he had heard that a number of competitors who had won awards filled out past-performance references for their government customers in response to the agency's request for proposals. He was naive in not doing so, he said, but is planning to do so the next time around.

My friend called around the time I was compiling interviews I had conducted with government contracting professionals as a part of research on procurement reform. It was interesting how wide the range of comments was about the impact of increased use of past performance in government contracting. Some of my interviewees thought the use of past performance was the greatest thing since sliced bread, while others thought it never was a discriminator in source selection and that it was worthless. It almost seemed as if people were talking about different activities.

In fact, they probably were. A point that social scientists have made about social improvement efforts is that under a single rubric, such as "school tracking," dramatically different methods can appear. One study may conclude that school tracking is a failure and another that it is a success. But if you peel away the onion's layers, often it turns out that the studies do not examine the same thing, despite the common name.

The point is, although everyone in the government is now using "past performance," the way they use it varies greatly. Those agencies using it effectively, I suspect, generally are finding that past performance is working well to produce better vendor performance. Conversely, those using it ineffectively probably find that past performance is not helpful. So the government's task is to learn how to use past performance intelligently so that it can live up to its full potential.

Let us begin with my friend's complaint about vendors writing past-performance references for customers. I do not think this practice should be prohibited. One must assume that a vendor willing to ask a customer to allow it to write a reference must have a good relationship with that customer.

However, I do believe that vendors should be required to disclose in their proposals, or in the references themselves, which customer references they helped prepare. And agencies should be particularly sensitive to nonsubstantive fluff when examining such references. The government might state in its RFP something like this: "If the offeror has assisted a customer in developing a reference, the government will look for specific examples illustrating the offeror's performance and not just generalized judgments."

What are some other suggestions for effectively evaluating past performance? One important suggestion comes from Deidre Lee, administrator of the Office of Federal Procurement Policy, who says that past-performance evaluation should be seen as an ongoing tool of contract administration.

For example, the Defense Supply Center, Philadelphia, has semiannual reviews with its vendors and contract customers to discuss strong points and areas of improvement. In the multiple-award service delivery contracts they administer, the exercise of option years and the proportion of the total workload going to different vendors is based on customer satisfaction ratings. To reference back to the earlier discussion, any agency that waits until it receives a reference request to develop materials on a vendor's past performance is losing much of the incentive and learning value from the exercise.

Vendors should not be allowed to cherry-pick contracts they use for references. Agencies should specify in RFPs selection criteria for reference contracts and request that vendors provide a specific number of recent contracts fitting that bill.

Agencies also should form more sophisticated questions to ask references. Agencies should ask specifically what is best and worst about a vendor's performance. They should ask references to state whether a vendor's performance, compared with other vendors that have conducted similar work, is better, worse or average. They should ask for specifics. Past-performance ratings are less subjective, and more valuable, the more they relate to performance measures or service levels in the contract or task order.

The vendor community clearly has become more focused on past performance and believes the government is serious about it. Once the government demonstrates its seriousness, we will reap the benefits.

--Kelman was the administrator of the Office of Federal Procurement Policy from 1993 to 1997. He is now Weatherhead Professor of Public Management at Harvard's Kennedy School of Government.