Vendors get average grades

Maybe it's a reflection of agencies' changing view of vendors, or maybe it's just the freedom to be candid that anonymity offers. Whatever their motivation, agency information technology leaders say the vendors they work with do only a so-so job.

That's one of the findings in a new report from Market Connections Inc. The market research firm asked IT professionals working in agencies to rate 17 vendors on 15 factors, such as quality of products and services and responsiveness to

inquiries.

The survey, which also measured the name recognition of companies and included questions about other topics, divided vendors into three categories: manufacturers, resellers and systems integrators.

Out of 15 factors, no company rated higher than a B-minus average score, said Lisa Dezzutti, president and chief executive officer at Market Connections.

The overall average score, counting all the companies across all 15 criteria, was a C-plus.

Dezzutti said the survey, which her firm conducted by telephone in January, was structured so that only people with some knowledge of a company were allowed to grade its performance. More than 600 IT professionals participated in the survey, she said, although not all of them took part in the ratings. No one rated more than three companies.

"They had to have a certain level of familiarity to respond," she said. However, both customers of a company and those who simply knew it by reputation could offer ratings, she said.

"It is a perception of performance," she said. "We didn't just talk to customers."

Dezzutti did not release the names of companies with their grades. At least in some cases, she said, the low scores surprised her.

"I was surprised, particularly on some of the resellers, that we didn't see

a little bit higher marks," she said. "Nobody's getting gold stars here. Nobody's

doing really bad, we're not getting Ds or Fs."

Although the 2004 survey is the fourth annual report of its kind that Dezzutti's firm has produced, it is the first one to

include grades, which Dezzutti calls the Quality Perception Index, or QPI.

The survey also measured federal IT professionals' awareness of various vendors. When asked aided questions with multiple-choice answers, the responses showed that systems integrators, as a whole, had the lowest awareness levels compared to manufacturers and

resellers.

When asked unaided, unprompted answers, the responses showed that IBM Corp. and

Lockheed Martin Corp. rated highest among integrators, CDW Government Inc. was the best-known reseller, and Dell Inc. and Hewlett-Packard Co. were the manufacturers most often mentioned.

So what explains the lackluster performance numbers? Larry Allen, executive vice president of the Coalition for Government Procurement, suggested that the freedom of anonymity played a role.

"Not every contractor performs perfectly every time, but you would expect some companies to score well," he said. "The fact that they didn't lends credence to company concerns that customers feel free to rag on people."

Many contractors are wary of customer feedback, Allen said. Customers take advantage of chances to vent frustration, even if they are really more satisfied than their griping might suggest, he said.

"The contractor ultimately is only going to be able to control so much of the equation," he said. "There are parts that are going to always be subjective."

To lower the chances of bad feelings or poor feedback, vendors and their agency customers need to talk more, Allen said.

"I think there's a need for better communication upfront between customer and contractor," he said.

Chip Mather, senior vice president of Acquisition Solutions Inc., offered a different take. Agencies are increasingly holding contractors accountable for results, he said.

"It used to be that industry wasn't being held accountable," he said. "They did what the government told them to do, and [failure] was mostly the government's fault. The bar is being raised. What used to be acceptable is no longer. It's not enough to just show up and do what the government tells you."

The Office of Management and Budget has become more likely to demand that agencies justify the cost of IT projects, while concepts such as performance-based contracting are putting more pressure on contractors to come up with successful solutions, Mather said.

Another question on the survey indicated that agencies consider sharing information among one another as the most important aspect of meeting their homeland security missions. Despite outreach efforts such as the FirstGov Web portal, they consider sharing information with the public to be the least important aspect.

Civilian agencies are much more interested in getting information to the public than are defense agencies, according to the

report.

"It did surprise me," Dezzutti said. "There has been so much emphasis on e-commerce, service to the citizen." However, she added, "they were very focused on homeland security, rather than information sharing in general."

Allen saw in the response evidence that agencies are behaving secretively.

"There is a reluctance to share things, even information that is known to be public, with anyone," he said. "I think there is a disconnect between the people who

set out these agendas [such as e-government] and lay out what they do, and the line people."

***

Average grades

In a recent survey conducted by Market Connections Inc., federal information technology managers graded the companies they do business with at about a C-plus average. On 15 performance factors, no factor got higher than a B-minus.

The 15 performance factors vendors were graded on:

1. Quality of products and services.

2. Wide selection of products and services.

3. Ability to resolve problems.

4. Experience in the federal market.

5. Customer service and support.

6. On-schedule delivery.

7. Skills and knowledge of staff.

8. Reputation/past performance.

9. Responsiveness to inquiries.

10. Thoroughness of the work performed.

11. Price.

12. Ability to provide complete solutions.

13. Large variety of contracts to purchase from.

14. Ability to provide value-added services, such as training, integration and network services.

15. Ability to provide integration services.

Source: Market Connections Inc.