Making it count

Agency managers must justify information technology investments to obtain the Office of Management and Budget's approval. It's a necessary step, though not always an easy one. How do you conduct a cost-benefit analysis when effectiveness cannot be measured in profit and loss as in a commercial business?

SAP is the latest firm to try to answer that question. The company has marshaled an array of resources, ranging from its own customers to academic institutions and think tanks, to figure out how agencies should measure return on investment.

The Public ROI initiative, led by SAP's Global Public Services organization, will develop a system for analyzing and measuring IT investments based on how well they effect social and political change. Accenture launched a Public Sector Value model in 2003 with a similar goal.

Agencies increasingly need a way to measure value, said Chip Mather, senior vice president of Acquisition Solutions.

In addition to Accenture and SAP, the General Services Administration and other government organizations have various tools to help agency leaders perform analyses. Although such tools are useful, Mather said, agency officials must remember that they're only tools.

"I'm personally not a big fan of tools that reduce complex things to numbers," he said. "It's a subjective judgment, and if all you do is turn a subjective judgment into numbers, you get a false sense of accuracy."

Steve Peck, president of SAP's federal division, said that in addition to monetary concerns, agencies have several reasons to measure ROI. Although agencies don't have the profit motive that drives businesses, they depend on appropriations to fulfill their missions, he said.

"Many times entities are reluctant to say, 'We saved $50 million in these areas,' because they find out that they'll get rewarded with a budget cut," he said. "So when you're looking at a public project, it needs to be more than just hard dollar savings."

Agencies also face a loss of expertise due to worker retirements and have a wide variety of missions. The factors that agency officials should measure to determine the value of an IT project must be tailored to the individual agency's mission, Peck said.

"We've seen so many dedicated public servants who know they have to do more with less," he said.

The initiative formally starts this month, but a date for a final product has not been announced, he said.

From SAP's perspective, a method for agencies to calculate ROI would be a marketing boon, Peck said.

"Our ability to help people establish business cases will open up more opportunities for us," he said. "We're going to be using the methodology and applying it to some things in our consulting services."

In contrast with Accenture's Public Sector Value model, the purpose of SAP's Public ROI is not to use the methodology for long-term consulting engagements, he said. Instead, it will be best-suited for quick analyses that can guide an agency's planning.

However, the methodology will be tailored to fit organizations' missions and needs, he said. In the Defense Department, for example, "understanding where the leadership is trying to take the military over the next five or 10 years, we can better bring in systems to support it," he said. "Otherwise, you're just doing process for process' sake."