Survey: CIOs Struggle to Measure Success in Dollars and Cents

Thanks to a 2014 IT reform law aiming to empower agency chief information officers, members of the federal C-suite say they’re collaborating to better manage major IT investments.

That’s according to a new survey of agency CIOs, chief financial officers and procurement executives published Jan. 19 by the Association of Government Accountants. Per that survey, 100 percent of respondents said their agency’s IT strategy is formed with input across the agency boardroom -- a key aim of the 2014 Federal IT Acquisition Reform Act, known as FITARA.

But despite combining powers with the agency coin counters, CIOs say they still struggle to communicate cost savings associated with successful IT projects and to measure the risks of future projects, according to the survey, which was unveiled at AGA’s Federal Financial Systems Summit in Washington, D.C.

When Congress approved FITARA in December 2014, one of the biggest open questions was how CIOs would control the purse strings of IT spending at their agency -- and how willing CFOs would be willing to share that traditionally accounting function.

Not to worry, according to a panel of agency CIOs and CFOs, who dissected the survey results and cited numerous areas of renewed collaboration.

For example, at the Social Security Administration, officials are rethinking their traditional investment review process.

The previous method had been “painful, not very well designed or certainly not executed well," said CIO Rob Klopp, who founded several Silicon Valley startups before joining the Obama administration’s U.S. Digital Service effort and later joining SSA.

The new process, currently undergoing a facelift, “will allow the business to suggest the mission-critical kinds of things that they want us to build for them, and more importantly for the various components of the business to work together to prioritize so we take them on in the right order,” Klopp said.

At one of the agency’s quarterly senior staff meetings to discuss “critical” projects, brainstorming among business and IT leaders resulted in a new initiative to expand online services. SSA kicked off planning a few months ago to work on a new project finally allowing people to request replacement Social Security cards entirely online, said Michelle King, the agency’s CFO.

Still, CIOs say it’s often difficult to communicate success stories to other parts of their organization because of the difficulty of calculating cost savings.

More than half of the CIOs surveyed -- 63 percent -- said they struggle to calculate the return-on-investment of successful IT projects or to communicate those cost savings to other parts of the agency.

And as more agencies adopt more modern project-management disciplines -- such as agile development -- those question marks on the balance sheets will only proliferate, CIOs predict.

"The whole idea of measuring progress as you go and keeping on top of these investments becomes interestingly complicated if you do agile, because the whole point of agile is that we don't know what the end result is,” Klopp said.

For years, the federal government managed major IT projects using the so-called waterfall approach. Planners meticulously gathered system requirements upfront, which were then turned over to the IT development shop responsible for cranking out a product. Years later, the agency may have had a functioning system -- or not, given the long string of failed big-bang IT projects.

Agile development, on the other hand, prioritizes quick deployment of a minimum viable product and then regularly rolling out improvements and added functionality.

The new approach “completely changes the game” for auditors and accountants, Klopp said. And it may not be to their liking just yet.

“When I get in front of OMB and I mention agile, they jump up and down and applaud and go, 'Oh, you're fantastic,'” Klopp said. “But then I go, 'That's great, but you know that that means you can't hold me accountable for all of the things that you try to hold me accountable for.’”

Another challenge in calculating IT costs is tied in to the government’s legacy IT challenge.

Nearly two-thirds of the government’s $80 billion IT spending annually goes toward operations and maintenance of so-called legacy systems. Federal CIO Tony Scott has called the lopsided spending trend a “crisis.”

Luke McCormack, CIO of the Department of Homeland Security, said he and his counterparts have struggled with calculating IT savings and ROI, "because we had a hard time understanding current costs."

"It was weaved into these legacy systems," he said. "There's not good accounting in there. There's not good forensic accounting as I like to call it, to unravel those costs and understand what you're going to retrieve when you go to a new system.”