GAO pushes DOD for better financial software processes

As part of its review of the Defense Department's financial management systems the General Accounting Office is pushing the Defense Finance and Accounting Service (DFAS) to improve software development processes at its Indianapolis facility.

DOD has been under scrutiny for several years because of what GAO calls the department's inability to produce "accurate auditable financial statements and other reliable management reports."

The DFAS Financial Systems Activity in Indianapolis is responsible for the development or modernization of a number of automated financial systems that DOD believes will help address its financial management difficulties.

In a report issued earlier this month and addressed to DOD comptroller John Hamre GAO noted that Indianapolis has taken steps already to improve its development processes.

However after evaluating activity on four sample projects GAO concluded that "significant risks remain that investments made in new software development will not achieve their operational improvement objectives and that software will not be delivered consistent with cost and schedule estimates."

GAO recommended that DFAS delay any major investment in software development at Indianapolis - "beyond that needed to sustain critical day-to-day operations" - until its weaknesses were addressed.

A DFAS representative was not available for comment however according to the report DOD "partially concurred" with many DFAS findings but it disagreed with the recommendation to delay spending on software projects.

GAO based its findings on the results of a software capability evaluation (SCE) a method of assessing development processes created by the Software Engineering Institute a federally funded research and development center.

Maturity Levels

An SCE evaluates a development organization based on its use of sound repeatable processes across different activities and its management of those processes. The institute uses a Capability Maturity Model that breaks out five levels of maturity an organization might achieve - from Level 1 which is characterized by lack of processes to Level 5 marked by a focus on continuous process improvement.

The software development community generally views Level 2 which focuses on using some basic repeatable processes as a fundamental goal. Most organizations undergoing an assessment generally attain only Level 1. DFAS initiated a software process improvement program to bring its development shops up to Level 2.

GAO conducts SCEs as a means of heading off problems that frequently dog software programs. "Every time we go somewhere to look at these development projects through government they are always late always over cost " said David Chao the SCE team leader at GAO. "We at least want to go into the agencies and establish a baseline for these guys so they know where to go from where they are."

Joseph Morin president of Integrated System Diagnostics Inc. Pocasset Mass. said a capability evaluation is based on the assumption that "process and product are related." ISD provides training and services in process improvement to government and private organizations.

Generally that assumption makes sense but ISD recommends that an SCE be viewed as only one component of an overall assessment. "We believe the current [evaluation] method is meeting its goals in terms of being a good diagnostic tool but it is not the only tool " Morin said.

In conducting its evaluation of DFAS GAO looked at two projects that were under the process improvement program and two that were not. According to GAO - but disputed by DOD - the Indianapolis organization does not yet satisfy the criteria for a Level 2 assessment on any of the four projects that were reviewed.

DOD disagreed with GAO's SCE findings pointing out that numerous individual projects had been evaluated as qualifying as Level 2 since the evaluation was completed.

In its report GAO acknowledged the dispute but noted "The major issue is not whether the FSA-Indianapolis has any supportable Level 2 projects but instead when will FSA-Indianapolis become a Level 2 organization?"