VA, GAO fight over system development process

The Veterans Affairs Department and the Government Accountability Office are fighting over the process used in developing the VA’s Post 9/11 GI Bill education benefit claims processing system.

VA is supposed to be using a process called Agile development, in which projects are segmented and functionality is delivered in small increments. In an assessment released Dec. 1, GAO said VA is missing a critical component of an agile program -- a velocity oversight mechanism. Such a mechanism is a process that measures rate of work completed and changes to project scope over time.

“Without the overall velocity — a key mechanism under the Agile methodology — VA does not have the information to understand the expected effort to complete the total scope of work and the associated length of time to do so,” GAO said.

The dispute could have implications for the White House’s promotion of incremental development approaches for IT programs.

Zients outlines five strategies for IT management

Bite-size procurements can minimize big-time problems

However, Roger Baker, the VA’s assistant secretary for information and technology, disputed that assessment and suggested the GAO might not understand agile development.

“Because the Agile methods are not broadly used in the federal sector, this may have been the first exposure the GAO team performing the audit had to the methodology. Limited exposure to Agile could possibly have caused GAO to present incorrect assumptions as fact,” Baker wrote to Valerie Melvin, director of IT and human capital issues at the GAO, on Nov. 12.

Baker also contended in the letter that VA’s agile development the claims processing system resulted in on-time and successful releases of functionality for the first three phases.

“The results speak for themselves,” Baker wrote. “All three releases deployed during the GAO audit were installed with no significant (defined as Level 1 or 2) errors.” Since installation, he added, “processing with the new systems has been nearly flawless, with no significant bugs encountered.”

The GAO report said although the agile process has given the VA added flexibility and helped to bring in early phases on schedule, the department also has reduced functionality and experienced schedule delays for later phases.

GAO also noted that the VA was nearly two months late in completing the conversion of data from the interim solution to the final solution that was supposed to be finished in June. In addition, some of the functionality intended for the third and fourth phases was delayed, the report said.

GAO made recommendations to the VA to establish appropriate performance metrics; maintain full bidirectional traceability for legislation, policy, business rules and test cases; and establish criteria for “done” work at each phase. The VA agreed with recommendations to improve those areas.

The GAO report recommended that the VA improve its testing, but the VA also disputed that recommendation, saying adequate testing procedures were in place.

In recent months, White House officials that include Vivek Kundra, the federal CIO, and Jeffrey Zients, chief performance officer, have talked about the advantages of incremental acquisition programs for development in federal IT programs to reduce costs and minimize the chances of a project failure.

For example, Kundra spoke about the strategy of incremental deliverables in a speech to the CIO Council on Sept. 20. Zients included building a federal acquisition corps familiar with agile development and other complexities as part of the structural change strategy he outlined on Nov. 19.