How CIOs can use FITARA to get away from waterfall

The newly enacted Federal Information Technology Acquisition Act provides an opportunity for federal CIOs to eliminate waterfall as an accepted approach for IT systems development, bringing the government in line with the private sector.

FITARA requires that the CIO of each federal department "certify that information technology investments are adequately implementing incremental development." More importantly, it provides the CIO with a lever for enforcement, requiring that the CIO approve every contract, and every budget request, for IT products or services by the department. Using these two levers, CIOs can effectively eliminate the primary cause of large-scale systems development failures.

The concepts behind waterfall software development date back to the 1950s and are based on the premise that IT projects should use the same project management disciplines used when creating a building or an aircraft carrier. Similar to physical construction projects, lots of dollars were at stake, and big IT projects typically took years to complete. Discipline was needed, because changes in requirements get increasingly expensive as implementation work progresses. Waterfall became the accepted process for doing large-scale IT systems "right." And government had a lot of large-scale IT projects.

But "right" didn't translate into "successfully" often enough. In fact, the landmark Clinger-Cohen legislation used a 1994 report on pervasive federal IT failures entitled "Computer Chaos" to justify large-scale changes in federal IT management. Similar research at the time by the Standish Group, "The Chaos Report," documented that only 9 percent of IT projects in large organizations (public and private) came in on schedule and on budget.

In hindsight, defining static requirements for IT systems has proven to be more difficult than for physical construction, especially where business processes or user interfaces are involved. In these areas the rate of change of user requirements has increased dramatically, commensurate with the rate of change in available personal technologies. For projects such as consumer-facing web systems or mobile applications, the rate of requirements change effectively has become continuous.

To deal with this, the private sector has moved to techniques like agile software development. While no process can guarantee success, approaches like agile accept the reality of today's constant change by embracing it. Perhaps even more importantly, agile enables incremental, "succeed fast" management that can be used to eliminate the huge software failures too frequently experienced in government. Two-week development sprints and frequent delivery of working code make it easy to tell if a program is off track.

With a long legacy of existing programs whose budget justifications have been based on waterfall planning, the federal government has been a laggard in the conversion of systems development programs to incremental and agile. But these programs waste billions of dollars annually by perpetuating the fallacy that huge IT programs that have yet to deliver any working functionality are "on schedule" because planning, coding, or infrastructure preparation are proceeding, especially when experience documents that more than 60 percent will eventually fail.

One of the first policy actions all department CIOs should announce under FITARA is that they will not approve any contract or budget request for an IT program that will not deliver production functionality during calendar 2015. For 2016, make it every six months (the current OMB definition of incremental).

For those who just screamed "they can't do that," FITARA clearly makes it possible, and it is what the Department of Veterans Affairs did in July 2009, resulting in the conversion of all IT systems development projects to six-month increments within one year. It also improved VA's on-time delivery rate for IT projects from 30 percent to more than 80 percent.

As the CIO at the Department of Commerce in 2000, I attended a high-level review of a another agency's system that, when implemented, would potentially save hundreds of law-enforcement lives each year. When challenged about the lack of progress in the program, the program manager stated -- and I swear these were his exact words -- "the first seven years have been spent planning!"

It's time to finally end waterfall in government. Lives and taxpayer dollars are at stake.