NASA pushes software frontier

The first signs that NASA needed to improve its software engineering were apparent more than a year ago, before the failures of two Mars-bound missions.

But the software and management problems that contributed to the losses of the Mars Climate Orbiter and Mars Polar Lander in late 1999 convinced NASA's chief information officer that it was time to make changes.

Lee Holcomb, who heads the office at NASA's headquarters that is in charge of the space agency's information technology investments and policy, last month briefed senior managers on a plan to improve NASA's software development and management. Although the loss of the two Mars spacecraft highlighted software engineering shortfalls, Holcomb stressed that the need for better software is a national problem.

Holcomb hopes to raise the bar for NASA's internally developed software with a four-part plan (see box). The plan will be implemented quickly and aggressively, and a business plan will be delivered to Congress June 1.

"Research could do a lot to provide tools to improve the productivity of building software," Holcomb said. "We face the same problem everyone faces — the need for talent."

NASA also plans to incorporate the Capability Maturity Model into its software strategy, Holcomb said. The CMM, developed by the federally funded Software Engineering Institute at Carnegie Mellon University, has become a staple at many agencies as a way to assess an organization's overall software engineering practices, with a focus on establishing documented, repeatable processes that carry over from program to program.

Its proponents include the Defense Department, the Federal Aviation Administration, and more recently, the Internal Revenue Service, the Immigration and Naturalization Service and the U.S. Agency for International Development.

The CMM sets out five levels of "maturity," based on the extent to which those engineering practices are in place and documented. Level 1 indicates an ad hoc environment, which means software quality could vary widely from project to project, while Level 5 represents a highly structured environment. Most organizations aim for Level 2 or 3.

The CMM has achieved more credibility as larger organizations attribute their internal improvement to it, said Bill Peterson, director of software engineering at SEI. "It's impossible to test [for] quality and fix the defects late in development," he said. "If you find it at that point, you might as well do it again. The idea is to move it up front into the requirements and design."

NASA will have technical guidelines and procedures for software development in about two months, Holcomb said. In that same time period, the agency will use a more vigorous independent verification and validation program, bringing in experts to make sure that software projects perform as expected.

As a result, a program's schedule might slip, he said. "Part of this software initiative is just doing business the way it's supposed to be done," he said.

NASA will train its work force to manage software development, and software vendors will be expected to achieve CMM Level 3 or higher.

Computer Sciences Corp.'s NASA programs unit achieved software CMM Level 5 in November 1998. By using a combination of international standards and the CMM benchmarks, CSC was able to involve its whole NASA organization in improving productivity and return on investment, said Jerry Page, a vice president in CSC's NASA programs unit.

The goal of CMM is not the benchmarks but improving the product and its productivity, Page said. "You could have a process that is slick and still have junk coming out of it," he said.