Making software NASA-tough

The Sustainable Computing Consortium (SCC), formed last month by NASA, Carnegie Mellon University and other organizations, aims to do nothing less than completely remake the way software is engineered and its reliability measured — something that software engineers have attempted for 30 years but has so far proved elusive.

The goal is to develop standards, guidelines and architectures that will lead to the development of more reliable software, along with automated tools that will be able to check code and rate its reliability using a well-defined scale.

That in turn will make it easier for department heads or chief information officers to manage projects dependent on that software with measurable and insurable risks in the way other types of projects are now managed.

But it will take an effort well beyond anything that's been tried so far, because the basic knowledge of reliability is weak, at best, software experts say.

"The truth is we are in a pretty sorry state" as far as understanding reliability is concerned, according to William Scherlis, co-director of the SCC and principal research scientist at Carnegie Mellon's School of Computer Science. "Words such as unbreakable, trustworthy and impervious are confined to advertising literature. They have no measurable limits."

Along with NASA and Carnegie Mellon, SCC founders include Microsoft Corp., Oracle Corp., Raytheon Co., Alcoa and UPMC Health System in Pittsburgh. They will tap initial funds of almost $30 million from research grants and SCC membership fees.

The consensus seems to be that NASA, among others, has a good track record of building reliable software, particularly in its space shuttle program.

The question is: How can NASA's experience be applied to software development in general?

"With the space shuttle code, the people involved did the kind of due diligence that needed to be done wherever they could to get it right, and that could be a great place to start," said Samuel Keene, an industry consultant, a fellow at the Institute of Electrical and Electronics Engineers and a former president of the IEEE Software Reliability Society.

But most organizations are working with commercial off-the-shelf software, Keene said, and that is one field that could see the greatest benefit from the SCC.

Others are not so sure how well the technology will transfer. The kind of process NASA went through isn't what the commercial world can afford to invest in, according to John Pescatore, an analyst with Gartner Inc.

"It's like the difference between perfectly secure and secure enough," he said. "Perfectly secure is too expensive for most companies in the real world."

Pescatore said the SCC's best strategy might be to focus on developing tools and techniques for measuring the reliability and security of software under development, so there is some way of grading software, and letting the market choose how much "strength" it needs and is willing to pay for.

The SCC seeks to create a safe forum in which all of the issues associated with the problem — technology, economics, policy, law and understanding of markets — can be brought together to come up with a "holistic understanding" of the challenge and how to move forward, Scherlis said.

"The SCC is not about developing a magical technology bullet, but rather to understand this issue as a market phenomenon and to try and move it along," he said.

Robinson is a freelance journalist based in Portland, Ore. He can be reached at hullite@mindspring.com.

***

An idea that works

Carnegie Mellon University and NASA, together with 15 software companies, are already members of the 2-year-old High Dependability Computing Consortium (HDCC), which was formed to improve NASA's ability to create dependable large-system software and now will be one of the working groups within the Sustainable Computing Consortium (SCC).

The HDCC focuses on the requirements of systems used in such areas as air traffic control, space exploration, highway safety and health care.

As the SCC gathers steam, it might create more projects like the HDCC to address NASA-specific problems, with those projects possibly serving as test beds for technologies developed as a result of SCC endeavors, said Jan Aikins, chief technology officer for the information sciences and technology directorate at NASA's Ames Research Center in California.

"In general, however, SCC will act more like the board of directors of a company," she said. "It will work to get other players to look at reliability problems, and it will act as a broker among the various stakeholder industries."