Measuring the benefits

When Ira Grossman, an information technology architect at the National Oceanic and Atmospheric Administration, worked last year on his agency's enterprise architecture plans, one aspect of the task gnawed at him. "There's no metric that allows me to record and measure my improvement [in developing and applying the architecture]," he said. "How do I know that I am improving year to year?"

He found his answer in a management tool not widely associated with enterprise architecture — the Capability Maturity Model (CMM).

Carnegie Mellon University's Software Engineering Institute, a federally-funded research organization, created CMM to help software development organizations improve their operations, but the model has since been adapted for other IT development tasks, including enterprise architectures.

Like some other quality management techniques, the focus of CMM isn't on the end product — the piece of software or an enterprise architecture plan — but on the development process itself. The assumption is that a good, disciplined process that is regularly reviewed and improved will typically yield a higher quality product.

The Commerce Department uses the CMM's five levels of process maturity (see chart) to track its progress on a list of key tasks in the creation and use of an enterprise architecture. Measurable tasks include linking the architecture to agency business strategies, getting involvement from senior management and operating units, and coming up with a way to ensure that the architecture is used to govern new IT investments and development.

The value of applying CMM is in large part the accountability factor. "The big bang theory of building enterprise architectures where you say, 'Give me three years to build this,' doesn't work," said Scott Bittler, vice president of enterprise planning and architecture strategies services at the META Group Inc., which helped Commerce develop its CMM. "People need the ability to assess their progress and improve on a regular basis."

Grossman said he still needs to fine-tune some of the categories in the CMM to make them more pertinent to the department. But overall, he said the model has been useful and well-received by his own department's Office of the Inspector General and by the Office of Management and Budget.

***

Commerce Department's IT Architecture Maturity Model

Level Characteristics

0 — No IT architecture program.

1 — Initial. Informal and ad hoc IT architecture process under way.

2 — Formal but basic IT architecture process is being developed.

3 — IT architecture, including detailed written procedures and technical reference model, is defined and largely followed.

4 — Managed and measured IT architecture process with performance feedback mechanisms in place.

5 — Optimizing. Continuous improvement of IT architecture process based on feedback.