FAA sees EVM dividends

Officials at the Federal Aviation Administration can empathize with their counterparts at other agencies who are under pressure from Congress and the Bush administration to improve management of information technology programs.For years, FAA, like many agencies, has had a tough time determining accurate cost and schedule estimates for complex air traffic modernization projects. However, its situation is changing, particularly at FAA’s Air Traffic Organization.Much of the improvement stems from using an earned value management system.  EVM provides managers information for understanding the health of a program by relating data about project resource planning to costs and schedules.  The Government Accountability Office has praised FAA for promoting EVM throughout the Air Traffic Organization to help improve management of IT investments and estimate baselines for IT projects’ costs and schedules. Agencies need help to implement project management disciplines, refine their processes for identifying problems and make their earned value process mature to spot problems and involve agency leadership, said Dave Powner, GAO’s director of IT management issues.FAA and the Internal Revenue Service are examples of agencies that, despite some poorly performing IT projects, have had some recent successes delivering projects on time and on budget, Powner said. In both cases, the agencies emphasized on program governance, and they established cross-agency centers of excellence to help program management offices.“They aren’t perfect, but there are pockets of success in those agencies,” he said at a July 31 hearing of the Senate Homeland Security and Governmental Affairs Committee’s Federal Financial Management, Government Information, Federal Services and International Security Subcommittee.Unfortunately, replicating FAA’s success might prove challenging for many agencies. In fiscal 2005, the agency began implementing EVM as a keystone element of project and acquisition management and more sharply assessing its IT business cases. Its goal was to have all projects under development in 2007 earn a “green” ranking for management from the Office of Management and Budget, said Robert Rovinsky, director of IT Enterprise Services in FAA’s Office of the Chief Information Officer.  FAA’s Air Traffic Organization accomplished that goal last December, he said. However, GAO warns that an EVM system is only as good as its quality of information feeding it and the expertise of the managers using it. The baselines that agencies use to track the progress of their IT projects’ cost and schedule plans often are inaccurate, Powner said. Agencies have to re-establish the baseline in 48 percent of their IT projects, sometimes more than once, he said. GAO has found that agencies use EVM in about 25 percent of projects, but the process is immature at most agencies and managers need better training in it, he said.“Without knowing the planned cost of completed work and work in progress, or the earned value, it is difficult to determine a program’s true status,” Powner said.  As part of its initiative, FAA identified employees who understood project management and who could build relationships with other managers. The net result was an EVM Council that could monitor the Air Traffic Organization’s modernization projects, Rovinsky said. All the FAA components that dealt with acquisition contributed to the assessment, including financial, safety and air traffic, system engineering, risk management and the CIO’s office. The EVM Council, which includes representatives of operations and policy areas, proposes changes in policy and guidance for IT project improvement .When a project is launched, managers typically track cost, schedule and performance and whether the project meets the requirements. They outline the timeline, budget and list of deliverables. “There is a mistaken belief that watching cost, schedule and performance is the way you manage a project,” Rovinsky said, adding, “We call those the three evils because they’re easy to do but give a false sense that things are on track.”The problem is that problems with cost, schedule and performance typically do not begin to show until a project is perhaps halfway or three-quarters done, he said. On the other hand, with EVM, an agency breaks down a project into specific work units and tracks their units’ progress against select measures. That makes it possible to identify problems perhaps a third of the way into projects he said.“EVM is an early warning system for management to take constructive actions and not just rely on the project managers, who tend to be a little bit optimistic,” Rovinsky said. Implementing EVM improved program management across FAA’s organizations because it forced managers to evaluate how to set up programs, what kind of data to use, how to identify and address risk, how to put contracts together, how to track against schedule and costs, he said. “We used EVM as a way to turn over rocks,” Rovinsky said. “We exposed a lot of weaknesses in program management operations that we fixed or addressed.” FAA set up teams that evaluated the work breakdown structure, costs, tools, data and financial systems. The team made recommendations for improvements that FAA has been adopting.In fiscal 2009 and 2010, the agency plans to establish a surveillance process to verify that IT projects continue to use EVM and follow best practices to retain their green rating, he said. Based on its efforts to improve program management, Rovinsky said he is cautiously optimistic that GAO will remove the Air Traffic Modernization programs from its next high-risk list — after 14 years — when GAO publishes it in January.The challenge of program management is to deliver programs effectively while managing appropriate risk, said Alfred Grasso, chief executive officer of Mitre. He said the oversight procedure needs to change and focus as much on programs that have overcome challenges and gone from bad to good as programs that have slid into trouble so agencies can broadly apply the best practices.“We recommend a more enabling oversight role in the future — modeled by Congress, executed by OMB and supported by the Government Accountability Office,” he said.Federal managers will respond more effectively to a structured, thoughtful discussion of why some troubled programs fail and some improve through a variety of programs, both public and privately held, he said. Grasso recommended a cross-government Program Management Officers Council, similar to the existing CIO Council and Chief Financial Officers Council, where program leaders could work together to establish government standards and share best practices. OMB could sponsor workshops on program and project management, highlighting program cases as examples and meeting in private venues that would encourage discussion of issues, he said.