EVM sounds the alarm

Tim Young likes to tell audiences about being a project manager early in his career when he had a demanding boss. The boss told Young he would fire him if he couldn’t report at a moment’s notice whether projects were on schedule, on budget and within scope.

Young, who now serves as associate administrator for e-government and information technology at the Office of Management and Budget, said OMB won’t be as tough on federal agencies as his former boss was on him. New OMB policy guidelines, however, require agencies to become more disciplined about how they manage major IT projects. With the federal government spending about $65 billion a year on IT, agencies should make every effort to prevent IT projects from failing and wasting taxpayer money, Young said.

OMB officials have asked agencies to avoid costly failures by using a project management discipline known as earned value management. With few exceptions, EVM is not well-understood outside the Defense Department. One exception is the Office of Personnel Management. OPM, which recently made EVM a standard for managing IT projects, has learned valuable lessons about EVM’s benefits and limitations.

Setting up EVM departmentwide poses challenges. OPM officials found that EVM is worthwhile because it tells them when projects are in trouble, and it does so in time for them to fix problems. But EVM is not infallible. Most experts say EVM cannot help agencies that won’t accept bad news. They say EVM is most valuable if agencies use it to help people learn from their mistakes rather than to punish them.

OMB issued its EVM policy guidelines in two memos dated August 2004 and August 2005. In addition to requiring federal agencies and their contractors to use EVM for managing all major IT projects, OMB set new reporting requirements. Agencies must include EVM data when they submit Exhibit 300s, documents in which they present their business cases for major IT projects. OMB requires agencies to use EVM to calculate and report each project’s estimated total cost and completion date.

Agencies must answer other EVM questions. Is the date that project managers planned to complete certain work on a project months earlier than the date they actually completed it? If major gaps exist between the work that the agency planned to do and the work it actually did, OMB wants to know that, too.

Likewise, OMB asks for EVM data that reveals significant differences between how much the agency planned to pay for work and how much the work actually cost. OMB puts a major IT project on its watch list of projects requiring corrective action if it is off by more than 10 percent from the original schedule, budget and functional assumptions.

OMB’s 10 percent rule is reasonable for all but the largest projects, said Joel Koppelman, founder and chief executive officer of Primavera Systems, a project portfolio management software company. “If I were doing a $10 billion project, I wouldn’t want to wait until I was $1 billion off” before taking corrective action, he said.

The underlying principles of EVM are not new. DOD has been using cost and schedule controls on aerospace and defense projects since the mid-1960s. In 1991 DOD canceled the Navy’s A-12 stealth carrier aircraft project because EVM reports revealed the project — by then $1 billion over budget and a year behind schedule — could not recover. “It certainly got the attention of a lot of program managers who were not using EVM as a tool for managing their projects,” said David Christensen, accounting professor at Southern Utah University.

Despite early doubts, most experts now agree that EVM is suitable for managing major IT projects, Christensen said. “People for many years felt that IT projects were not measurable and that you couldn’t apply earned value to those projects,” he added.

EVM gives agency executives who sit on IT review boards a means of identifying IT projects that are likely to exceed their budgets and deciding whether those projects deserve to get additional funding to put them back on track, experts say. As a set of standard practices for managing projects, EVM makes it difficult for project team members to cheat or claim more progress than they have made.

EVM standard practices are described in an American National Standards Institute document known as ANSI Standard 748-A. “It’s a whole system of checks and balances,” said Elizabeth Mautner, OPM’s director of administration and finance for the human resources line of business, a governmentwide initiative to consolidate federal human resource services. But can someone put inaccurate data into the system? “We try to make it as hard as possible,” Mautner said.

Some experienced project managers say the benefits of EVM are offset by certain limitations. A common criticism of EVM is that it measures the quantity but not the quality of work performed. Another limitation of EVM is its underlying assumption that problems derive from poor project execution rather than inadequate project planning. Sometimes poor planning is the culprit, Christensen said. “If the project plan is not accurate or reasonable, then you’re comparing actual progress against a bogus plan,” he said.

Nevertheless, in various areas of the government, federal agencies are learning how to apply the principles of EVM to manage IT projects. OPM is among the first to make a commitment to use EVM agencywide. After successfully experimenting with EVM to manage five e-government initiatives in 2005, OPM transferred its EVM activities to the office of Janet Barnes, OPM’s chief information officer, and made EVM a requirement for managing all major IT projects.

“EVM has moved from being nice to have to being critical for daily operations at OPM,” said Norman Enger, program director of the human resources line of business initiative at OPM.

Getting to that point was not easy. “This is something that takes a lot of commitment and effort,” Enger said. In September 2004, OPM purchased Primavera’s EVM software, which included Primavera’s Budget Manager, Time Sheets, Cost Manager and myPrimavera, a personal workspace collaboration tool. Then OPM project managers spent a year learning how to get useful reports from the new system. “It takes that much time to put the infrastructure in place and change the culture,” Enger said.

One significant change was having OPM employees and contractors record all labor costs and other direct costs in the EVM system. “Contractors are not ecstatic about putting their labor hours and [other direct costs] into the EVM system,” Enger said.

EVM required integrating the Primavera system with OPM’s older financial systems — another significant effort. “This is the type of integration that takes a while, a lot of time and a lot of culture change,” Enger said. That change, he added, couldn’t have happened without senior-level support from OPM’s chief financial officer and CIO.

OPM now relies on EVM to manage IT projects budgeted at more than $30 million a year, and the number of projects is growing, Mautner said. The EVM system generates monthly reports that the agency’s investment review board uses when it meets quarterly to review and prioritize projects.

In April 2004, OPM began offering training to introduce EVM concepts to project management employees and senior executives. It continues to offer refresher courses to keep employees involved and ensure appropriate use of the EVM system.

Project managers use the term “change management” to refer to ways that managers can encourage employees to accept workplace changes. Change management is the most difficult aspect of bringing EVM into an agency or Cabinet-level department, Mautner said. “This is something you have to continue to pay attention to, and you’re never done with it,” she said.

Some companies include change management consulting with their project management services, although agencies typically do not include it in a contract’s statement of work. A change management consultation might begin with an assessment of employee attitudes and include strategies for communicating the value of proposed changes to employees and executives, said Greg Grosser, corporate development director at Macfadden, a professional services company based in Silver Spring, Md.

Change management is the responsibility of managers, some of whom blame an organization’s culture for their inability to get people to do something, Grosser said. When managers choose not to accept responsibility for the culture or even seek help in changing it, not much else can be done, he added.

People disagree about whether the difficulty of implementing EVM is comparable to the challenges of converting from older financial systems to an enterprise resource planning system, for example. “EVM is a lot easier than doing an ERP because it doesn’t involve nearly the number of people,” Koppelman said. But others say that when practiced on a large scale, EVM poses many of the same challenges as ERP. Both require the use of data dictionaries or agreement on definitions before people enter data into the systems. Reaching that kind of agreement among people from a variety of program offices requires many meetings.

Another challenge of using EVM is recognizing that although it is designed to provide objective data about a project’s progress, someone could generate false EVM reports. “If you’ve got powerful people and they refuse to be judged objectively, they’re going to find ways to screw the whole thing up,” said Dennis White, practice manager for performance management at Robbins-Gioia, a project management consulting company.

Other experts say the biggest danger in using EVM is that some senior executives will dismiss accurate but negative reports because they instinctively resist bad news and try to suppress it. EVM can be a problem for people with can-do attitudes, Christensen said. “They’re too optimistic about their ability to recover,” he added.

He said the way to deal with that problem is to ensure that early warning signs of a troubled project are not used to punish people. “They should be viewed as opportunities for improvement rather than as [occasions] for somebody to be fired,” he said.

EVM is valuable because it can identify problems early enough so that something can be done about them, said Neil Albert, president of the College of Performance Management at the Project Management Institute. Sometimes that means releasing reserve funding for a project, putting more people to work on a critical phase of a project or offering additional training to project staff.

EVM is also valuable at the end of a project when managers have time to review all the EVM data and learn from it, Koppelman said. “Good judgment is the result of experience,” he said. “The way to become a good project manager is by having bad experiences and learning from them.”