Rethinking Plan B

From a disaster preparedness standpoint, Sept. 11 was the second dramatic wake-up call for the National Finance Center. The first was a 1998 hurricane that kicked contingency plans into gear and "scattered employees to the wind," said NFC Director John Ortego.

Unlike NFC and other organizations directly affected by the terrorist attacks, most agencies have not had to shift into emergency mode and rely on backup systems and plans in the midst of chaos.

Still, government agencies like never before are expanding on their existing — and often limited — disaster recovery efforts. The idea now is to plan for scenarios in which key personnel are unavailable and to shield underlying infrastructures and key programs, rather than focus on protecting data and computers.

Traditionally underappreciated, disaster mitigation efforts have mainly been viewed as rote chores and delegated to lower-level employees. Disaster recovery plans focused primarily on procuring backup facilities, or "hot sites," and related services. And such efforts seldom fared well in agency budgets.

Because disasters were considered improbable, the prospect of preparing for them rarely captured the attention of senior officials. Obviously, however, the Sept. 11 attacks have changed how government officials perceive their level of readiness.

Budgets are beginning to look more generous. And overall, there seems to be a heightened awareness of and a greater appreciation for the value of disaster planning efforts.

"Continuity of operations has always been a key issue here," NFC's Ortego said. "But since Sept. 11, we have looked at it harder and tried to re-examine what we are doing."

Ortego is now frank about the sobering reality of risk. "It's like realizing you are mortal, that you can get cancer," he said. "Major financial institutions were attacked and brought to their knees. So you realize that, yes, it could happen."

Continuity, Not Just Recovery

In fact, even before the attacks, disaster recovery was beginning to give way to more comprehensive business continuity concepts. The importance of continuity was underscored during the past several years by the onslaught of computer viruses and some high-profile hackings of major e-commerce Web sites.

The precise differences between the disciplines of disaster recovery and business continuity, however, are not easy to pin down.

"The traditionalists, or at least the purists, say that business continuity and disaster recovery differ in planning, scope and focus," said Terry Rice, technical project manager in CACI International Inc.'s information assurance division.

Some define disaster recovery as the technological response to a major interruption and continuity as the practice of assessing risk.

Others tend to define business continuity as planning that stretches beyond the dramatic incidents of terrorism, fires and floods to more mundane "disasters" such as security breaches and systemic viruses.

"In business continuity, there is also the notion that business processes and [information technology] are interwoven and that you can't separate the two," Rice said. "So even small events are considered disasters, since they can have a significant impact on business."

At the same time, industry watchers, including Donna Scott, a vice president and research director at Gartner Inc., now characterize disaster recovery as going a step further.

"There is a blurring [of the line] between disaster recovery and what I would call high-availability services," Scott said. "Certain types of government services, such as health and safety and even some e-commerce services, can't be down more than an hour or two, or that is considered a disaster."

As the term implies, high-availability computing entails distributed systems and technologies that proactively safeguard the systems and infrastructure behind critical services.

Many agency officials and vendors seem to be using disaster planning terms interchangeably, and distinctions may largely boil down to semantics, Rice said.

"Ask 10 different people, and you'll get 10 different answers," he said. "I would argue [that] it's all academic."

Working Around People

Regardless of the label, sound disaster planning is now likely to center around methods to sustain operations and capitalize on an agency's knowledge base — with or without key personnel.

Secondary concerns include how to preserve data, secure backup facilities and create alternate networking strategies. (See sidebar, Page 20, for details on traditional and emerging disaster planning technologies.)

In a resounding way, the Sept. 11 attacks drove home the grim importance of developing plans that do not hinge on the participation of a select few individuals.

"On the IT side, folks are asking, 'What is the chain of command?' " said Bill Keller, vice president of consulting at Gartner. "This is not just for cases in which people die. What happens if transportation is cut off and planes are grounded? Who runs the show when the senior networking person is out of town and can't get back?"

Ortego experienced many of these issues when Hurricane George struck in 1998 and NFC's IT staff scattered — along with almost everyone else in the battered New Orleans area.

When disasters strike, an agency's ability to respond often hinges on factors that are easily overlooked, Ortego said, such as whether team members have been assigned Web-based e-mail accounts in the event of a massive intranet failure.

Now armed with Microsoft Corp. Hotmail accounts, NFC employees "can communicate [via] their laptops by walking into any library or motel where they've found refuge," Ortego said.

"It is all about continuity. We've got to work even though we are scattered to the wind," he said.

Developing that mind-set, however, will not be easy given government's attitude toward telecommuting, Keller said. "A lot of organizations [impacted by the terrorist attacks] had not allowed employees to access systems from afar," he said. "Telecommuting has been controversial in government, which hasn't trusted employees. And [on] Sept. 11, that was a disaster."

But in the event of a disaster, employees may be dealing with more immediate concerns for their own safety, so more and more agencies are turning to vendors for solutions that include backup manpower.

That way, at the onset of an event, contractors can jump-start restoration "scripts," or preformulated plans for recovery, said Tom Sobocinski, a federal account executive with SunGard Recovery Services LP.

New Ways of Doing Business

There are also reports that agency officials are contacting their counterparts at other government sites in an effort to negotiate using their resources in a disaster.

All of these scenarios are designed to buy an agency more time to reassemble teams during crises. In the aftermath of Sept. 11 and the subsequent anthrax attacks, "many agencies are asking, 'What happens if we are not here?' " Sobocinski said.

Along with a new approach to vendor-based emergency backup systems, the focus on disaster preparedness should also bring increased government funding for recovery plans, some industry experts said.

"It's not that there hasn't been a large number of regulations and guidelines," said Cole Emerson, chairman of DRI International, a group formed in 1988 to promote business continuity standards and education. "It's that there has been an ongoing problem with funding."

Vendors and federal managers, including Ortego, see signs that the Office of Management and Budget is planning to incorporate more funding for disaster planning.

"I would say that government is well ahead on business continuity, as it should be," Keller said. "Hopefully, government will be able to have a huge, positive impact in this area."

Still, time is of the essence. "To be honest, with every day that goes by without a terrorist attack, there will be more of a tendency to ignore lessons learned," he said.

Jones is a freelance writer based in Vienna, Va.