Disaster recovery back in spotlight

New Mexico's Information Systems Division (ISD) put together its first disaster recovery plan in the mid-1980s but, like information technology managers in others states, New Mexico officials found themselves focusing on other issues in the 1990s. By the end of the decade, they knew they had a lot of work to do.

Now, after just more than a year of a focused effort to pull together a new plan, the state is entering the final stages of a process that will enable it to guarantee its client agencies that their systems and support functions will be recoverable within 48 hours after any disaster.

"Because of things such as lack of resources and [the Year 2000 problem], in the late 1990s we had begun neglecting the plan," said Tim Kisser, deputy director of ISD (www.state.nm.us/gsd/isd). "We were not monitoring it as aggressively as we should have been."

When the division turned its focus back to disaster recovery, it found that some things had changed. Originally, ISD had managed much of the state agencies' IT needs centrally through its mainframe-based data center. Conceptually, the recovery plan looked the same, but many of the relationships between ISD and its client agencies had changed.

A big difference was that agencies had become self-sufficient for services they had relied on ISD to provide. What had previously been centralized, host-based e-mail, for example, now occurs in client/server environments and is the responsibility of the agencies themselves.

The agencies still look to ISD for "large-scale stuff" such as online processing capability for major development programs. But the division is now more of a service provider for agencies, providing help on an as-needed basis.

ISD is using a "building block" approach to putting its plan together, making sure network connections are established and tested, for example, before dealing with different operating environments.

This approach is something that should be common in most recovery planning, said Bill Langendoerfer, the state's disaster recovery coordinator, but is particularly appropriate to ISD's political arrangement.

"We have areas of unique specializations," he said. "The communications folks are one, another is technical support for software, and so on. This kind of organization allows us to test the recovery of various platforms in one area independent of those in another, so we can see which approach gives us the most productivity in the environment we are in."

The step-by-step approach also has a built-in error check. Each part of the recovery plan is a key component of those that follow, so testing one part highlights any errors and omissions that might have been made in previous parts. Those can then be fixed before any part of the plan is called complete.

"Once we are at the end of that process, we can be pretty confident that we have a good plan," Langendoerfer said.

One potential sticking point has been getting the cooperation of client agencies. Each has its own disaster plan, but they are all intimately involved in what ISD needs to do, either because they are dependent on the division for a particular service or have independent applications that use ISD resources. ISD recovers the platforms the agencies depend on, while the agencies recover the applications.

All of the affected agencies have expressed interest in the ISD recovery plan, Kisser said, although that didn't necessarily pre-ordain their enthusiasm.

"From my perspective, I knew that culture would be an issue," Kisser said. "So we went ahead and built the infrastructure that would be necessary for disaster recovery and when we wanted to test it for a particular application, we put feelers out to the agencies and asked which of them were ready to test."

Agencies are becoming more aware of the need for disaster recovery, Langendoerfer said, due in part to an increased focus on security brought about by Sept. 11 and by federal mandates, such as the Health Insurance Portability and Accountability Act.

Ultimately, the state will end up with a plan that encompasses the needs of all of the government's agencies, Kisser said, including ISD. Although the division is unlikely to be the governing authority for that, Kisser does believe that the plan ISD is putting together likely will be the basis for this statewide plan.

And that would be a sweet recognition of the effort that his organization has put out. Disaster recovery is in many ways an undervalued facility since "it's something no one really wants to do because there always seems something better to be doing," Kisser said.

In this newly security-conscious world, however, perhaps it's what people are increasingly recognizing as something they need to do.

Robinson is a freelance journalist based in Portland, Ore. He can be reached at hullite@mindspring.com.