Disaster pain pills

A fire in the Cook County, Ill., Administration Building in October 2003 killed six people and displaced more than 2,000 government workers. The fire broke out on a Friday, which meant that the Illinois local government had to find temporary office space by Monday. Officials also had to ensure that residents could continue to reach county agencies via phone. Employee resourcefulness, with a big assist from technology, eased the 48-hour sprint to maintain government services, said Catherine Maras O’Leary, the county’s chief information officer. Workers toiled through the weekend to reprogram the county-owned voice and data network so that agency employees could be reached at their usual phone numbers, despite employees’ reassignment to more than 80 buildings throughout the county. O’Leary said the county had a continuity-of-operations (COOP) plan in place for such an emergency. She also credited the county-owned network for facilitating a rapid recovery from the disaster. “We were able to put people across the entire county and have them maintain their own phone numbers,” O’Leary said. “If we didn’t have our own voice and data network, that couldn’t have been done.”For Cook County and other government entities, information technology and COOP have grown ever more closely linked. A handful of new technologies — some just taking off and others settling into the mainstream — are putting traditionally high-end COOP capabilities well within the reach of most agencies. Read more details about six of the most promising options and decide if some — or all — are right for your agency.Servers and entire data centers can be brought down by fire, flood or other disasters. Having an off-site copy of data becomes critical in such situations. Remote replication to disk is a valuable capability in such situations, but the equipment required has been expensive. However, the Internet SCSI (iSCSI) protocol promises to reduce remote replication costs, making the practice an option for more agencies. Storage-area network products built on iSCSI let customers use existing IP networks as opposed to more expensive, specialized Fibre Channel storage connectivity. Kansas State University uses iSCSI-based remote replication to makes sure its Plant Diagnostic Information System (PDIS) application remains available. PDIS, a laboratory information management system, supports the Agriculture Department-funded National Plant Diagnostic Network. Researchers use the network in detecting pests and pathogens.“For our application, it was really important to have off-site back up of all the data,” said Will Baldwin, associate director of the Great Plains Diagnostic Network at Kansas State. He said replication would have been nearly impossible to accomplish without iSCSI. The school previously tried two other replication methods that failed for various reasons.Kansas State installed two EqualLogic iSCSI SANs: one at the university and the other at Cornell University as a backup. Data is replicated every 30 minutes. Vail, Colo., meanwhile, installed an iSCSI SAN from LeftHand Networks. The city uses that company’s SAN/iQ Remote Copy product. At night, the product delivers a point-in-time data snapshot via a secure virtual private network tunnel through the Internet to a remote site, said Ron Braden, Vail’s IT manager. The objective is to have those snapshots available in case a disaster strikes, he said.Server virtualization partitions a single physical server into multiple virtual servers, each running its own software independent of the others. The technology, typically associated with server consolidation projects, is now emerging as a disaster recovery tool.“We see virtualization for disaster recover as the killer application for virtualization in 2007,” said Bogomil Balkansky, director of product marketing at virtualization vendor VMware. At least a third of virtualization vendor SWsoft’s customers use the technology to support disaster recovery routines, said Carla Safigan, the company’s director of enterprise marketing. Consolidation remains the primary market for now, she said.A primary attraction of server virtualization for COOP officials is its usefulness for controlling disaster recovery costs. Balkansky said traditional methods call for organizations to keep a production site and a disaster recovery site as mirror images of one another. Each side must operate identical hardware and applications. “That process of keeping two sites …completely in sync is a very expensive value proposition,” Balkansky said, citing equipment and labor costs. Virtualization, in contrast, lets organizations recover applications and data on virtual machines as opposed to physical servers. Hardware symmetry is no longer necessary, which reduces costs. VMware officials, citing customer data, said the virtualized server approach can reduce capital costs by 50 percent to 70 percent a year. With VMware’s solution, a virtual machine operating system, application and data are contained in a single file called a vmdk, for virtual machine disk format. Balkansky said that encapsulation eases the task of keeping an application at the disaster recovery site in lock-step with an application at the production site. The Defense Contract Management Agency is using VMware to consolidate servers, but it is also exploring its use in disaster recovery. Peter Amstutz, chief of network design at DCMA, said the agency’s development organizations have begun using VMware for disaster recovery on a limited basis. DCMA has yet to move forward with a more expanded deployment, which would involve the replication of vmdk files to an off-site location. “The benefits would be to have a full [disaster recovery] site that is kept current within an hour of the hot site,” Amstutz said. “This would be a very complicated scenario without VMware.”Government COOP plans often depend on telework as a way to keep employees working when they are unable to reach their offices. But even in emergencies, officials must protect data as it flows in and out of their agencies’ internal systems. A Secure Sockets Layer VPN, or SSL VPN, provides a mechanism for secure remote access. A VPN uses a tunneling protocol to encrypt data as it traverses the Internet from one site to another. The SSL variety provides some flexibility in that regard because it doesn’t require the installation of special software on the user’s computer. As a consequence, remote users can gain access to enterprise applications and data from any computing device equipped with a browser. Such flexibility can be a godsend during an emergency. That feature helped Florida Guardian ad Litem, a legal advocacy organization for neglected children, re-establish services following Hurricane Wilma in October 2005. A number of Guardian ad Litem workers were displaced after the hurricane severely damaged the Broward County courthouse. Government employees worked at home, in other government offices or public libraries, but they were able to access IT resources through an SSL VPN connection built into the agency’s Citrix Access Suite server software. “As long as they have an Internet connection, they can get back into us and get their data,” said Johnny White, Guardian ad Litem’s chief information officer. The agency data and applications are safely housed on a Citrix Presentation Server, located in a hurricane-hardened facility in Tallahassee. For such reasons, SSL VPNs have become an increasingly popular means of remote  access for COOP, said Tim LeMaster, director of systems engineering at Juniper Networks. In those applications, the technology has gained popularity at the expense of the traditional remote connectivity option, IPsec VPNs, which LeMaster said have gradually fallen out of favor. LeMaster said the clientless SSL VPN technology eases the burden on IT managers and help desks. In contrast, an IPsec VPN requires someone to install client software on every laptop and remote desktop and troubleshoot should a problem arise.Online collaboration systems establish virtual workspaces that may serve as COOP rallying points when government workers are dispersed. “If you can build a virtual collaborative workspace out there…it doesn’t really matter where your people are,” said Ray Schwemmer, chief executive officer of CollabraSpace, a collaboration software company. “They can log in anywhere as long as they have network access.”Bantu’s Bantu Platform, CollabraSpace’s CollabraSuite, Jabber’s Jabber Extensible Communications Platform (XCP), and Microsoft’s SharePoint are among the collaboration offerings available to government agencies. Such products harness instant messaging as an enterprise communications tool, enabling groups of people to discuss an event — a natural disaster or other emergency — in real time. Enterprise collaboration wares typically go a step further. They let users know when colleagues are available online. That so-called presence information plays a role in the Capital Wireless Integrated Network (CapWIN). CapWIN supports federal, state and local law enforcement in addition to fire and other public safety agencies in the Washington metro area. CapWIN’s client software, which enables network access via devices such as laptops and personal digital assistants, provides messaging and presence information, said Bruce Barney, CapWIN’s deputy program manager. Users can communicate in real time, and they can consult a user directory that tells them what skills their colleagues have, where co-workers are located and whether they are online. CapWIN uses Jabber for instant messaging and presence capability. Jabber’s technology lets organizations communicate across jurisdictional lines and communications protocols, said Theresa Kloser, director of marketing at Jabber. Forty-seven agencies participate in CapWIN, according to the network’s Web site. “We are interoperable with multiple protocols, and they need interoperability across agencies and devices,” Kloser said.Mass-notification systems have two main functions in COOP. They mobilize first responders, and they notify citizens during emergencies.During situations in which governments may have used telephone trees in the past, notification systems let agencies broadcast messages through telephone, mobile phone, fax, pager and e-mail, among other communications vehicles. Companies such as 3N and MIR3 market mass-notification systems, which the companies install on the users’ premises or sell as a hosted service.Such systems shrink the time it takes to pull together an emergency response. Margi Schmidt, MIR3’s vice president of business development, said Disaster Medical Assistance Teams that have used MIR3 have reduced muster time from five hours to 45 minutes. DMATs, sponsored by the Homeland Security Department, are medical workers designated to provide medical care in a disaster. MIR3’s systems are designed to repeatedly contact individuals through various communications modes until they receive a response. “Relentlessness is important in a crisis situation,” Schmidt said. An increased focus on COOP planning is one factor driving interest in notification systems, said Kate McCurdy, analyst of government technology at market researcher Datamonitor. She also pointed to a general desire among government agencies to boost efficiency.“A lot of agencies see intelligent notification as the way to get the message out,” she said.Notification systems, when used to keep people informed of events, can also ease the pressure on other emergency management systems.  For those reasons, government entities are purchasing reverse 911 systems, which notify residents of emergencies. However, those systems, McCurdy said, are not as broad as intelligent notification systems that can send messages across multiple channels. Reverse 911 systems place telephone calls and then play recorded messages to people within an area affected by an emergency. Voice over IP (VOIP) was originally touted as a bandwidth cost-saver. But in the COOP context, software-based functionality trumps cost considerations. Reinhard Koch, business continuity consultant at Avaya, said VOIP has become a significant factor in COOP planning. Software features enable displaced government workers to register phones at their new locations into the VOIP system and continue to receive calls on their regular phone numbers. “What VOIP does is, to some extent, it frees the worker from the facility,” Koch said.Software offers that flexibility, he added. Koch said VOIP takes the functionality associated with the traditional telephone switch and places it in software residing on a general-purpose computer.Video moves over IP networks, too. Barry Morris, vice president of federal operations at Polycom, said video over IP is a good match for COOP and telework. Polycom offers software that supports videoconferencing for users equipped with a PC and a USB Web camera.