Long-haul storage

There's no rest for the weary when it comes to storage. Information technology managers who have toiled in recent months to install storage-area networks now must connect them.

Fibre Channel, the predominant SAN protocol, has a distance limitation of about six miles. Yet organizations would like to interconnect SANs over wide-area networks for remote data mirroring, backup and file sharing. The potential for cost savings provides additional motivation. Managing a single fabric is less expensive than operating multiple, independent SANs, or so the thinking goes.

"People want to be able to interconnect SANs," said Paul Ross, director of storage networks marketing at EMC Corp. "They want to get consistency and better utilization of resources," he added, noting that many EMC customers have five or more SANs.

Conceptualizing SAN linkages is one thing, but making them happen is another. Today, government agencies have several approaches for overcoming distance-challenged Fibre Channel technology. Some solutions involve specialized appliances that let data flow between Fibre Channel SANs through a transport vehicle capable of longer distances, such as a Synchronous Optical Network (Sonet). Prices can approach $25,000 for these products. Another solution is Fibre Channel SAN switches that perform SAN extension chores. Such solutions cost more but may eliminate the need to buy additional boxes.

Interconnection strategies typically result in some performance degradation. Vendors, however, offer compression to boost throughput. Another concern with SAN interconnectivity is fault isolation. Organizations want to confine network problems so that a glitch in one part of a linked SAN does not spread elsewhere.

IP-based storage networks provide another SAN interconnection alternative. The iSCSI protocol, for example, permits storage over wide-area networks without any of Fibre Channel's limitations. But this approach raises security issues not found in closed Fibre Channel networks. In addition, industry executives do not agree that the technology is suitable for long-distance SAN linkages.

Why span the SANs?

Industry executives contend that the phrase "SAN extension" isn't on the tip of most customers' tongues. Rather, customers usually have a broader problem to solve in which SAN extension plays a role.

"The bulk of the SAN-extension issues are cropping up to support other efforts in the data center," said Dan Smith, enterprise technology consultant for GTSI Corp.'s storage technology team. The task of bridging SANs is typically undertaken for disaster recovery and business continuity purposes, he added.

"We are...seeing [SAN extension] almost exclusively for data mirroring," Smith said.

The data mirroring process continuously copies data from primary storage to a remote storage device. In synchronous mirroring, data isn't written to primary storage until remote storage confirms it has received the mirrored data. This process not only coordinates primary and secondary storage but also introduces latency because of the checking. Asynchronous mirroring does not wait for confirmation. This method addresses the latency problem but could result in data inconsistency in the event of a network failure.

The Air Force's 45th Space Wing is creating a mirrored environment. The unit has two SANs, one at Patrick Air Force Base, Fla., and the other at Cape Canaveral, Fla. The 45th Space Wing has been working toward cross-site data replication and synchronous mirroring for the most critical data. If there is a failure at Cape Canaveral, data will be readily available at the other site, said Glenn Exline, manager of advanced technologies at Patrick Air Force Base. The SANs are based on Brocade Communications Systems Inc. SAN switches, Dell Computer Corp. servers and EMC storage arrays.

Ross agreed that remote mirroring is the primary application for SAN extension. The next biggest application, he said, is remote vaulting, in which an organization with storage in multiple locations sends all its data to one site for archiving, typically on tape.

Officials in Jefferson County School District in Colorado used this technique to bolster their disaster-recovery posture. The school system's central administrative center pulls in data over a wide-area network from more than 150 locations and backs it up to a support location a few miles away. A storage router appliance from CNT Corp. enables this remote tape vaulting process.

"One of my priorities was to upgrade the tape capability and build into that initiative a way to duplicate some of these tapes at another district facility," said Don Lohman, the district's technical support director.

Backup and disaster recovery aside, some organizations seek SAN interconnectivity to facilitate data sharing. Gregg Pugmire, vice president of business development for LightSand Communications Inc., said homeland security is boosting interest in this aspect of storage networking. Prior to the rise in homeland security concerns, agencies collected data, wrote reports and submitted them. But now, he said, agencies want to see the raw data and not just the report.

"Instead of passing a 100M file that was a report, they want to see the 100s of gigabytes that went into creating the report," Pugmire says. "Those are the applications that are driving the whole desire in government to share data."

The Naval Research Laboratory is one organization that has employed SAN interconnectivity to share data. The lab has evaluated LightSand's SAN extension product to speed Fibre Channel data containing 720p High Definition Television across the Pentagon's Advanced Technology Demonstration Network.

Building bridges

Agencies draw upon more than the typical range of methods when bridging Fibre Channel SANs.

"Federal is more diverse than...the commercial space," Ross said. For example, Asynchronous Transfer Mode (ATM) technology may end up as the most common basis for SAN interconnectivity, which may not be the case in other markets. "The federal sector is probably one of those last couple of places where we find ATM networks," he said.

Yet those networks are important for federal agencies that demand encryption. This type of encryption is less expensive than Sonet encryption, which drives some agencies interconnecting SANs toward ATM, Pugmire said.

LightSand is among the vendors linking Fibre Channel SANs via ATM. The company's SAN extension gateway contains a multiprotocol switch device that resides on an application-specific integrated circuit. The gateway links to SANs through a Fibre Channel port and then connects to ATM networks via Sonet, the underlying transport for ATM. To link SANs, gateways are positioned at each SAN site.

CNT also covers the Fibre Channel-over-ATM space. The company's edge storage router product, like LightSand's gateway, sits between a given SAN and the ATM network.

Beyond ATM, some agencies have opted for SAN interconnectivity via Dense Wave Division Multiplexing (DWDM). The Naval Research Laboratory provides one example. The lab used LightSand's SAN extension gateway to stream Fibre Channel data containing HDTV. The application involved sustained bandwidth of about 2 gigabits/sec through a single wavelength of a DWDM circuit, according to LightSand officials.

Optical approaches such as ATM, Sonet and DWDM improve on Fibre Channel but may bump into distance limitations of their own. "Optical technologies are limited in terms of the distance they can provide," GTSI's Smith said. For most optical approaches, the limitation is around 60 miles, although that distance can be extended with repeaters, he added. Repeaters amplify signals to extend transmission distances.

IP storage grows

A rapidly growing, long-haul alternative for storage is IP. Brian Larsen, senior director of connectivity and extensions at CNT, said IP has taken the lead over ATM and Sonet in the past several months.

"More customers are doing IP transport," Smith said.

He said that some vendors have been using Fibre Channel through IP with their own proprietary protocols. But a move toward standardization has arrived in the form of the Internet Engineering Task Force's Fibre Channel-over-IP (FCIP) specification. FCIP embeds Fibre Channel frames within an IP packet for transport, Smith said. The embedding process is sometimes referred to as encapsulation.

The IP approach may call for the use of specialized appliances and a SAN switch. Jeff Silva, vice president of marketing and cofounder of Maxxan Systems Inc., said such appliances had been the only option for Fiber Channel-to-IP encapsulation. But Maxxan performs encapsulation in software within its SAN switch, eliminating the need for external boxes. This approach saves money that would otherwise be spent on maintaining separate appliances, said Vic Mahadevan, Maxxan's president and chief executive officer.

Be they switches, gateways or routers, a range of solutions are available for SAN extension. Those products may bring forth an era in which no SAN is an island.

Moore is a freelance writer based in Syracuse, N.Y.