Feds turn to hierarchical storage management to migrate data

As the federal local-area network culture matures and storage requirements multiply, users are demanding automated storage management. Rising to meet the demand are new, integrated and robust hierarchical storage management (HSM) products that promise greater efficiency at affordable prices.

HSM removes the hands-on involvement previously required in migrating less-often-used files from faster, magnetic storage to slower but less expensive optical and tape media. HSM makes migration and recall transparent. The shuttling of files is scheduled to occur automatically according to preset parameters. Pricing for HSM can go as high as $50,000, depending on the number and type of hardware platforms covered.

"HSM is a way to simulate a very large disk farm by using a small set of magnetic disk drives with a large number of optical disk drives behind it," said Hill Carter, senior consultant with Digital Equipment Corp.'s Federal Government Region in Greenbelt, Md.

An Emerging Market

The HSM client/server market is "still preliminary," said Jeffrey Finkle, general manager of Cheyenne Software Inc.'s Unix Division, Roslyn, N.Y. Cheyenne offers HSM for Unix and NetWare. Industry estimates of the 1996 HSM market range between $20 million and $200 million.

Whatever the estimate, the market is experiencing an uptick in the federal sector, according to industry executives.

"Some of the strongest demand is in the federal market," said Tom Joyce, product manager for EMC Corp., Hopkinton, Mass. "Many [federal] customers have been using HSM technology more aggressively and for a longer period of time" as a result of storage-intensive applications such as document imaging.

EMC in March introduced a high-end, turnkey, hardware/software HSM solution, based on Epoch Systems software. Known as the EMC Data Manager, the integrated, HSM/backup Unix solution is catching on in Defense and intelligence entities. The product is sold through such companies as Electronic Data Systems Corp., Government Technology Services Inc. and Northrop Grumman Corp.

Federal customers are looking for integrated HSM/backup capability, according to vendors. "When we see [requests for proposals] for backup or HSM, a key question is, `Do you offer integrated backup?' " said Julie Stewart, manager of national accounts for Alphatronix Inc., an HSM developer in Research Triangle Park, N.C.

The company's Unix-based, enterprise-oriented Inspire family of migration, backup and related tools is used in NASA's space shuttle program as well as in Defense and intelligence agencies.

But while the "government is using HSM software extensively," there are cases where it "may not be the best solution," Digital's Carter said. The FBI, for instance, uses "custom retrieval software for searching across optical disk drives."

Digital's HSM Unix product is available under its PolyCenter systems management product line. However, "there's a lot of similarity [among] the various Unix packages," Carter said. "Normally, the application initially drives the [HSM] opportunity." If users are employing Digital's Alpha servers and optical jukeboxes, they might want to use PolyCenter's HSM too.

Competitor Hewlett-Packard Co. provides integrated capability with OmniStorage, a module within the company's OpenView management platform, said Bill Dwyer, HP's product marketing manager. HP's solution is aimed primarily at large environments, managing hundreds of gigabytes to terabytes, he said.

Wang Laboratories Inc. acquired Avail Systems Corp., an HSM vendor targeting Novell Inc. NetWare environments. The company, now know as Wang Software Colorado, is expanding into Microsoft Corp.'s Windows NT. The Army, the Energy Department and the Environmental Protection Agency use Wang's HSM products.

High-Volume HSM

Mainstream HSM can meet the needs of the most demanding applications, others contended. Lockheed Martin Corp.'s facility in Moorestown, N.J., for example, realized that its "volume of data and on-line requirements" required HSM, according to Joe Anderson, a senior engineering staff member.

But three or four years ago there were few off-the-shelf HSM choices, and those were very expensive. Because of this situation, Lockheed Martin, a Navy contractor, started to develop its own HSM system, Anderson said. That system was for a "very narrow application" of migrating and recalling recorded radar signals, he added.

When that attempt failed, however, Lockheed Martin turned to Platinum Technologies Corp. Lockheed's application, which uses Platinum's integrated HSM and backup capability, combines magnetic, optical and tape storage media. Specifically, the storage devices include a 36G magnetic disk farm, a collection of eight optical jukeboxes, five tape "data wheels" and off-line tape storage.

Platinum's use of a standard, rather than proprietary, file format is also important, Anderson said. The product is "seamless to the environment."

An even more stressful environment is the Energy Department's physics research lab in Newport News, Va. One of its three experimental halls—when it goes on-line next January—will pump data out at 10 megabit/sec, or 1 terabyte, per day, said Rita Chambers, the computer center manager at the lab.

The DOE facility is using Computer Associates International Inc.'s Open Storage Management software in a version optimized "for the production and collection of physics data," she said.

Another special need for managing high-speed physics data is storing the vestigial stub files off the system hard drives, Chambers said. Data is downloaded to magnetic tapes and then migrated to a tape silo, whereas stub files "go immediately off-line."

The pitch on the savings accrued by moving files to less expensive storage media is not always enough to sell products these days. "The No. 1 customer concern is...integrated HSM and backup," said Dave Demlow, product manager for Seagate Software, Naperville, Ill. Seagate markets Storage Manager 4.0 for Windows NT, which integrates HSM, backup and archiving.

The reason integration is so important to many agencies goes beyond HSM. "People are looking for ways to lower the head count," Alphatronix's Stewart said. "Client/server is generally very personnel-intensive, so they are looking to centralize data management and share resources."

The most popular HSM software offered by federal reseller ISYX LAN Systems, Rockville, Md., is the Seagate integrated backup/HSM solution, vice president Toby Victor said.

Because the Seagate software is "already integrated, it saves on management hassles for the LAN administrator, so the network costs less to manage," Victor said.

Customers are saying the same thing. The National Agricultural Statistics Service at the Agriculture Department moved from an all-magnetic environment to an integrated HSM/backup solution from Cheyenne, using optical jukeboxes, said John Hoge, a computer specialist at the service.

Although the integration was difficult because of hardware/software compatibility issues, it was worth it, he said. "Before, we spent a lot of time compressing things down," whereas now, files can be migrated, and "you don't even know they're gone." But users should take care when integrating HSM and backup systems, industry executives noted.

Disparate HSM and backup systems can lead to undesirable interactions between the two, Lockheed Martin's Anderson said. When the backup product tries to back up a disk that has been "HSM'd," it can trigger the HSM system, which will try to restore the data, he said. This can flood the system and make it "crash and die." Anderson uses Platinum's NetArchive-HSM and NetArchive-Backup, which were designed for integrated operation.

In another scenario, the backup system can keep backing up static data that has been through the HSM system. "Most customers don't want" redundant backups of data already migrated onto long-term storage, EMC's Joyce agreed. "We back up the stub files" on a continuous basis but not the unchanged, migrated data.

EMC claims its product offers deeper backup integration than its competitors do; its backup utility "tracks at a finer level of detail what files have been migrated off," Joyce said.

Unintegrated HSM and backup utilities can also lead to problems in contention for peripheral devices, Alphatronix's Stewart said. "Only one set of software can be in control of a peripheral device," she said.

Micro Design International Inc., meanwhile, offers SSM, or simplified storage management, a concept tailored for departmental servers,said Joe Pollock, sales engineer for the Winter Park, Fla., firm. Users include the U.S. Postal Service and the Smithsonian Institution's Museum of Natural History, Washington, D.C..

Micro Design's SCSI Express allows optical disks to look like native file systems to NetWare servers. Thus, "all the utilities and diagnostic features and tools you can use in association with the hard disk off the server can be applied to optical storage devices too," Pollock said.

With conventional HSM, moreover, requested files have to be "brought back to the hard disk" before they can be used, he said. Because Micro Design allows optical storage to look like magnetic storage, files can be accessed where they are without copying them back, thus saving time. The limitations: The software only works with single-server systems and does not write to tape.

USPS plans to use Micro Design's software at 12 sites nationwide, said Gil Lugo, an information systems specialist at USPS. The product not only allows migration of older files to less expensive optical storage but allows users to look at files on optical directly, without writing back to magnetic.

**

Adams is a free-lance writer based in Washington, D.C.