NASA tackles planetary-size backup job

At the NASA Center for Computational Sciences (NCCS) in the Earth and Space Data Computing Division at Goddard Space Flight Center, the role of tape subsystems that support NASA supercomputers is nothing less than mission-critical, said Ellen Salmon, mass storage team leader at the Greenbelt, Md., facility.

The center has three Cray Inc. supercomputers — two SV1s and a T3E — that are used by scientists to analyze and predict the behavior of the climate, oceans and natural phenomena. Every project is different but most involve huge amounts of data derived from observations collected over more than two decades from data-collection satellites.

The data is refined and processed into first-, second- and third-generation data products, often expanding in size with the application of algorithms and rich content such as digital images, Salmon said. Without tape — and a hierarchical storage management (HSM) system from UniTree Software Inc. that moves data from disk to tape according to predefined policies — the staggering size of the information pool would overwhelm the disks in short order.

NCCS has deployed more than 280 terabytes of data to tape, with the least recently used and especially large files purged from disks as space is needed. The purged data is still technically online and accessible, but requests for the data are processed by the HSM application, which maintains a metadata database noting where purged files reside within the pool of IBM Corp. Magstar 3494 and Storage Technology Corp. 9310 tape libraries.

On request, the data is retrieved from tape by the robotic silos, moved onto a 5 terabyte EMC Corp. disk array, which serves as a disk cache, and made available to the requesting user or application. The disk cache itself is monitored on an ongoing basis to ensure that it is used efficiently.

Salmon credited the HSM and tape solutions for ensuring that data is protected from loss and does not overwhelm expensive disk-storage resources. As data is migrated from disk to tape, a copy is also directed to another tape silo resource at a nearby Goddard Space Flight Center building using Ultra SCSI channels extended over the campus fiber to minimize the risk of data loss.