NWS forecasting hampered after supercomputer fire

The National Weather Service's central supercomputer used to generate weather forecast models caught fire and subsequently stopped operating late last month, forcing the agency to rely on backup systems and delaying delivery of data to users.

NWS' National Centers for Environmental Prediction (NCEP) had been using the Cray Research Inc. C-90 supercomputer since 1994 to generate and run mathematical models used by weather offices and private firms to develop weather forecasts. However, late last month a fire started in one of the supercomputer's internal power supply units and burned through the top of the system.

The fire eventually was extinguished by Prince George's County, Md., firefighters, but the combination of the fire and the dry chemicals used to extinguish the fire left the supercomputer inoperable, said Louis Uccellini, director of NCEP. Representatives from computer contractor Silicon Graphics Inc. and the National Institute of Standards and Technology determined that the computer had been significantly damaged and could not be repaired.

NCEP is still operating in what Uccellini described as a backup mode, using two smaller Cray machines at NCEP and relying on NWS' Forecast System Laboratory in Boulder, Colo., as well as Air Force and Navy systems, to run numerical weather models.

"We have access to all those operational model runs or access to their product suite in case of an emergency like this," Uccellini said. "We have an arrangement where we have servers that can plug into their operational data stream. We're also accessing information from the European center and Canadian meteorological center to access some of their global model output."

The arrangement has slowed the speed with which the center gets some data to users. The diminished capacity also prevents NCEP from running all the models it does under normal circumstances. For example, NCEP normally runs a global weather model that enables forecasters to make 16-day forecasts. Now the model provides data for 10-day forecasts and must be supplemented by models running elsewhere.

Rodger Getz, president and chief executive officer of Agriculture Weather Information Services Inc., Auburn, Ala., said the delays have had a major impact on his business. AWIS provides weather and climate information to agriculture and other clients who need detailed local weather. AWIS gets its weather data feeds from data providers under contract with NWS and also has developed its own weather models.

The company now must run some models twice, first using preliminary information it has collected and then using the latest NWS models. "This puts quite a strain on our computer processing load," Getz said.In general, the loss of the Cray supercomputer has caused numerous problems, Getz said. "We use National Weather Service forecast model outputs as part of our forecasting outputs," he said. "Their models coming out many hours later than normal means our forecasting has to be pushed back as a result."

The destroyed Cray will be replaced with a faster IBM Corp. RS/6000 SP supercomputer in the Census Bureau's Bowie Computer Center in Bowie, Md. NWS awarded IBM the $35 million contract last year and plans to have the system in production mode by Nov. 15.