NOAA's satellite data hub will be a 'one-stop shop' for priority users

When the National Oceanic and Atmospheric Administration's next-generation environmental satellites launch and become fully operational, they'll produce more than 20 terabytes of weather data per day.

Using existing data processing systems, forecasters from the National Weather Service and Defense Department would face extreme difficulty finding the information they need pouring from their end of the satellites' firehose of data.

The traditional data stovepipes that grow up around individual satellite systems further compound the problem.

But a development project under NOAA's National Environmental Satellite Data and Information Service aims to revolutionize the way priority users can access the new datasets, allowing them to search and subscribe to various products from the range of NOAA's satellite data holdings.

The program, called the Environmental Satellite Processing and Distribution System (ESPDS), was contracted to Arlington, Va.-based Solers in 2010 for $40 million, with partner Lockheed Martin providing its main engineering backbone.

ESPDS will act as a data distribution portal for NOAA's new satellites – the Joint Polar Satellite System (JPSS) and Geostationary Operational Environmental Satellite (GOES-R), worth a combined $22 billion dollars. The system will also compile and distribute data from NOAA's legacy geostationary satellites as well as selected data from Defense satellites and foreign partners.

"The portal is going to allow for the consolidation of data of all the satellite holdings, and allow access by priority users to a one-stop shop of the information-rich products they need out of the giant store of data," said Jamison Hawkins, deputy program manager for the NOAA ESPDS program within Lockheed Martin IS&GS-Defense.

"That effort has to work in order to maximize the exploitation of multi-billion dollar systems in space," Hawkins said. "For forecasters, getting just the right files and information on a particular situation should be as easy as just downloading the song you want from iTunes."

The first GOES-R satellite is expected to launch in early 2016 and will produce four times more continuous data than existing geostationary satellites. A small sampling of the data the craft's six state-of-the-art sensors will generate includes forecast products for volcanic ash advisories, fog products, sea surface temperatures, smoke monitoring and lightning detection.

An air traffic controller at the Federal Aviation Administration looking to route flights away from volcanic ash after an eruption could use the portal to access just the volcanic ash product relevant to the query, without sifting through extraneous data, increasing efficiency and reducing bandwidth consumption.

The result, Hawkins said, is "selective, real-time information and a user-centric way to access it."

At its peak in 2022, ESPDS will ingest more than 20 terabytes of data from U.S. space systems and foreign partners daily, satisfying up to 600 concurrent data transfers from priority users, which include main NWS forecasters.

"The modernization and distribution portal is a complete enterprise modernization which allows critical, real-time users to get true information products as opposed to having to wade through volumes of data to get what they want," Hawkins said. "It was great foresight from NOAA that IT had advanced to the point where the perpetual reinvention of multiple stovepipes of data was not the way to go."

Collected data won't be lost, and should improve existing stored datasets used by climatologists for long-term weather science. The product distribution and access center will store weather and satellite data for a short period of time, likely three to five days, before it is aggregated into NOAA's Comprehensive Large Array-Data Stewardship System (CLASS), the scientific agency's library for environmental data.