Research Center Preps for $35M Supercomputer to Study Earth as a Complex System

The National Center for Atmospheric Research is poised to unleash a powerful supercomputer next year that’ll help researchers studying a nascent branch of science gain a more comprehensive grasp of the Earth system—and phenomena like solar storms, climate change, and extreme weather events that threaten humans’ home planet.

Equipped with energy-efficient features, NCAR’s machine will replace an existing one that’s currently in operation at the federally funded research and development hub. The supercomputer will cost at least $35 million, and it’ll be built and installed by Hewlett Packard Enterprise this year at the NCAR-Wyoming Supercomputing Center, or NWSC, in Cheyenne.

“We're very much looking forward to getting the new system fully operational and advancing science in ways that will lead to a better understanding of the world around us and help society become more resilient,” Irfan Elahi, director of NCAR’s High-Performance Computing Division, told Nextgov over email Wednesday. As the project director of this specific procurement, Elahi offered a glimpse into what all will go into the system’s production. 

Broadly, funding for the NWSC flows in from the National Science Foundation and the state of Wyoming, via an appropriation to the University of Wyoming. HPE was selected following a competitive open procurement process for the new supercomputer, which will be fully financially backed by NSF and worth $35 to 40 million.  

The name of that new system is yet to be determined. To figure it out, NCAR is hosting a statewide contest for Wyoming schoolchildren to submit ideas.

More than 4,000 individuals from more than 575 institutions spanning the world have tapped into NWSC’s various computing tools and services since the facility opened up almost a decade ago, according to NCAR’s release. Though much of those studies encompass Earth-y science disciplines including oceanography, air pollution, space weather and more, when the modern pandemic hit, NWSC also joined the White House-led COVID-19 High Performance Computing Consortium, volunteering its resources to accelerate research against the global health threat.

Cheyenne, a high-performance supercomputer released for production work in 2017 was a system among those assets made available for that pandemic-fighting work. According to HPE’s announcement, compared to Cheyenne, the new, in-the-making machine is anticipated to be a “significant upgrade.” The sophisticated system is expected to have the theoretical ability to perform 19.87 quadrillion calculations per second.

“That is almost 3.5 times the speed of scientific computing performed by the Cheyenne supercomputer, and the equivalent of every man, woman, and child on the planet solving one equation every second for a month,” NCAR’s release confirmed. “Once operational, the HPE-powered system is expected to rank among the top 25 or so fastest supercomputers in the world.”

In the announcement, officials also pointed to certain energy-efficient features like “accelerated computing with NVIDIA A100 Tensor Core graphics processing units,” or GPUs. Such processors offer more power than traditional central processing units, though those will also contribute to the new system’s computing capability. NCAR noted that GPU computing adoption is better suited for nascent artificial intelligence and machine learning techniques and will also prepare NWSC for the next generation of computing on the horizon: exascale.

“The advances in computing that are captured in this upgrade, and the potential for impactful application of its results, are tremendous,” Ed Synakowski, vice president of research and economic development at the University of Wyoming, said. He also noted the institution’s researchers aim to “make great use of the new system to better understand areas of fundamental and economic interest impacted by flows in the atmosphere and underground.”  

And those energy-efficient assets are, according to NCAR, meant to enable the supercomputer to “use 40 percent more electricity than Cheyenne—which is itself highly energy efficient—despite being almost 3.5 times faster.” It’ll be able to “burst out” to commercial cloud computing services and offer up nearly double the high-performance storage capacity of its predecessor, reaching 60 petabytes.

“The current plan is to continue to operate Cheyenne until July 2022 after which it will be decommissioned,” Elahi confirmed.

Elaborating on what will happen between now and then, he explained that facility prep work has already been kicked off to boost electrical and mechanical capacity to meet levels required by the next and more powerful supercomputer. And so far, selected vendors and NCAR have started to collectively work on the site plan, facilities engineering, and finalizing hardware and software configuration details. 

“The software environment and applications that can use the GPU partition, including machine learning and Earth System models, are being identified and readied in advance of production operations,” Elahi said.

After appropriate assessments led by NCAR’s technical team unfold in August, he noted, the supercomputer “will be crated up and shipped to the NWSC facility, where it will be installed and go through a designated” post-delivery test. And once necessary benchmarking occurs and designated success criteria is met, the system will be accepted. Elahi emphasized that “pandemic precautions will be taken throughout this process to ensure the safety of vendor and facility staff.”

Beyond that point, and further user environment customizations—likely in January 2022—the supercomputer will be opened up for applications and workloads through the Accelerated Science Discovery initiative. Deemed ASD for short, it’s a mechanism that provides large-scale computational resources to a small number of projects for short durations, following acceptance of a new HPC resource. 

“These projects are selected to help put the new system through its paces and pursue scientific objectives that would not be possible through normal allocation opportunities,” Elahi said. 

NCAR’s broader community of users are expected to gain access months after that, in the second quarter of the year. Still, those involved also have a pretty clear idea of how it might be used from the get-go, in efforts to push forward the emerging arena of Earth system science. Scientists in this realm explore the planet as a whole system and hone in on its interconnected physical, chemical, and biological elements and processes that can drastically impact humanity. 

Advanced computing capabilities—like those that’ll stem from this in-the-making system—allow scientists to run a number of simulations at once and provide a "more complete picture of the Earth system” NCAR’s release noted. Such “ensemble modeling” can help quantify possible effects of given events, and inform response-focused officials and policymakers regarding how to confront risk.

The document lists a range of advancements and research topics for which the incoming system could accelerate new breakthroughs. Among several others, they include: more accurate models for the simulation of the subsurface flows of water, oil, and gas; expanded computing power for better representations of physical processes to help forecast wildfire threats and behavior; and deep, three-dimensional simulations of the Sun's plasma flows and magnetic fields supporting solar storm predictions for weather events that can disrupt communications means and power grids.

"More powerful supercomputing is a vital component of the research infrastructure of our nation, enabling scientists to advance fundamental research and deepen our understanding of the complex and interconnected nature of the Earth system,” NCAR Director Everette Joseph said.