Is cloud computing fast enough for science?

Early results from the Energy Department’s Magellan cloud computing testbed suggest that commercially available clouds suffer in performance when operating Message Passing Interface (MPI) applications such as weather calculations, an official has said.

“For the more traditional MPI applications there were significant slowdowns, over a factor of 10,” said Kathy Yelick, division director for the National Energy Research Scientific Computing division. NERSC is partnered on the Magellan cloud project with the Argonne National Laboratory.

Under the Magellan project, DOE is exploring many aspects of the application of clouds to scientific research, including performance, efficiency, and suitability for various types of research. At least six different tests currently are being carried out in the two-year $32 million project, Yelick said. About 3,000 NERSC scientists and other people have been invited to participate in Magellan.

Magellan explores the cloud as a research tool

Feds could get access to advanced cloud computing technologies

“The big performance differences were in the MPI applications,” Yelick said.

However, for computations that can be performed serially, such as genomics calculations, there was little or no deterioration in performance in the commercial cloud, Yelick said. Magellan directors recently set up a collaboration with the Joint Genome Institute to carry out some of the institute’s computations at the Magellan cloud testbed.

Magellan’s purpose is to explore a wide range of scientific issues regarding cloud computing, and ultimately to advise DOE how to incorporate cloud computing into its research, Yelick said.

“Some people want to understand the performance factor: seeing how fast the cloud will run. Others are really interested in the impact of virtualization and to what extent can we run multiple jobs on the same node without a performance impact,” Yelick said. Although commercial cloud providers give an “illusion of elasticity,” in reality there are limits on performance and some jobs are sharing the same processors, she added.

Currently, scientists with computational research needs typically purchase and configure computer clusters. But NERSC is hopeful that some of that research could be performed in a cloud, Yelick said.

“We’re really looking at this as an alternative to scientists buying their own clusters,” Yelick said. “Our goal is to inform DOE and the scientists and industry what is the sweet spot for cloud computing in science; what do you need to do to configure a cloud for science, how do to manage it, what is the business model, and do you need to buy your own cloud.”