Storage optimization: Flash finds some government niches

Storage performance can spell the difference between a successful technology venture and one that brings systems to a standstill. Storage optimization tools, including software-based accelerators, aim to reduce or eliminate that bottleneck.

Another approach that promises better storage performance is flash technology, which is familiar to anyone who has ever used a USB thumb drive. In the enterprise context, flash technology takes the form of cards that plug into servers or solid-state drives housed in storage arrays.

Although storage optimization can jump-start a stalled IT system, the extra performance comes at a price. Indeed, the higher cost of flash compared to disk storage has limited its use outside the consumer space. But more favorable flash economics are spurring broader adoption.

The New York Army National Guard, for example, uses flash technology to handle the storage challenges of its virtual desktop infrastructure (VDI) deployment.

"The price of solid-state drives is definitely falling, so [flash storage] becomes more attractive by the day," said Clarke Caporale, the guard's information assurance manager. "In [the Defense Department], we have to collapse down the budget. We are actively trying to get these [flash] resources, normally in the hands of users, into the data center as a cost-saving measure."

Why it matters

Industry experts consider VDI a classic storage drain. Users access remotely hosted virtual desktops instead of resources housed on a local hard drive, which means operating systems, applications and data reside on a server. Therefore, storage performance plays a crucial role in whether a virtual desktop can offer a user experience comparable to a conventional PC.

Another VDI storage wrinkle is the uneven nature of user demand. When employees arrive at work in the morning and all start accessing their desktops at the same time, the so-called boot storm can put a severe strain on storage and degrade the user experience.

Storage optimization, however, can help organizations avoid performance problems. The New York Army National Guard used flash from the beginning of its VDI project — first testing server-based flash technology during a pilot phase and then incorporating flash-based solid-state drives into Dell EqualLogic storage-area networks.

Caporale said the VDI's performance equals or exceeds that of traditional desktop and laptop PCs. "I don't think we could have done that without solid-state drives," he added.

VDI is well-suited to flash technology, but other performance-sensitive applications also stand to benefit from faster storage. For example, Lawrence Livermore National Laboratory (LLNL) is exploring flash technology in a couple of areas — in one case, by using flash to supplement computing cluster storage. LLNL handles high-performance computing tasks on Linux clusters, some of which have thousands of compute nodes. The clusters rely on a parallel file system to support large-scale storage, and LLNL employees periodically save data at various checkpoints in case a computing job fails to complete.

The frequent saves put pressure on the parallel file system, but perhaps only one out of 10 checkpoints are important enough to save, said Matt Leininger, deputy for advanced technology projects at LLNL. Therefore, lab officials would like to store the bulk of the saves with flash technology and reserve the parallel file system for the more strategic saves.

"We're looking at how we can include flash as a storage hierarchy between the parallel file system and compute nodes," Leininger said.

Jim Damoulakis, chief technology officer at storage consulting firm GlassHouse Technologies, said such special-purpose applications and VDI are key areas where adding solid-state storage can have a significant impact.

Software can also contribute to storage optimization. Last year, the U.S. Army Medical Information Technology Center awarded an $8.7 million contract for VDI that includes Atlantis Computing's ILIO storage optimization software.

Some deployments depend on a mix of software and flash-based optimization. The New York Army National Guard, for example, derives some of its storage optimization gains from VMware View, Caporale said. The software lets organizations provide desktops from a central data center and includes a storage acceleration component.

The fundamentals

Flash tools are faster than hard drives because they handle storage chores electronically, while their traditional counterparts rely on moving parts — specifically a spindle motor that rotates disks and a mechanical arm that reads and writes the data.

Flash's lack of moving parts delivers another benefit: lower energy consumption.

However, flash drives typically offer less storage capacity than an electromechanical hard drive. For that reason and others, such as cost (see below), organizations tend to use a combination of flash and conventional storage technology.

In the enterprise world, organizations typically use flash storage on servers, as part of a traditional storage array or as a specialized appliance. Server-side examples include PCI Express flash cards from vendors such as EMC, Fusion-io and Virident Systems.

Storage arrays vendors — including Dell, EMC and NetApp — incorporate flash technology into their products. In addition, Astute Networks, Nimbus Data Systems and Violin Memory offer flash appliances.

Organizations deciding between server-side optimization and an external flash storage device should consider the level of performance required, the number of applications that call for storage acceleration and the management features.

Damoulakis cited scale among the top factors. "If an organization has only one or a handful of applications that require flash, it may be more efficient to simply put cards inside the servers," he said. "However, if this is going to be on a large scale, it might be better to have an array that can provide that service across the board so you don't have to buy a...card for every server."

In addition, if administrators opt to use flash-based cards in servers, they should consider data-management and data-protection responsibilities. Damoulakis said features such as replication are often provided at the storage-service level, so shifting storage to the server side means someone else has to provide those functions.

"Whoever manages the server has to be able to provide that versus leveraging an existing service provided by the storage group," Damoulakis said. The management question becomes a non-issue if the organization already replicates at the server level, he added.

The hurdles

Cost has been one of the main obstacles to flash storage. The price differential, although shrinking, still leaves flash technology at roughly twice the cost of the serial-attached SCSI or Fibre Channel drives often found in arrays. Sam Lee, solution architect at Force 3, said flash storage can cost $30 to $40 per gigabyte, while the drives run about $15 per gigabyte.

On the other hand, the way an organization uses flash can change the equation. A flash drive-equipped array that provides a 10-speed performance boost is more cost-effective than conventional technology on a cost per input/output operations per second, Lee said.

Caporale said aggregating flash as a shared resource on a storage-area network also makes the technology more affordable. The New York Army National Guard uses both flash-only and hybrid storage devices.

In addition, storage optimizers like flash might not solve every performance problem on their own. Damoulakis said the benefits of using flash to deal with bottlenecks might fail to fully materialize for reasons that have nothing to do with storage technology.

"The issue may be in the application and how the application is written," he said. Therefore, executives who want to wring every microsecond of performance out of an application might need to do more than introduce flash drives, he added.

"There are probably going to be application and operating system-level tweaks that will be necessary to realize the full performance gains," he said.

Leininger said one of LLNL's flash efforts has yet to bring huge performance gains, but he thinks it might be a software problem related to the parallel file system, which was written for disk drives. He said the lab needs to rewrite software and middleware to take full advantage of flash memory storage.