Finding chinks in the armor

Two years ago, Transportation Department officials were poring over a report generated by the agency's vulnerability assessment software, a tool that scans computer and network devices and outlines potential security problem spots.

"There were piles of paper reports, so we had no easy way to deduce what were the most important items that we needed to fix," said Lisa Schlosser, DOT's associate chief information officer for information technology security.

Department officials were not alone in their frustration. "Users found that while implementing a vulnerability assessment tool sounded good in theory, they were often difficult to deploy in

reality," said Eric Hemmendinger, an industry analyst at the Aberdeen Group. "In fact, a number of organizations become so disgusted with the products that they simply stop using them."

That's because not only were the assessment reports difficult to assess, the tools ate up valuable system resources and took hours, even days, to scan all the systems. As a result, the products were not always viewed as practical for the large networks that agencies such as DOT, which has 60,000 users, support.

Due to these shortcomings, the vulnerability assessment market has progressed in fits and starts. Some early players, such as Vigilante Inc., went out of business, while others, such Network Associates Inc., stopped selling vulnerability assessment products.

Suppliers who were not driven out of the market are now focusing on making products easier to use by automating assessment functions and helping administrators prioritize problem areas. Integrating the software with other products, such as patch management systems, is another new direction.

Besides off-the-shelf products, agency officials have other options for getting the job done. Several good open-source tools are available for free, and vulnerability assessment services-for-hire exist for those who don't want to buy and manage their own

software.

Although tools are improving and agencies have more options, government networks are increasingly complex and will be a challenge for even the best scanning solutions. That's why government IT managers are learning that how they use these tools is just as important as which solution they choose.

Test and verify

In the past few years, legislation such as the Government Information Security Reform Act of 2000 and the Federal Information Security Management Act of 2002 have required federal agencies to do routine assessments of their security systems. Officials at the Office of Management and Budget are pressuring agencies to make sure the testing happens. And last fall, the National Institute of Standards and Technology released guidelines on how to test systems. Included in the suggestions was a specific recommendation that agencies use vulnerability assessment tools.

Such tools can help IT administrators make sure that systems remain secure by proactively searching for potential system and network weaknesses. Among the problems they root out are unprotected systems, open files, misconfigured operating systems and firewalls, and application security holes.

"Once it has the vulnerability information, an agency's security team can close its security gaps so no information is compromised," said Mike Russo, chief information security officer at Florida's State Technology Office in Tallahassee.

Although the potential benefits are easy to discern, realizing them has not been as straightforward for a number of reasons, starting with the growing complexity of desktop PC and server configurations. "In our database, there are more than 58,000 vulnerabilities, and the number increases almost daily," said Ronald Van Geijn, director of product marketing at Symantec Corp.

Tracking vulnerabilities is also an issue. For example, a vulnerability assessment vendor will identify a problem, but the application or operating system supplier will disagree with the deduction.

"I tend to believe the security firms because the other vendors have a vested interest in making it seem like their products are safe," said Jim Buston, IT director for the city of Auburn, Ala.

Vendors may not want to admit a potential breach exists because no fix is available. "In a few cases, vendors rushed fixes to market and created more problems than the potential vulnerabilities," Hemmendinger said.

In addition, assessment tools are not foolproof. "There can be a number of false positives with these products," Van Geijn said. Because a vulnerability scanner is an outside probe, it cannot examine everything about a computer or a network device.

In most cases, the scanner makes a best guess about how a system is configured, but it cannot tell if a system definitely has a hole. Instead, it reports the likelihood of one. So it sometimes reports a vulnerability that has already been fixed.

Also, assessments provide only a snapshot of an agency's systems. Many computers are active around the clock, and configurations could change minutes, or even seconds, after a scan is run. So how often should an agency scan its systems? "We run our scanner once a week, so there is a report outlining any vulnerabilities on my desk every Monday morning," Buston said.

To be safe, other users incorporate multiple scanning tools. "We rely mainly on [Internet Security Systems Inc.'s] Internet Scanner to examine the status of our network, but we also use Nessus [an open-source tool], just to make sure we haven't missed anything," said Ben Chisel, IT critical infrastructure program manager at the Commerce Department in Washington, D.C.

But administrators running multiple scanners will often end up with a longer list of potential vulnerabilities than they can address. Helping users prioritize them has been an area of recent emphasis for suppliers, and most products now divide problems into categories.

"We will immediately try to fix a high vulnerability, but won't spend much time — maybe not any — with a low item," said Micah Van Maanen, IT director for Sioux County, Iowa.

Yet even these rankings are not foolproof. "Vulnerability scanners tend to treat servers running different applications but the same operating system in a consistent manner," Hemmendinger said. "Yet an e-commerce server may be much more important than a file-transfer system." Vendors have started developing products that can make such distinctions.

Although scanners identify and prioritize vulnerabilities, users are still responsible for making fixes. This is another area that vendors are working on. "We have been tying our scanning product into leading patch fix systems, so users can just click a button to institute a fix," said Steve Cooker, vice president and general manager in ISS'

public-sector business unit.

The various vendor initiatives have left the market in a state of flux. "Because basic scanning functions have become commoditized, vendors are trying to add value to their systems via enhanced reporting functions and tighter integration with other security products," Hemmendinger said. "As a result, there is a distinct possibility that the market leaders in a few years could be quite different from those currently selling the most products."

Korzeniowski is a freelance writer in Sudbury, Mass., who specializes in technology issues. He can be reached at paulkorzen@aol.com.