Net tool key to Hubble project

In order to keep up its steady flow of spectacular space imagery, the Hubble Space Telescope ground system requires a lot of behind-the-scenes network service—the equivalent of 120 Ethernet networks linked together over a high-speed backbone—to support the myriad of people involved with the program.

Although HST was launched nine years ago this month, the network continues to evolve, supporting a growing number of users and ground control systems. Over the last year, NASA and the HST prime contractor, Lockheed Martin Corp., Greenbelt, Md., increasingly have relied on a network monitoring system from Concord Communications Inc., Marlboro, Mass., to track the health of its network.

The task is critical, because the network supports a large number of users and applications. The Hubble Space Telescope Network, or HSTNet, includes some 2,000 machines located in multiple buildings at the Goddard Space Flight Center, Greenbelt, Md., as well as three external sites—the Space Telescope Science Institute at Johns Hopkins University, Lockheed Martin's support facility and an off-site software development facility.

Although the HST program originally shared a 10Base5 network on the Goddard campus, the program's performance, reliability and security requirements necessitated a move to a higher-bandwidth, more modern architecture.

"We needed bandwidth and we wanted to segregate our traffic for security reasons," said Wynn Watson, HST ground systems development manager for NASA. "We provide large stores of scientific data online," amounting to up to 4.5 terabytes daily with half a million hits per day on the program's public World Wide Web servers, he said. External and internal traffic volumes have gone up exponentially in the last few years, Watson said. "We need to support all of it simultaneously."

HSTNet is designed to support NASA's rewrite of HST ground control applications, handle administrative tasks, and, especially, support the mission of providing science data to the world. Lockheed Martin has used Concord's Network Health software to monitor routers, Ethernet switches and other network components as well as to report on system health as a whole.

A key strength of the Network Health product is its ease of use, said Daniel Carrick, a network systems manager with Lockheed Martin. He generates weekly 17-page reports using a browser interface to pull down data from a dedicated Concord server. "Health and safety issues can be identified immediately with their health tool," he said. Individual screens show current problems, traffic volume and availability issues. Using Simple Network Management Protocol, Concord continuously polls the network devices, extracting performance data.

The Lockheed Martin team originally bought Network Health about a year ago, when they needed to produce quick reports on the performance of the network, Carrick said.

Since then, the network-monitoring software has helped Lockheed Martin support new applications, including a new HST ground control system, which became operational about two and a half months ago, Watson said. The new system was designed to provide new features, such as graphical rather than numeric presentations of spacecraft instruments and telemetry monitors, and to get away from an outmoded infrastructure based on minicomputers.

To keep an eye on performance, Watson requires detailed weekly performance reports with daily statistics on each local-area network and network device and the system as a whole. Watson looks for information such as trends in network usage, bandwidth problems and other issues that might affect overall performance.

Network analysis, based on historical data, is Concord's forte, Carrick said. "We monitor with Concord to make sure we're not overutilizing any of the connections." At one point, for example, he used the package to show the need for more bandwidth on a connection with the Space Telescope Science Institute.

"We've also used [Concord] for trouble-shooting things that are not really network-related," Watson said. For example, the software helped uncover some design issues with new client/server applications. Performance on a server might dip because "many different processes are simultaneously accessing the server's network interface card," he said. NASA was able to see that some of the new applications "weren't working exactly correctly," and then the agency applied fixes, he said.

Engineers are developing virtual private networks that connect the Kennedy and Johnson flight centers to HSTNet.

New Project

NASA said the software will help support a mission this October, in which astronauts, lofted to the satellite by the space shuttle, will upgrade the telescope's main flight computer and replace gyroscopes.The consequent outpouring of telemetry and video data will put additional stress on the network, Carrick said. During the mission, HST will be "streaming video data out from the orbiter in real time" for the first time, Watson said. The Johnson Space Center will downlink the video feed and broadcast the data to the HST people at Goddard, who will then distribute it over HSTNet, he said.

Lockheed Martin ran some video streaming tests last year, with up to 60 nodes watching video, and, using Concord software, it found that total bandwidth use increased by no more than 3 percent, Carrick said.But the flood of telemetry data will be more substantial, potentially exceeding 40 percent of the network's total available bandwidth during peak transmission times, he said. The contractor is getting ready to run a set of simulations to monitor performance on key ports and interfaces in case there is a need for extra capacity.

Network Health is used by other agencies, including the Census Bureau, the Defense Department and the National Oceanic and Atmospheric Administration, said Christine Washburn, Concord's director of marketing. "It tells you overall how your network is performing and gives you historical data so you can predict problems." The product aggregates and summarizes performance data but does not make recommendations, she explained.

NASA also uses other software tools to monitor and manage the network, especially for real-time surveillance, a capability Network Health does not provide, agency officials said.

Hewlett-Packard Co.'s OpenView, for example, provides a map of all the network components and indicates whether they are up or down. Also important are NetCool by Micromuse Inc.—which is tied into OpenView—as well as systems and application management tools from Tivoli Systems Inc. and BMC Software Inc.

-- Adams is a free-lance writer based in Alexandria, Va.