DHS Special Report | Component approach aiding IT infrastructure consolidation

Until last October, the Homeland Security Department did not even have a global access directory—a basic requirement for a large organization. Employees couldn’t find each other if they were in different DHS directorates.Now, secretary Michael Chertoff can send an all-department e-mail with a few select addresses instead of sending it to each component agency and ordering it to cascade through those organizations.Once the decision was made to create this simple but necessary directory, it took just one month for the Coast Guard to push it to all the DHS components.Although elementary, the global access directory is an indication that DHS is moving ahead on projects that supply backbone support for the homeland security mission but which have languished in the planning stages.While the global access directory still did not reduce the number of e-mail systems—though that is one of DHS’ priorities—it is part of the department’s IT consolidation that, as recently as a year ago, was a concept without focus.A disciplined process and an organized plan for execution promise to transform DHS’ patched-together networks and data centers into a singular and standardized IT infrastructure, said Anthony Cira, director of Information Operations for the Office of the CIO.The fulfillment of that promise will be better performance across the department, Cira said.“This is like a [corporate] merger and acquisition. You’ve got 22 companies, and it’s how fast can you get them together,” he said.New leadership last year, with the selection of DHS secretary Michael Chertoff and CIO Scott Charbo, and a disciplined implementation plan have shifted the infrastructure transformation forward, Cira said.DHS plans to consolidate six WANs to its OneNet, fold 18 data centers into two and combine six e-mail systems into one, he said. The target date to accomplish this is early next year.The infrastructure transformation now has momentum, said Lee Holcomb, outgoing DHS chief technology officer.“Like a rock, it sits there and doesn’t move. But once it starts rolling, it picks up momentum,” he said.That momentum can be traced to a year ago, to a meeting at which DHS officials shifted the concept of implementing responsibility for the infrastructure transformation from a department-level office to a series of stewards within the component agencies, Holcomb said.“That was the change, driving it down into the component agencies. It lent the credibility of the steward,” he said.The plan has backing from the top: Chertoff sent out a memo dictating the parameters of the program, Holcomb said. Customs and Border Protection was to lead the data center and network consolidations, Coast Guard the e-mail consolidation, and the Federal Emergency Management Agency the video operations.“The alternative [to] the CIO not having control of all the IT resources in the department is you take the next best thing. And that is to get them all in one circle and make them part of the problem. That’s what we did,” Cira said.Officials determined who had the biggest and best chance of success. The governance model, however, is still at the CIO level. CIO Charbo conducts a monthly program review and controls the budget for the transformation effort.The driving force behind this pocket of promise is that DHS set the policy at the department level but pushed the requirements down to the component organizations, said a former DHS official, who requested anonymity. Charbo has led the move away from doing modernization at the department level.While the previous and first CIO Steve Cooper created a vision and direction, Charbo is working on pieces of the vision with the IT infrastructure transformation as a prime example, said Jim Flyzik, a former homeland security official and now president of the Flyzik Group.“Charbo’s philosophy is not to do projects with a big bang but do incremental progress,” he said.In the last two years, DHS also has steadily improved its governance process, decision-making among component CIOs and project management, Flyzik said.The challenge, Flyzik said, is implementing the modernization against a moving set of policies.“They’re being asked to do this while laws and regulations are being passed, and that will impact the technical solutions,” he said.Others have said the lack of effective senior management contributed to the delay in initiating the IT consolidation.“This was a Janet Hale management issue,” said a congressional source, who did not wish to be identified. Hale was the former undersecretary for management who resigned in March and left DHS in May.With Charbo both CIO and acting undersecretary for management, consolidation should pick up, the congressional source said.DHS also is an overstretched agency with a limited number of employees.“It hampers the progress of these programs because they’re asked to do more than the job they’re hired for. It’s fair to say that the lack of staff hampers much progress,” the source said.DHS’ Holcomb also said the shift to implementing transformation at the agency level was made, in part, to take advantage of their project staffs.“The drive to components responds to not having enough workforce,” Holcomb said.To build the first data center, DHS is using the Navy as the co-contractor and program manager. The prime contractor is Science Applications International Corp. of San Diego, through the Millennia Lite contract, a General Services Administration governmentwide acquisition contract, Cira said.“Every system that’s in this department will eventually be there, our crown jewels, so it has to be done well. It will have classified and unclassified data,” Cira said.DHS anticipates ultimately having two active-active data centers, which are mirror images of each other to ensure uninterrupted operations.The first data center will occupy 125,000 square feet, and will combine the operations of 18 data centers that inhabit 167,000 square feet around the country. It is located at NASA’s Stennis Space Center in Mississippi, about 30 miles from the Gulf Coast. Congress stipulated the location of the first data center in fiscal 2005 budget documents.The first 24,000 square feet of the first data center opened last month, with workers moving in the first set of systems and conducting tests. DHS will install the first applications by early July, Cira said, and hopes to have the next 50,000 square feet built by the end of the year.DHS has requirements set for the second data center, for which the House has approved $41 million.Officials expect to release a request for information this summer and depending on the final budget, a request for proposals in 2007, CIO Charbo said.Private industry and the Defense Department have documented that consolidations reduce maintenance and overhead costs by 35 percent, Cira said. Data center consolidation also will reduce overlapping applications.“You can start doing computer-to-computer interface instead of network-to-network interface. Traffic to the department will disappear because it will be internal to the data center. There’s a significant high-speed LAN that will connect all those computers,” Cira said.The network consolidation of six WANs into OneNet is straightforward. It’s a matter of establishing where all the routers are and what’s the configuration, he said.DHS in April used for the first time its operations network modeler, part of its process to confirm performance and validate pictures of the network as well as network design. One of OpNet’s modules is application testing—so, for example, when U.S. Visitor and Immigrant Status Indicator Technology program is ready with a model for software, DHS can run it in the lab and predict the performance of the application. Then they can make adjustments in the real network.DHS plans to use the governmentwide Networx contract when it is awarded next year to convert its networks into a two-carrier multiprotocol label switching (MPLS) environment, similar to that of the Social Security Administration’s network.“Our problem is that, given the size of this thing, we’ve got to do 300-400 locations per month,” Cira said.Until then, the department is cleaning up its networks as much as possible to smooth the transition and taking them from asynchronous transfer mode into the MPLS environment. DHS has locked down its networks for any configuration changes, unless the new configuration control board approves them.DHS also completed its network operations and security operations centers in April. These centers conduct periodic vulnerability assessments and penetration testing.In the fall, the department will start converting the wireless networks and then the voice networks.OneNet also will let network administrators push out patches within 48 hours.“We’ve got to get to that. You can’t push patches when you have a lot of networks. You have to do it manually,” Cira said.Consolidation will drive performance across this department, he said.“Ten fingerprints would have been an impossible task six months ago,” he said, referring to efforts to coordinate U.S. Visit’s two-fingerprint Automated Identification System (IDENT) with the FBI’s 10-fingerprint Integrated Automated Fingerprint Identification system (IAFIS).“If I look at where we were a year ago to where this place is now in network and data centers, it’s not a matter of doing it or making it happen, it’s how fast can you get it done. We’ve already made the tough decisions, the specs and the design. Now it’s conversion and monitoring its implementation,” Cira said. n