Closing a security gap

Until a year ago, the city of Seattle had to provide lots of care to the technology it used for screening its e-mail inboxes from annoying, and sometimes, harmful messages. In the end, the commercial application the city was using wasn’t as effective as the city needed, said Michael Hamilton, the city’s chief information security officer.But today, an e-mail-filtering service hosted by Postini, a subsidiary of tech giant Google, screens incoming messages, traps junk mail, quarantines infected messages and sends a small digest of the daily catch to city information technology staff members. In addition to stemming the spam flood and allowing IT workers to focus on more productive duties, the service has been significantly more effective in stopping viruses. “Local government doesn’t have the resources to throw at a problem like this, so it’s a better value proposition for us to leave it up to the experts,” Hamilton said. “This experience has opened up the city’s IT leadership to the value proposition of managed services.” Managed-services providers have been helping public-sector agencies offload routine IT tasks for years, but until recently, some agencies balked at trusting third parties with something as crucial as security. That’s changing as IT managers in local governments and federal agencies alike are looking to managed-security service providers (MSSPs) to help them rein in costs, cope with shortages of technical talent and keep pace with the endless stream of new vulnerabilities. However, CISOs and others warned that financial savings and protection levels can fall short of expectations when agencies don’t account for hidden costs and service-level agreements (SLAs) lack detailed performance measurements.  “The biggest challenge is understanding exactly what you’re contracting for, and where the risk transfers are in that relationship,” said Ron Ritchey, a principal at consulting firm Booz Allen Hamilton. “It’s easy for an agency to believe it’s eliminated risk, but through the misalignment of the contracted service and the provided service, that might not be the reality.” One measure of growing MSSP acceptance comes from forecasts of federal government spending in the next five years on security services. Public-sector company Input projected a compounded annual growth rate of nearly 8 percent through 2013, which would bring spending to $9.6 billion. The projected uptick comes in part because service offerings have improved, and agencies are increasingly looking to focus their in-house resources on core systems, said Timothy McKnight, vice president and CISO at Northrop Grumman, which includes managed security in its technology portfolio. “It’s not always good to own everything,” McKnight said.Agencies can mix and match secu-rity services ranging from the basic, such as virus and spam filtering, to the comprehensive, with outsourcing arrangements from IBM’s Internet Security Systems, Northrop Grumman, Symantec and others that cover the installation and management of software patches, firewalls and intrusion-detection/intrusion-protection systems (IDS/IPS). In addition, MSSPs can provide early alerts about botnet attacks and virus outbreaks gleaned from data aggregated from all their customers’ security logs.The choice of how much or how little an agency decides to outsource often depends on the depth of its in-house security expertise. Agencies have been more willing to contract for operational components such as IDS/IPS, while keeping overarching security policy development and governance in their walls, McKnight said. The right MSSPs and SLAs offer a variety of benefits, with potential financial savings — sometimes approaching 30 percent or more, initially — and the infusion of security expertise being what agencies look for most often, analysts said. Financially, agencies can avoid costs by optimizing IT staffing. “If you no longer have six people dedicated to security monitoring, it’s then easy to plug in six times whatever you would pay those folks to determine the cost savings in that scenario,” said Kelly Kavanagh, principal research analyst at Gartner. Predictable fees also aid budget planning. Because security-service contracts charge set subscription fees for the contract term, planners aren’t hit with surprise expenses when they need additional staff time or technology to fend off a virus attack or spam storm.Expert advice can come as a welcome relief to IT departments struggling to stay current with the onslaught of new threats, management best practices and technologies. “You don’t have to have an anti-spam expert on the premises to make sure that the filters are updated and the latest patches have been downloaded,” said Adam Swidler, product marketing manager at Google. Otherwise, “IT teams find themselves running around in a daily firefight to try to keep ahead of what’s getting through their spam filters. It’s a never-ending cat-and-mouse game.”For Seattle, the ability to tap outsiders offers relief in the difficult hunt for security talent. “Attracting and retaining highly skilled individuals in the IT sector is exacerbated by the concentration of Fortune 500 corporations in Seattle waving bags of cash,” Hamilton said. In contrast, “our proposition is you get Ground Hog Day off,” he quipped. Other outsourcing pluses for federal agencies can include help in meeting Federal Information Security Management Act (FISMA) reporting requirements. Kavanagh recommended that agencies evaluate a potential service provider’s ability to document security activities. “Ask for examples of the actual reporting they provide to their customers to see if it’s easy to transfer into a FISMA reporting format,” he said. “The ease with which you get that report and the ability to plug it into your FISMA reporting distinguishes one [service provider] from another.”  However, MSSP veterans warned that hoped-for benefits don’t always materialize, especially when SLAs aren’t detailed enough to sync both parties’ expectations, McKnight said. When problems occur, agencies can feel like they’re undertaking something akin to hostage negotiations, he added. His advice for assuring understanding is to seek another form of outside help. “If you don’t have that expertise in outsourcing in-house, bring in a third party that does,” McKnight said. Hidden costs can also siphon away as much as half of the expected savings in a contract. Overhead from managing the agreement is one culprit because of its demands on staff time. McKnight compares the oversight responsibility to a project management office’s role. “You are going to need a group of bodies to maintain that contract and [compare] performance against the contract,” he said.Other cost drains happen after a security breach. For example, Postini offers credits on service fees if a virus breaks through its filters, but it doesn’t compensate for damages. Even in the most extensive outsourcing arrangements, MSSPs don’t typically take full financial responsibility for problems. “Contracts may stipulate a fine of $50,000 or some other nontrivial payment, but the true cost of an attack to most organizations is far in excess of that when you consider remediating the desktops, going through backups and doing the restores,” Kavanagh said.Taking overhead, transition costs and other factors into account, Hamilton said the annual cost of $6.60 per person for Seattle’s e-mail-filtering services is on par with what the city had been spending for licenses and staff time to support its former on-premises application. Although some MSSPs say their early-warning systems provide protection against large-scale outbreaks, Hamilton said he’s skeptical. “I’m not a fan of sending security-event data off-site,” he said. “There are things that we would not want to disclose, even when you know the extent to which the information is being protected. You never want your dirty laundry getting out.”Experts also warned that although hiring outside experts can be beneficial, over-reliance on others can cause in-house security skills to atrophy. In addition to staying current with the latest security trends and technologies to fully protect themselves from hackers, agencies should remain knowledgeable so they can adequately monitor and evaluate MSSP performance. Some liken the danger to what happened to intelligence agencies after the breakup of the Soviet Union in the early 1990s. “A lot of people left the community, and it robbed some of the younger generation of opportunities to hone their skills on the IT and the operational side,” said John Sano, director of business development for the Global Government Solutions Group at Cisco, which works with telecommunications companies’ managed-security divisions.One way to offset this effect is to add formal knowledge-transfer mechanisms into contract requirements. For example, service providers could use regular conference calls and in-person meetings to review incidents and countermeasures in the past months and offer an outlook for threats and technology developments in the near future, Kavanagh said. In addition, “the better service providers, through their Web portals, provide a tremendous amount of information that an interested customer can go into to look at specific events that triggered alerts and at the responses the service provider has taken,” he said. “There is an opportunity for that knowledge transfer to take place, but you have to want it and make that a part of the arrangement.”Finally, because achieving security requires a delicate balance with users’ practical needs, agencies should be wary of MSSPs that don’t incorporate business considerations into their efforts to fulfill contracts, said Andy Singer, a Booz Allen Hamilton principal and recently retired Navy rear admiral who was director for intelligence in the Pacific and deputy of Naval Network Warfare Command. When security problems arose during the initial implementation of the Navy Marine Corps Intranet, the Navy tightened the requirements of its SLA, and the MSSP responded by limiting the capabilities of the attached desktop computers. The result is that NMCI is among the least intruded networks in the Defense Department, Singer said. But the tradeoff was staff productivity. “As director of intelligence for all the Pacific, I sure wanted to know what all our friends and those maybe not so friendly were thinking,” he said. “But with NMCI, they locked me out of Web sites. It was difficult to get my job done.I’d end up using a separate system not connected to the main system. But is it the best thing for a knowledge worker to go down the hall to get the information he needs?”