CDM in the trenches

The Continuous Diagnostics and Mitigation program — a $6 billion effort to better secure networks and systems across government — is a complicated beast. It covers some 169,000 tools and services, and it is managed by the Department of Homeland Security's National Protection and Programs Directorate (NPPD) while the General Services Agency runs the acquisition contract.

Funding is provided to most civilian agencies. It flows through DHS and covers most, though not all, of what's required to deploy CDM. And although Phase 2 of the program is just getting underway, the current contract expires next summer, so current deployment efforts are also informing plans for the next-generation acquisition vehicle.

FCW gathered CDM stakeholders on March 23 to discuss their experiences to date. The discussion was on the record but not for individual attribution; a list of participants is included below. Here's what they had to say.

Show me the money

A common refrain was that many agencies weren't prepared to provide the additional resources needed to fully implement the DHS-funded CDM tools.

One executive said many components in her agency "didn't put a line item in their budget for CDM because they just knew" would NPPD pay for it. They quickly realized that other elements were still needed. For integration, "you've got a program management piece, you've got to deploy it," she said. "Somebody's got to manage it. What about the hosting services?"

Another agency official recalled that "I had to convey to everybody, 'OK, so we're going to get the engineering support with Phase 2, but we're going to have to provide our own hardware.'" That instantly put CDM in competition with other projects, he added, "because everybody needs hardware."

Budget uncertainty has compounded agencies' challenges, several participants said. The hassles that accompany short-term funding are not unique to CDM, but stopgap funding, personnel freezes and likely budget cuts could all hit "at a time where we're starting to transition from the license and maintenance costs that DHS is covering over to the agencies," one executive said.

"I know that the Office of Management and Budget has been working to make sure that the agencies have built into their budget the funding to cover the costs once they've transitioned over to the agency," another said. "We'll just have to see how that plays out."

Participants also wondered whether dedicated CDM appropriations will continue beyond next summer and whether they will continue to flow through DHS.

A desire for ongoing funding seemed unanimous, but one executive noted that not everyone loves the current funding model. "As a buyer of CDM services, my hope is that OMB decides to centrally fund a large portion of it because then my interagency agreement with DHS to receive that funding and place it on contract is a really simple pathway," he said. But CIOs and other agency leaders might be "hopeful that not all that money goes to the CDM program."

"You want to have access to that money because it's ultimately your responsibility as the agency head to provide cybersecurity," he added. "You may want to have some discretionary spend where you can do a portion of CDM, a portion of some penetration testing and a portion for some other purpose."

Security is more than CDM

Participants whose agencies are early in the process expressed concern that CDM implementation could conflict with or devalue other efforts. One official said her agency has been following the security controls in the National Institute of Standards and Technology's Special Publication 800-53 for a long time, and now she is wondering how well those efforts will mesh with CDM.

A DHS participant said the CDM program works closely with NIST, and NIST has tested "a working CDM implementation from implementation of sensors down to the endpoints, fed up through the agency dashboard [and] up to the federal dashboard."

Full coordination is still a work in progress, he added, but a planned part of CDM's Phase 3 "is mapping all of the 800-53 controls [and] everything that falls in the cybersecurity framework against the requirements that we establish with CDM."

Other officials said some agencies must also deal with the pride of ownership that security teams feel. "They're very proud," one participant said. "When you're doing the initial introduction to CDM, you get feedback like, 'We already have tools in place. We're doing this, we're doing that.'"

He recalled a conversation with a network engineer who was concerned because his team had "just deployed something that we spent a million dollars on. [I said,] 'I'm not asking you to get rid of it, but this is what we're going to have in addition.'"

Figuring out how CDM fits into an agency's broader security strategy is essential, another participant said, because CDM doesn't come close to doing everything.

"You need to remember that CDM, in a nutshell, only provides you with a baseline configuration of your environment," the security expert said. "Making sure that their hardware is secure is one level, but monitoring what actually is coming in and what's leaving the agency, I think this is more important."

CDM "is one aspect of the security," he added. "It's not the whole thing."

Sharing lessons when one size won't fit all

Participants from customer agencies raised another friction point: They lack a good sense of how other agencies are approaching CDM, though they also said cookie-cutter approaches are unlikely to succeed.

"We need to do more homework before the contractors can start," one official said. "Each agency has its own requirements, culture, size. It's not one-size-fits-all, unfortunately."

Those who've worked on CDM at multiple agencies agreed. As one executive said, "Those types of processes in terms of change control, the speed at which the changes have to be made and the communications around all that have introduced problems, agency by agency, throughout the whole federal space."

NPPD is collecting lessons learned at the program level, and "in terms of playbooks, I think that's been more at the integrator level, but I think that's a good idea," he added.

Yet those playbooks can be taken too far, several participants said. "Our vendor has already done CDM work at other agencies," said one official whose agency was nearly done with Phase 1 implementation. "They have an assumption that whatever happened in agency one is going to happen in agency two."

When they learn it can take three months or more "to get a user authorized to work on this environment," he said, the vendor's playbook doesn't have an answer.

Agencies must adapt as well, a few participants noted. "Obviously, if you're implementing a new technology and it's something that you're getting through a shared-services type of arrangement, there can be process realignments that need to occur in the way things are done," one executive said.

And when those changes are made, every agency should embrace "this idea of capturing the lessons learned and feeding it back into the playbook for others," another participant said.

Want good CDM? Practice good IT.

Several participants said the biggest challenges they see with CDM are the same ones that crop up with any enterprise IT project.

"What this whole discussion points out to me is that we've got a lot of work to do in modernizing the way we do IT," one executive said. "You don't make a change to a system without going through business process re-engineering. In my experience, pulling federated systems [and] federated organizations together is really about getting in the ditch and developing those relationships so that you can make this work. It's not specific to CDM. It's good management, good IT, good relationship building. That's what gets you where you need to be."

An executive from a large federated agency, meanwhile, stressed the importance of assembling the proper team. "We are very large, and we came to the party late," the official said. "Every component that we have is a snowflake, and it's important for us to manage that culture so that we understand exactly what they do."

The agency created an executive steering committee for CDM that includes not just component representatives, but "also headquarters, our [chief information security officer], our program managers, our ops," the executive added. "Everybody who needs to have a need to know is on there so that when we say something ridiculous, they are here to push back."

Another participant described a similar approach and said it is already paying dividends. "One of the things that we are starting to develop is a release management group that includes the different IT departments — not just security operations, but network operations, desktop engineering, enterprise infrastructure, all of these groups," he added. "They just developed a release management charter. It's a matter of somebody actually taking responsibility for it."

Other participants agreed. "One of the things that has happened because of CDM — and I think this is a plus — this is an enterprise system across the federal government," one said. "I think this system has now pulled them together to really acknowledge what needs to be done. We now have a technology roadmap that says, 'Before you put anything in our environment, this is our plan for putting things in.'"

The contractor conundrum

One of the main problems participants reported is a common one in federal IT: not getting the right contractors.

"When you have a time-and-materials type of contract, you get not-really-qualified IT professionals who come to us from the integrator," one agency official said. "They don't really have full comprehension of what it takes to configure a system."

"We're actually going through that right now," another participant said. "I just had to have that type of phone call with a vendor because they provided us some engineers who were not familiar with the federal product."

All the hours were used up on researching the problem, and only when the agency made a fuss did the vendor send someone with the necessary expertise, he added.

"I ditto that," a third participant said. "We have contractors who don't really have the skill sets to do the job. And there are a lot of things in our current task order that the CDM vendor is not required to do but we as an agency are required to do."

The group agreed that part of the problem is the thicket of stakeholders involved in the CDM contract.

"We have contractors coming on board, but the contractor's not my agency's contractor," one executive said. "I have to convey, 'Look, this is a DHS contract. It's not our agency's contract, and it has a GSA number on it.'" That creates confusion about expectations and problems in getting personnel cleared to work on an agency's systems.

Several participants said agencies need more control over vendors. Although DHS and GSA have contractual authority for CDM, one official said, "we have the responsibility and the accountability to get it delivered. That is a huge issue."

A GSA participant said the tension is not imagined — it's written into the CDM contract.

Agencies "want this CDM solution," he said. "They want it quick, they want it fast, they want the most aggressive technologies. But GSA, on behalf of DHS, has negotiated very tight cost controls with the integrators that are coming to us. The prices that the government is paying for the tools that they're bringing to bear are unheard of across the federal space."

In return for those prices, he added, the integrators have certain expectations about agency-provided resources and standardization.

"When an agency says, 'No, we don't do it that way. Here in our agency, we have this federation, and you have to go through this change control board,' now you start increasing their labor, which doesn't jibe with the deal that they have," the GSA official said. "The deal they have with the government is quick in/out, best prices available."

Another GSA participant added, "That's why we're looking forward to a new type of contract solution than what you've got."

What comes next?

GSA has started discussions about the acquisition vehicle that will power CDM after next summer and is stressing two goals: a stronger emphasis on services and a better way to blend funding streams.

"During Phase 1 and Phase 2, we were purchasing with a heavy tools focus," a GSA participant said. "We were buying commodities and then an integration component to go along with them."

Once those commodity tools are in place, the official said, "it's a different approach to contracting. Rather than contracting for commodities, we're going to be contracting for integration services [and] labor, and we're going to try to find the best integrators with which we can establish these longer-term relationships."

And referring back to the question of whether DHS or customer agencies should control the CDM dollars, the GSA officials said their goal was to make that question moot. As one participant put it, "I'm trying to create a contract vehicle that, regardless of where the funding goes, you can get to the same contracting, get to the same integrators, that you have a consistent set of service providers and a consistent set of solutions to the agency."

"If we do this acquisition one time this year and allow for multiple funding streams and multiple solutions, it can be a provider that you can leverage for years to come," the official said.