DOD’s Long and Winding Road to (Hopefully) Deploying an Enterprise Cloud
Every new delay should “be an alarm that goes off in the hearts and minds of those who are looking at how we can be competitive in the future against China,” according to one official.
The award date for the Pentagon’s latest multibillion-dollar pursuit to provide departmentwide commercial cloud services was recently delayed, pushing out the release of some capabilities to at least early-2023—or roughly 4 years after the time initial deliveries were first slated under the original Joint Enterprise Defense Infrastructure cloud contract.
While Defense Department officials argue this prolongation is necessary to conduct due diligence for a modern multi-cloud contract that can align its capabilities with industry’s speed of innovation, some previous personnel are concerned about the potential consequences of yet another extension.
“The delays that are happening now impact a number of different things that are incredibly painful. There is no ability for [DOD] to easily deploy classified workloads on any cloud computing environment at all across the department. Anybody who tells you differently is either a liar, or they're not the person who's actually trying to do it,” a former senior Defense official explained in an interview last week. “The impact today is that most groups across the department cannot do the work that they need with access to unlimited computing power and unlimited storage, so they're coming up with workarounds that are nowhere near as good. It is incredibly painful.”
In separate discussions via phone and email, that official—who requested anonymity to discuss internal conversations and firsthand knowledge from inside DOD—and others briefed Nextgov on emerging implications of the Pentagon’s long and complicated journey toward enterprise cloud services.
“The department is learning to be agile and adaptive,” Office of the Secretary of Defense Spokesperson Russell Goemaere told Nextgov on Wednesday. “Our forward-thinking leaders are in constant communication with the services, agencies, and combatant commands to improve system capacities and ensure the warfighter has the tools needed to accomplish every mission.”
On the Brink of the Cutting-Edge…for Years
First conceptualized by DOD in 2017, JEDI was designed to be the United States’ war cloud—a modern tool that would enable one common and connected global information technology fabric for its sprawling enterprise. When initially opened to bidders, the contract was structured to place a single cloud service provider in charge of hosting and distributing mission-critical workloads and classified military data to servicemembers across the world.
But controversy surrounded JEDI and it was delayed almost right from its inception.
Early on and throughout the bidding process, rumors of impropriety in the contracting process muddied the waters. The contract was then held up for years in litigation, and the Pentagon opted to reconsider its single-award approach to departmentwide cloud computing along the way. While multiple corporations competed for the contract, Google dropped out of the process in late 2019, and Oracle and IBM were eliminated shortly after, leaving AWS and Microsoft as the final bidders. Eventually, Microsoft won JEDI twice, but a federal court’s injunction in response to AWS’ continued protest of the contract largely prevented significant work from unfolding.
This all culminated last July, when DOD canceled the embattled JEDI program.
“With the shifting technology environment, it has become clear that the JEDI Cloud contract, which has long been delayed, no longer meets the requirements to fill the DOD’s capability gaps,” a Pentagon spokesperson said in an announcement shared with media at the time.
The cancellation of that contract coincided with the unveiling of JEDI’s replacement, the Joint Warfighting Cloud Capability program, or JWCC.
“Since the conception of JEDI, the department has increased its use of commercial cloud and is better situated to exploit the benefits of cloud computing. While JEDI—the contract—did not come to fruition, the value the department gained from going through the process to develop JEDI was immeasurable,” Goemaere explained. “Now, JWCC, having multiple cloud service providers offering their unique and cutting-edge commercial cloud solutions to the department, enhances the value for the department through increased capabilities and pricing.”
While JEDI was structured to be a “winner-takes-all” award tasked to a single cloud service provider for up to $10 billion over 10 years, once awarded and operational JWCC will be a multi-cloud, multi-vendor indefinite-delivery, indefinite-quantity contract valued at up to $9 billion. In November, the Pentagon confirmed that it issued formal solicitations for JWCC to potential prime vendors—AWS, Microsoft, Google and Oracle—but none are guaranteed an award.
“JEDI was a very opinionated version of enterprise cloud computing at all classifications around mission elements around the world. It was the idea that a piece of software could be written once for JEDI, and that capability could be used at any other location without any amount of rework in order to work on other platforms or capabilities. JWCC differs from that as it's less opinionated,” the former senior Defense official told Nextgov. “JWCC says, ‘Hey, we should still have enterprise cloud compute—but service elements or mission partners or programs can pick what they believe the right platform is for what they're creating that may require rework if you want to deploy it in other parts of the [DOD],’ so it's less opinionated.”
That lack of “opinion” then has impacts on the overall decision-making process.
“I would wager that JEDI was more realistic about where the department is in its ability to identify technical requirements to choose a platform in most places, meaning less technical people in the department have the ability to make that determination,” the official continued. “JEDI made the determination for them, whereas JWCC leaves it up to mission partners. So, there's a little bit of a tradeoff there.”
Upon first unveiling JWCC in 2021, DOD officials said they expected to announce direct awards in April 2022. But shortly before that anticipated deadline—in late March—the Pentagon pushed the award to December of this year.
Conducting deep reviews with four vendors warrants more time than previously expected, senior Defense officials said at the time. They added that this pivot would result in cloud capabilities for more sensitive and secret workloads not being deployed until at least mid-2023.
“Upon award, vendors will deliver unclassified cloud service offerings for DOD consumption. Vendors will next deliver CSOs at the Secret classification level within 60 days post-award; and CSOs at the Top-Secret level within 180 days post-award,” Goemaere confirmed.
Notably, this sets the delivery of certain enterprise solutions a fourth-year beyond when they were first predicted for early 2019. In the time that has passed since then, multiple DOD components began implementing their own cloud capabilities. The Army worked to move to an enterprise cloud, for instance, and the Defense Information Systems Agency generated milCloud 2.0.
While previously serving as the Air Force’s first chief software officer, Nicolas Chaillan also helped build out the Cloud One environment to underpin the delivery of specific services to DOD customers. In an interview with Nextgov last week, Chaillan explained that with JEDI, “people kept making claims that it was coming and coming and coming, and every single time it was delay after delay.”
Instead of implementing Cloud One as it was available, he said DOD components—and particularly officials from the Joint Artificial Intelligence Center, or JAIC—“were like, ‘well, we have to do right by the boss, so we're going to stick to JEDI,’ even if it wasn’t a thing.”
“Two years later, they pretty much came back to us, you know, crying and saying, ‘Hey, we're going to use Cloud One, because we can't wait anymore.’ So, they moved to Cloud One,” Chaillan continued. “So that's where JEDI probably for many programs cost a two-year delay, which is insane. I mean, the impact on the warfighter is just out of control. I guess people learn from that, and that's kind of sad because people stopped trusting enterprise services.”
According to DOD’s Goemaere, the department’s intent is for JWCC to provide an enterprise cloud capability while allowing mission owners to continue using cloud vehicles they’ve adopted thus far.
“JWCC will bring cutting-edge enterprise cloud services and capabilities to make sure we are positioned to fight and defend the nation on future battlefields,” he said. “JWCC will not replace existing cloud contracts, but it will serve as a complementary solution to those capabilities. JWCC will allow mission owners to continue to use their cloud contract vehicles while acquiring comprehensive cloud solutions through JWCC.”
AI and machine learning algorithms hinge on heaps of data and processing power, which comes with cloud computing. While the JAIC, originally meant to be an early JEDI adopter, opted to pivot to Cloud One through all the delays, as Chillain noted, the other prior DOD personnel Nextgov spoke to expressed worry that the delays could reduce America’s ability to lead in meaningful AI applications, like computer vision.
As the former senior DOD official perceives it, at this point, it seems that “there is no ability for most of the department to adequately use, utilize or deploy any of those capabilities in the real operational mission because the most accessible way to use those technologies and acquire them will be only on low impact level deployments.” To them, it’s not clear that JWCC will be able to drive the technology adoption the department needs now.
The success of JWCC will lie not in the total number of vendors or value of contracts, they said, but in the number of capabilities that are deployed into the operational environment at classification levels that sit above unclassified workloads. “And if that number sits at zero,” in their view, the U.S. will “continue to lose our ability” to have information and decision advantage.
“That's not acceptable. It's certainly not acceptable when we think about future state information and decision advantage against a peer competitor like China, right? So it is incredibly important and every delay substantially removes our ability to be competitive, not a little bit, but by a lot… We have lost those years, we have given the advantage to somebody else,” the former senior Defense official said. “So, that's why every delay for JWCC should be an alarm that goes off in the hearts and minds of those who are looking at how we can be competitive in the future against China.”
Through a recent reorganization, the JAIC as well as several other DOD components, now operate within the newly formed Chief Digital and Artificial Intelligence Office. CDAO Public Affairs Officer Cmdr. Sarah Flaherty deferred Nextgov’s question regarding concerns that DOD can’t fully deploy AI and ML capabilities in the most effective and modern manner without JWCC in place to the Office of the Defense Secretary.
Still, she noted that while DOD is “looking forward to the delivery of an enterprise-level, multi-vendor acquisition vehicle that will fill longstanding requirements and capability gaps in support of the warfighter, the CDAO has been able to leverage the cloud to accelerate our data management, analytics and AI/ML journey to decision advantage.”
“For example, the Advana platform leverages the power of cloud and AI/ML technology today for a wide variety of use cases like fraud, predictive maintenance, and intelligence, surveillance, and reconnaissance,” Flaherty said.
In response to that same AI-focused question, OSD’s Goemaere said that the “department recognizes the need for an enterprise cloud solution that gives our warfighters the tools to fight tomorrow’s wars, today.”
“When fully deployed, JWCC will be the platform that provides enhancements to AI and ML capabilities, as well as other cloud resources, and will create the space for our warfighters to stay ahead of our competitors,” he added.
Losing Momentum?
Over the course of this years-long process toward modernization, DOD “enhanced its capabilities and capacity to better understand the benefits of cloud computing” to its missions across the globe, Goemaere noted. At the same time, he pointed out, industry has continued to develop significant technical and marketplace advancements.
And that’s something senior analyst John Caucis has been tracking at the technology market research and consulting firm Technology Business Research, Inc., over the last few years.
To develop their perspectives on the complex industry, TBR analysts take a “bottoms-up” approach to tracking the tech sectors they monitor by following the top tier vendors in each segment or sub-segment. For America’s federal IT market, that includes leading systems integrators, the IT-related operations of defense primes and other businesses.
“What [certain] vendors took the opportunity of the litigation-related delay and the protests-related delay of JEDI to do is to focus on expanding their own suite of cloud-related offerings,” Caucis told Nextgov on Monday. “Now, they were already moving in that direction. But, as you probably know, things in the federal space can tend to move at a very glacial pace. I have heard comparisons like federal IT maturity is anywhere from three, to five, to—in some cases varying by agency—seven years behind the commercial sector, and that's been true with the cloud.”
Through the long and protracted litigation process, he noted, the Pentagon’s mindset also evolved “from cloud-or-bust, to fit-for-purpose.”
“And this interestingly really parallels what we were seeing on the commercial side,” Caucis said.
TBR’s enterprise customers have been increasingly shifting their preferences to hybrid cloud deployment models that enable them to migrate to cloud environments workload by workload, which helps account for the varying maturity levels of its IT architectures.
“This is now, without question, the case with the DOD—and across federal in general. I'm hearing not just anecdotes, but I'm seeing evidence that the end-users want to avoid vendor lock-in and they prefer vendor agnosticism,” Caucis said. “So, just like their counterparts on the commercial side, IT managers, chief information officers and whatnot on the federal side want choice, and they want the flexibility that hybrid cloud provides.”
Still, while the DOD’s extension for cloud deployment deadlines is meant to enable the delivery of modern services to best meet what seem like ever-evolving needs, the former officials who briefed Nextgov said the repeated delays are impacting morale within the department.
“I'm still advising a lot of people,” former Air Force CSO Chaillan noted. “And there's a lot of people leaving [DOD] … We're losing momentum. We're losing adoption. We're losing urgency. You know, some people feel like maybe they shouldn't do this anymore because they're getting mixed signals from leadership in terms of the importance of doing this kind of work.”
“This is a big problem,” the other former senior Defense official also said. “Anyone who thinks that we're talking about contracts or technology is woefully unaware of the substantial impact that not having cloud computing capabilities available to the Department of Defense and to our warfighters is having. It feels like losing. It's very painful.”
But in the Pentagon, officials remain hopeful.
“Over the last few years, the DOD has evaluated lessons learned from our cloud experience, and have molded JWCC to be the cloud acquisition that will bring future capabilities to bear for the department,” Goemaere said. “The enterprise-wide holistic view, along with a better understanding of cloud solutions, and recommendations based on experiences of our sister agencies and communities, will be made evident in the success of JWCC for the DOD.”