CMS amped up cloud capacity a day before HealthCare.gov launched

The Centers for Medicare and Medicaid Services rushed to add cloud computing capacity to HealthCare.gov as the result of tests conducted just days before the site's scheduled Oct. 1 launch last year, according to contracting documents released on Jan. 24.

CMS modified the contract of Verizon-owned Terremark Federal Group to the tune of $8.7 million to add infrastructure "not contemplated at the time of the award," according to the document justifying the limited-source award. The contracting officer noted in the justification that Verizon and CMS could not have foreseen the need for additional capacity at the time the original contract was awarded. The adjustment raised the value of Terremark's hosting contract from $37.2 million to just over $46 million.

Performance and stress tests conducted on Sept. 24, 2013, confirmed the need for additional capacity because the Terremark cloud could support only 10,000 concurrent users, not the 50,000 CMS anticipated during peak traffic times. The original task order to Terremark did not specify a number of concurrent users.

CMS indicated that Terremark was "in the best position to add the required capacity as quickly as possible to their cloud environment." The document was signed on Sept. 30 by CMS CIO Tony Trenkle, who has since left for a private-sector job.

Hewlett-Packard will replace Terremark on the cloud-hosting contract at the end of the March open enrollment period for people to buy or change coverage via the health insurance exchanges on HealthCare.gov.

Left unsaid is what prompted CMS to seek additional infrastructure bandwidth. However, a related document explains the decision to contract with Accenture for continued development of the Federally Facilitated Marketplace after a flawed launch under contractor CGI Federal. That document, signed by then-acting CMS CIO David Nelson on Dec. 27, 2013, provides a laundry list of some of the deep problems in the development of the site and details ongoing problems with the creation of a financial management system to support subsidy payments under the Affordable Care Act.

Under a one-year, $91.1 million contract, Accenture is tasked with building a financial management platform that monitors eligibility and enrollment, tracks payments to insurance carriers and supports the calculation of tax obligations with Internal Revenue Service systems. That platform is due in mid-March. Failure to deliver on time will "result in financial harm to the government," the document states. Absent a financial management platform, "the entire health care reform program is jeopardized."

CMS decided to switch from CGI Federal in December after the contractor failed to deliver promised software to check for duplicate enrollments, track enrollment problems with insurance carriers and calculate tax credits. Accenture is also being asked to shore up existing pieces of the Federally Facilitated Marketplace that were still failing as recently as December, despite months of repair work led by prime contractor Quality Software Services Inc. and a government team led by former acting Office of Management and Budget Director Jeff Zients.