Don’t Blindly Trust Software Building Blocks, Report Says

After the SolarWinds and Kaseya attacks spread malware far and wide across government and business networks, Palo Alto Networks’ Unit 42 Cloud Threat Team wanted to investigate how it was possible for bad actors to infiltrate cloud-native applications by attacking the cloud supply chain.

The company has now issued a Unit 42 Cloud Threat Report, “Secure the Software Supply Chain to Secure the Cloud,” that identifies the serious problems lurking below the surface of open-source software repositories. The Cyber Threat Alliance hosted a webcast for the report’s author to walk through its key findings.

“What most people missed about SolarWinds, including myself, is what the CEO said” about the attack, said Matt Chiodi, Palo Alto’s CSO of Cloud. “He said, ‘The threat actor did not modify our source code repository. The malicious activity occurred within the automated build environment for our Orion Platform software.’”

In other words, the bad actors didn’t insert the malicious code into their main source code, they went after common software development build tools that actually create and handle the source code.

“Open-source tools are great, [and] anyone can scrutinize them, but no one’s obligated to maintain or secure them,” Chiodi said. “Some of the most popular modules haven’t been updated in years.”

He walked through the hypothetical creation of a very simple cloud infrastructure built in a cloud-native way to demonstrate the problem.

“Phase one is cloud infrastructure provisioning, where developers will typically use infrastructure as code,” he said. “In this case, the developer has chosen three specific modules” from a popular repository – storage, VPC and Kubernetes. “Storage is dependent on two others,” for a total of five modules.

“We found nine misconfigurations already, even though no applications are running yet,” Chiodi said.

In phase two, the developer started to deploy Kubernetes, using something called a helm chart – which itself depends on two other helm charts. “There are now five Kubernetes misconfigurations [to] add to the nine from above,” he said.

In the final phase, the developer began to deploy the container images. Inside each container is the application code and all its elements. “There are 40 application vulnerabilities in the application code in the container itself,” Chiodi said. “This is how vulnerabilities and misconfigurations can snowball within the supply chain. This would be fully functional, except the default configurations aren’t likely to be very secure.”

The most effective way to address the problem is to encourage developers to use safer tools. “You don’t want to stifle innovation, but get them to scan for misconfigurations and vulnerabilities to begin with,” he said. “As applications build up their dependencies, they build up their misconfigurations … One application we examined had 23 dependencies, and 92% of the misconfigurations were in those dependencies.” It is also important that DevOps and security teams have visibility into all of the bill of materials for every cloud workload, he said.

Chiodi laid out four steps for mitigation:

• The Cloud Native Computing Foundation released a white paper on software supply chain best practices. “Read it,” he said.

• Create a strategic plan for what “shift-left security” looks like, including the vision, milestones and metrics.

• Investigate and identify where software is being created throughout the entire organization.

• Implement quality assurance by making sure that in every step of the software development process, the developers can receive feedback and address identified security issues.

Using open-source tools to create new and innovative applications does an excellent job of speeding up time to use. But they require vigilance from developers and transparency for security professionals.

Editor's note: This article was updated to clarify the types of build tools hackers targeted.