Is UL's cyber assurance program ready for prime time?

One of the provisions of President Barack Obama's 2016 Cybersecurity National Action Plan was the creation of a Cybersecurity Assurance Program -- a partnership between the Department of Homeland Security and UL to certify networked devices. But that program is not passing muster with some industry experts, including one DHS official.

Consumers rely on UL listings to certify the safety and reliability of products such as light bulbs, batteries and smoke detectors. However, panelists at AppSec USA 2016 cast doubt on the value of the company's CAP and the UL 2900 testing tools and standards, which launched in April.

Kevin Greene, a program manager in the DHS Science and Technology Directorate's Cyber Security Division, said he supports the idea of a software certification system, but he is "just not sold on the fact that we have the necessary technology and innovation around software analysis tools to get to the point where we have enough evidence to really certify software."

He argued that effective certification requires a suite of automated tools, and right now, it's not clear what the existing tools can do.

"If we can get to ground truth with these technologies and tools, I think we're in a better position to help amplify what cyber UL is trying to do," Greene said. "One thing I would like to see is more transparency where you have some of the folks in academia who have been doing static analysis, doing dynamic analysis, binary analysis chime in to figure out what's the best way to go about leveraging static analysis in the UL certification."

UL conducts various tests -- static, dynamic, binary and fuzzing -- to evaluate whether network-connectable products contain any known vulnerabilities or software weaknesses and whether they can be mitigated or patched.

Anita D'Amico, CEO of Code Dx, and Joshua Corman, director of the Cyber Statecraft Initiative at the Atlantic Council, also raised concerns about the cost of the certification program and its potential to create a barrier for smaller vendors and developers.

Furthermore, the three panelists said the certification process could create a false sense of security because a device or piece of software could be secure at the time of certification, but vulnerabilities might have slipped through the testing process or new ones could pop up in the future.

They also said the UL process runs the risk of emphasizing compliance rather than a risk management approach to software security.

On the whole, the panelists expressed the view that the UL certification should be made more robust before people start relying on it.

No one from UL participated in the discussion, but Joe Jarzombek, former director for software assurance at DHS, was in the audience and came to the defense of the UL program. He is now global manager for software supply chain management in the Software Integrity Group at Synopsys, which provides testing tools to UL.

Jarzombek told FCW that UL has raised the software security bar. "UL is not saying this is 100 percent guaranteed -- they're qualifying exactly what that is," he said, adding that concerns about UL pushing developers to build products to pass a test were misplaced. "To me, that's a wonderful thing. You've built a product that has no known vulnerabilities."

Ken Modeste, leader of cybersecurity technical services at UL, told FCW that the panelists raised valid concerns, but he said UL has been taking such concerns into consideration, and the program is valuable for industry and consumers.

"There is no tool out there that can do 100 percent," Modeste said. "Right now,...the majority of vendors and manufacturers are not using any, and so using a tool that could provide a solution that solves 50 or 60 percent of your problems is better than using nothing."

UL is looking for new and innovative tools that can better test and evaluate software, but for now, it has to make the most of what's available, he said. He added that the current program's broader objective is to push industry to incorporate security and manage risk.

"The objective is what does the vendor do when they find anything, and that's really the behavior that UL is trying to drive," Modeste said. "Part of the process is working with the vendor so that they can develop mechanisms to identify if there are any new things that are found within their product post-certification."

He added that no products will be permanently certified, and the initial certification will expire after 12 months.

In terms of the cost, Modeste acknowledged that it might be difficult for some smaller companies to afford a process that can range in cost from $40,000 to $150,000. He said UL had to balance a variety of factors to make the certification process both effective and affordable.

Right now, the company is focusing on certifying medical devices and critical infrastructure products, which require more rigorous standards. Modeste said that in the future, UL will expand the program to consumer devices, which have a lower set of standards and therefore cost less to certify.

He also said UL is already working on the next iteration of cyber testing and certification standards, which officials hope to finalize in the next 12 months.