MITRE Piloting Evaluations to Validate Cybersecurity Firms’ Protection Claims 

Buyers trying to make informed decisions about which cybersecurity firms are best suited to protect their organizations from a particular threat actor might soon be able to use a public tool produced by the MITRE Corporation for that purpose.

MITRE announced plans to assess firms’ effectiveness detecting and protecting against threats from a hacker gang known as the Carbanak group—associated with attempts to infiltrate banking transfers and ATMs—in a press release the organization issued today.

In the early summer of 2018 firms such as Carbon Black, CounterTack, CrowdStrike, Endgame, Microsoft, RS, and SentinelOne submitted their threat detection tools for MITRE, a nonprofit which manages federally funded research and development centers, to test their products against known threats cataloged in the MITRE ATT&CK compendium. 

The Cybersecurity and Infrastructure Security Agency this week referenced the MITRE website in alerting the public to details used in an attack on a natural gas compression facility. 

Since 2018, more firms have participated in the evaluations conducted by MITRE, which says it’s filling a void that existed in neutral authorities that could validate claims vendors make about their capabilities. 

But the evaluations have so far only addressed threat detection capabilities. That could be changing.  

“During the previous evaluations, vendors would note when they believed a protection would have prevented the execution of specific evaluated behaviors,” said Frank Duff, MITRE’s ATT&CK evaluations lead. “By extending the offering to include protections, the evaluations will be able to definitively say whether this was the case.”   

Although MITRE doesn’t attach any sort of ranking or certification based on its evaluations, the vendors pay a fee to be evaluated, as MITRE makes all results public and consumers, including government agencies, can analyze them to make nuanced decisions about what’s right for their needs.

“We’ve heard from companies that have incorporated data from the first evaluations into their purchasing decisions that doing so has enabled them to make better-informed decisions faster and at a far lower cost—up to 10 times less than they would have spent evaluating the products entirely on their own,” Duff said. 

MITRE ATT&CK’s previous evaluations focused on threats from APT 3, thought to be a Chinese group monitoring Hong Kong-based political targets, and APT 29, which analysts believe worked with the Russian Government to compromise the Democratic National Committee.