MITRE releases evaluation results for 21 cybersecurity products

On April 21, McLean, VA-based MITRE released the results of an independent set of evaluations of cybersecurity products from 21 vendors to help government and industry make better decisions to combat security threats and improve industry’s threat detection capabilities.

Using its ATT&CK knowledge base, MITRE emulated the tactics and techniques of APT29, a group that cybersecurity analysts believe operates on behalf of the Russian government and compromised the Democratic National Committee starting in 2015. The evaluations, which were paid for by the vendors, include products from Bitdefender, Blackberry Cylance, Broadcom (Symantec), CrowdStrike, CyCraft, Cybereason, Elastic (Endgame), F-Secure, FireEye, GoSecure, HanSight, Kaspersky, Malwarebytes, McAfee, Microsoft, Palo Alto Networks, ReaQta, Secureworks, SentinelOne, Trend Micro, and VMware (Carbon Black).

“The ATT&CK Evaluations help the cybersecurity community by improving the security products that we rely upon and arming end users with objective insights into those product capabilities to detect known adversary behaviors,” said Jon Baker, MITRE department head for adversary emulation and orchestration.

MITRE developed and maintains the ATT&CK knowledge base, which is based on real world reporting of adversary tactics and techniques. ATT&CK is freely available, and is widely used by defenders in industry and government to find gaps in visibility, defensive tools, and processes as they evaluate and select options to improve their network defense.

MITRE previously evaluated products from Carbon Black, CrowdStrike, GoSecure, Endgame, Microsoft, RSA, SentinelOne, Cybereason, F-Secure, FireEye, McAfee, and Palo Alto against the threat posed by APT3, a Chinese group that analysts believe is currently focused on monitoring Hong Kong-based political targets, and began releasing those results in late 2018.

“We’ve seen a huge growth in participation from our initial evaluations based on APT3 to this round of evaluations because vendors have seen the value of this kind of testing,” said Frank Duff, ATT&CK Evaluations lead. “We bring a very collaborative approach to evaluations, by working with vendors who want to improve their products, which ultimately makes cyberspace safer for everyone.”

The ATT&CK Evaluations team chose emulating APT29 because it offered the chance to evaluate the cybersecurity products against an adversary that uses sophisticated implementations of techniques through custom malware and alternate execution methods, such as PowerShell and WMI.

The team also made changes to the way that it presented the results based on feedback on the APT3 evaluations from analysts, vendors, and end users. The ATT&CK Evaluations website now features a tool that enables users to select particular vendors and display a side-by-side comparison of how they detected each technique, as well as a data analysis tool to take a deeper look at how they handled those techniques.

The team has also released a Do It Yourself APT29 evaluation that leverages CALDERA, an automated red team system that MITRE developed using the ATT&CK knowledge base. This enables users who are intrigued by the evaluations to test security products in their own environments against the same adversary. This may be particularly useful for organizations that can’t afford to employ a red team, Duff said.

Source: MITRE