Chantilly, VA-based Perspecta Inc. announced on September 24 that its applied research arm, Perspecta Labs, received an award from the Intelligence Advanced Research Projects Activity (IARPA) to provide research on the Trojans in Artificial Intelligence (TrojAI) program.
Advancements in artificial intelligence (AI) and machine learning (ML) have added complexity to the threat landscape. By hacking the training phase of ML, adversaries can disrupt AI systems during development, causing them to make incorrect classifications and take dangerous actions. The goal of the TrojAI program is to detect adversarial Trojans placed into AI systems to prevent such attacks. Perspecta Labs will lead a team of researchers to study Trojan contamination and develop and provide a multi-faceted detection mechanism to significantly reduce Trojan security risks.
“At Perspecta Labs, we welcome the opportunity to support IARPA on research to defend critical AI systems against malicious interference,” said Petros Mouchtaris, president and general manager, Perspecta Labs. “We look forward to applying our leading expertise in adversarial ML combined with our vast experience in cybersecurity to develop new solutions that can automatically identify compromised AI systems in support of this critical work.”
To accomplish this task, the Perspecta Labs team will develop automatic tools to detect Trojans hidden within AI systems. Several techniques will be developed to provide multi-pronged detection, including: detecting Trojans by stimulating neurons in a neural network model and leveraging gradient magnitudes to identify Trojans among candidate triggers; confirming the presence of Trojans via differences in the distribution of vector representations of sample inputs generated from the AI; and detecting Trojans based on the clustering properties of adversarial directions.
By 
