IARPA seeks industry input on TrojAI BAA
On December 7, the Intelligence Advanced Research Projects Activity (IARPA) posted a draft funding opportunity description for the TrojAI broad agency announcement (Solicitation Number: IARPA-BAA-19-03). IARPA requests industry feedback no later than January 4, 2019.
Using current machine learning methods, an artificial intelligence (AI) is trained on data, learns relationships in that data, and then is deployed to the world to operate on new data. For example, an AI can be trained on images of traffic signs, learn what stop signs and speed limit signs look like, and then be deployed as part an autonomous car. The problem is that an adversary that can disrupt the training pipeline can insert Trojan behaviors into the AI. For example, an AI learning to distinguish traffic signs can be given just a few additional examples of stop signs with yellow squares on them, each labeled “speed limit sign.” If the AI were deployed in a self-driving car, an adversary could cause the car to run through the stop sign just by putting a sticky note on it (see figure below). The goal of the TrojAI program is to combat such Trojan attacks by inspecting AIs for Trojans.
This Draft Broad Agency Announcement (BAA) is NOT a formal request for proposal. Proposals are NOT being requested or evaluated at this time. A Final BAA will be released at a later date for which interested parties may submit proposals.
The Intelligence Advance Research Projects Activity (IARPA) is seeking interested parties to thoroughly review the Draft BAA Funding Opportunity Description and provide comments, questions, suggested changes, and feedback by January 4, 2019 in the format provided.
IARPA does not anticipate posting responses to any comments, questions, suggested changes, and/or feedback received; however, all input will be considered in developing the Final BAA. Additionally, the Draft BAA Funding Opportunity Description is subject to change in the development of the Final BAA as a result of
IARPA’s consideration of the input received from interested parties in response to the Draft BAA as well as other considerations.
Full information is available here.