On August 28, the Intelligence Advanced Research Projects Activity issued a Proposers Day notification for its Forecasting Counterfactuals in Uncontrolled Settings (FOCUS) program (Solicitation Number: IARPA-BAA-17-08). The Proposers Day will be held on October 19, in advance of the release of a new solicitation. Registration closes at 5:00 p.m. on October 13.
The FOCUS program seeks to develop and empirically evaluate systematic approaches to counterfactual forecasting. Counterfactual forecasts are statements about what would have happened if different circumstances had occurred. For example, a post mortem of an analysis failure may lead to a conclusion that analysts would have avoided the failure if they employed better tradecraft; perhaps by having double checked assumptions, perhaps by having considered a broader range of hypotheses, etc. Counterfactual forecasts about what would have worked in past circumstances are very often the basis for lessons learned for what to do in the future. And such lessons often evolve, over time, into best practices and tradecraft.
To date there has been little in the way of research that measures the extent to which different approaches to counterfactual forecasting yield accurate vs. inaccurate counterfactual forecasts. And there is a similar paucity of research on the accuracy of lessons drawn from different lessons-learned approaches. As a result, evidence-based guidance for approaching post mortem lessons learned activities, and for developing the counterfactual forecasts that are the core of such activities, do not exist. Also there is correspondingly little empirical evidence that would support a claim that current lessons learned practices usually yield good or accurate lessons.
FOCUS will address this research gap by developing and empirically testing alternative approaches to structuring the counterfactual forecasting process in ways that can be readily incorporated into lessons-learned activities related to improving analyses and analytic tradecraft in complex domains such as geopolitical analysis. FOCUS will concentrate on cognitive methods that systematically step individuals and teams through a series of considerations during counterfactual reasoning.
Although FOCUS will concentrate on analytic methods and tradecraft, the evidence-based lessons-learned approaches that will emerge from FOCUS should be applicable to any discipline or organization that routinely engages in formal lessons learned activities ranging from business case analyses, military hot washes, medical case studies, historical analyses, corporate knowledge management, transportation accident investigations, and a great many others. IARPA therefore encourages broad participation in this program reflecting a diversity of disciplines and perspectives.
Full information is available here.