DARPA AIDA program aims to make sense of big data

The U.S. government has always had an interest in developing and maintaining a strategic understanding of events, situations, and trends around the world. In recent years, however, information complexity has exceeded the capacity of analysts to glean meaningful or actionable insights as data pours in from disparate sources, across a variety of genres, and a mixture of structured and unstructured forms, from military intelligence to social media to accurate and inaccurate news. Now, in a new effort, DARPA seeks to overcome the noisy, conflicting, and potentially intentionally deceptive nature of today’s data environment through a program called Active Interpretation of Disparate Alternatives (AIDA).

The goal of AIDA is to develop a multi-hypothesis “semantic engine” that generates explicit alternative interpretations or meaning of real-world events, situations, and trends based on data obtained from an expansive range of outlets. The program aims to create technology capable of aggregating and mapping pieces of information automatically derived from multiple media sources into a common representation or storyline, and then generating and exploring multiple hypotheses about the true nature and implications of events, situations, and trends of interest.

“It is a challenge for those who strive to achieve and maintain an understanding of world affairs that information from each medium is often analyzed independently, without the context provided by information from other media,” said Boyan Onyshkevych, program manager in DARPA’s Information Innovation Office (I2O). “Often, each independent analysis results in only one interpretation, with alternate interpretations eliminated due to lack of evidence even in the absence of evidence that would contradict those alternatives. When these independent, impoverished analyses are combined, generally late in the analysis process, the result can be a single apparent consensus view that does not reflect a true consensus.”

The AIDA program hopes to determine a confidence level for each piece of information, as well as for each hypothesis generated by the semantic engine. The program will also endeavor to digest and make sense of information or data in its original form and then generate alternate contexts by adjusting or shifting variables and probabilities in order to enhance accuracy and resolve ambiguities in line with real-world expectations.

“Even structured data can vary in the expressiveness, semantics, and specificity of their representations,” said Onyshkevych. “AIDA has the potential to help analysts and military decision makers refine their analyses so that they are more in line with the larger and more complete overall context, and in doing so achieve a more thorough understanding of the elements and forces shaping our world.”

The AIDA program’s Broad Agency Announcement (HR001117S0026), which details technical goals and performer requirements, is available at: https://go.usa.gov/xXQQ8.

Source: DARPA