.: Hybrid Forecasting


Hybrid forecasting comprises techniques to combine machine forecasting and human judgments. This long has been practiced in the field of meteorology, by running several computer models (machine forecasting) in parallel, often giving significantly different results. Using these models as inputs, meteorologists make judgments on the basis of their expertise. Next, after observing the resulting weather, both the meteorologists and those who design and run the computer models will refine their techniques.

The national defense community is working on improving forecasts of geopolitical and geoeconomic events through similar techniques. From 2011 through 2015, the Intelligence Advanced Research Projects Activity (IARPA) funded competitions in both machine forecasting of such events, and human forecasting. Both techniques, using only open source information, have outperformed traditional intelligence community techniques. The EMBERS machine forecasting system, which won IARPA's mchne forecating competition, now has become operational within the CIA. The winner of the human forecasting competition, Good Judgment Laboratory, spun off Good Judgment Analytics, comprising a team of prove superforecsters who advise many government and private entities.

Currently, IARPA's Hybrid Forecasting Competition is underway, but with mixed results. None of the three competing contractors was able to even equal the control group. The program has been shortened by one year, one of the contrators was terminated, and the number of forecasting questions planned has been cut in half. IARPA also just finished its Geopolitical Forecasting Challenge, with mixed results. Perhaps in 2019, one of the contractors will break out of the pack and get us cheering. By comparison, in IARPA's human forecasting competition, Aggregative Contingent Estimation (ACE), just one competing contractor, the Good Judgment Project, did well, and indeed spectacularly well.

ISIT has been engaging in internal research activities in pursuit of machine forecasting along with our teaming partner, Basis Technology, which played a central role in EMBERS.

ISIT's Carolyn Meinel, who was an official superforecaster from the Good Judgment Project, also has participated with IARPA's test and evaluation team as we seek to realize the potential of Hybrid Forecasting.