What is the Good Judgment Project and why is it important?

The Good Judgment Project (GJP) is a project “harnessing the wisdom of the crowd to forecast world events.” It was co-founded by Philip E. Tetlock (left, author of Superforecasting with Dan Gardner as well as Expert Political Judgment: How Good Is It? How Can We Know?), decision scientist Barbara Mellers, and Don Moore.

Its workforce consists of volunteers. After a simple screening, 3,200 were invited to participate and began forercasting.It was a participant in the Aggregative Contingent Estimation (ACE) program of the Intelligence Advanced Research Projects Activity (IARPA) in the United States. Predictions are scored using Brier scores. The top forecasters in GJP are “reportedly 30% better than intelligence officers with access to actual classified information.” Moreover, forecasters on the small GJP’s superteams became 50% more accurate in their individual predictions.

According to its website, the Good Judgment Project‘s independent, non-partisan research has “discovered four keys to accurate forecasting: talent-spotting, training, teaming, and aggregation. Our scientifically validated process uses these four keys to unlock the secret to better forecasts – and better decisions.” More specifically:

Talent “Our evidence-based techniques reliably identify the most accurate forecasters.”

Training: “Combining our training methods with deliberate practice increased accuracy by 10-12% in Good Judgment research.”

Teamwork: “Forecasters working in teams outperform those working alone or in prediction markets.”

Aggregations: “Our state-of-the-art aggregation methods extract the maximum possible signal from the noise of crowd-sourced forecasts.”

Selected Good Judgment publications on the science of Superforecasting include:

I highly recommend Philip Tetlock and Dan Gardner’s classic work, Superforecasting: The Art and Science of Prediction, its second edition published by Broadway Books(September 2016).

Posted in

Leave a Comment





This site uses Akismet to reduce spam. Learn how your comment data is processed.