Expert judgment


Before the emergence of polls in the 1930s, judgments from political insiders and experienced observers were commonly used for forecasting (Kernell, 2000), as they still are today. When making predictions, expert analysts are assumed to be independent, and they each have experience in reading and interpreting polls, assessing their significance during campaigns, and estimating the effects of recent or expected events on their results. Each expert can be expected to use a different approach and rely on at least somewhat different data sources when generating forecasts. Thus, combining experts’ judgments should increase forecast accuracy.

We have conducted expert surveys as part of the PollyVote project since its launch in 2004. In these surveys, we asked experts to predict the national popular vote. As shown in the chart below, their forecasts have always added valuable information and thus contributed to the accuracy of the PollyVote.  On average, adding expert judgment to the combined PollyVote reduced forecast error by 15%.

Use of expert judgment in the PollyVote

Since 2004, Polly periodically contacts a panel of experts to obtain their estimates of the incumbent’s share of the two-party popular vote on Election Day.

Table 1: Polly’s expert panel results
Date N of experts Dem Rep
December 26-31, 2015 8 51.7 48.3
January 26-31, 2016 17 52.0 48.0
February 26-29, 2016 15 52.0 48.0
March 28-31, 2016 14 53.1 46.9
April 27-30, 2016 15 53.2 46.8
May 29-31, 2016 13 52.4 47.6
June 28-30, 2016 12 54.2 45.8
July 29-31, 2016 12 52.7 47.3
August 29-31, 2016 13 53.5 46.5
September 28-30, 2016 13 52.2 47.8
October 13-15, 2016 14 54.7 45.3
October 29-31, 2016 13 52.7 47.3
November 6-7, 2016 12 52.2 47.8

References