PollyVote Project

Polly the parrot uses the high-profile application of election forecasting to demonstrate advances in forecasting methodology.

Her focus is on the principle of combining by averaging forecasts from different component methods. This approach has lead to highly accurate forecasts for the past five U.S. presidential elections (Graefe et al. 2011).

Polly's caretakers are Scott ArmstrongAlfred CuzanAndreas Graefe and Randall Jones.

You have comments, suggestions, or corrections? Contact us at This email address is being protected from spambots. You need JavaScript enabled to view it. .

Search

Spread the word

With her Election Eve prediction of two party-vote shares of 51.0% for Obama and 49.0% for Romney, Polly, the predicting parrot, almost hit the bull’s eye for these preliminary results. 

The latest vote count has Obama at 50.6% and Romney at 47.9%. This result translates to two party-vote shares of 51.4% for Obama and 48.6% for Romney. That is, Polly missed the final two-party vote share for Obama by 0.4 percentage points. This is similar to Polly’s performance in her two previous appearances. For the 2004 and 2008 elections, the average error of Polly’s final forecast was half a percentage point.

In addition, Polly always predicted Obama to win since her first forecast on January 1st of 2011, more than 22 months prior to Election Day. In comparison, other methods such as polls or prediction markets at times predicted the Republican candidate to win. This is similar to 2004 and 2008, when Polly never strayed from Bush (8 months prior to Election Day) or Obama (14 months). 

The task of predicting U.S. presidential elections is ideal for demonstrating the usefulness of combining forecasts, as there are a number of different methods that use different sources of information, or process it differently. In addition, it is difficult to judge a priori which component forecast is likely to be most accurate at different times in a campaign. The historical track record of single models or methods is usually of little help. A prominent example of this year’s campaign is  the Iowa Electronic Vote Share Market (IEM), which provided the most accurate component forecasts in past elections (Graefe et al., 2012). This time, the IEM was among the least accurate of Polly’s constituent  forecasts, except for its Election Eve forecasts.

The Table here shows the mean absolute error of daily forecasts of the PollyVote and its components across the last 100 days prior to Election Day. On average, the PollyVote missed the actual results by less than half of a percentage point and clearly outperformed its components. The error reductions due to the PollyVote ranged from 41% (compared to combined expert judgment) to 77% (compared to combined index models).

For almost two years, Polly correctly predicted that Obama would be reelected, and with little variation in her forecast. For the general election observer or journalist, such a forecast may not grab headlines, but it beats the competition day-in and day out.

The PollyVote team consisting of Scott Armstrong, Alfred Cuzán, Randall Jones and Andreas Graefe organized a panel at the 2013 APSA Annual Meeting in Chicago. Below you can find the slides for each of the talks as well as links to the full papers if available. 

Innovations in Presidential Elections Forecasting: The Pollyvote 

Armstrong, J. S.; Graefe, A.; Jones, R. & Cuzán, A.: The PollyVote: Progress and Plans.

Armstrong, J. S. & Graefe, A.: Index Models for Advising Candidates.

Graefe, A.: Accuracy of vote expectation surveys in forecasting elections. (Full paper available here)

Jones, R. & Cuzán, A.: Expert Judgment in Forecasting American Presidential Elections: 2004-2012.

In a final pre-election survey, the PollyVote panel of elections experts predicted that President Obama will win 50.5% of the two-party vote, compared to 49.5% for Gov. Romney.  The results of this survey, conducted November 1-4, were slightly lower for the president than the 50.6% estimate made two weeks earlier.  The average of campaign polls taken during the latest survey period, 50.5% on a two-party basis, was identical with the experts’ composite estimate.  Also during this time, the average closing price of the Democratic vote-share contract on the Iowa Electronic Market was equivalent to 51.4% in Obama’s favor.  It is apparent that all three indicators are predicting a close outcome on Election Day.

Interestingly, the experts’ last prediction of the campaign was very little different from their initial long-range forecast of the president’s electoral prospects.  In the first expert survey, conducted in late December and early January, the panelists’ composite estimate for Obama was 50.7% of the two-party vote.  As the campaign now ends, the experts’ last prediction for Obama, 50.5%, is within 0.2% of their first forecast.  It is evident in the accompanying graph that the experts’ assessment of Obama’s electoral strength has been rather stable throughout the long 2012 campaign.  The difference between Obama’s high point of 51.6% in March and his low point in July (50.4%) is only 1.2%.  By contrast, polls and market trading prices have been more volatile, as also evident in the graph.  As noted, though, the indicators now are quite close in their current readings, presumably pointing to the same outcome.    

Read more: Final prediction of Polly's expert panel Obama will win 50.5% of the popular vote

After almost two years of collecting and combining forecasts from five different component methods, Polly has finally finished her job for the 2012 election. Polly's final forecast of the popular two-party vote shares predicts Obama to gain 51.0% (v. Romney 49.0%).

Over the past 22 months since January 1st of 2011, Polly has continuously predicted Obama to win the popular vote. As shown in the graph below, the highest value for Obama (54.1) was reached in May 2011 with the killing of Osama bin Laden. Shortly after, the PollyVote forecast decreased and reached its lowest value in November 2011, predicting Obama to win 50.4% of the popular vote. At that time, Polly could draw only on few component forecasts, as the expert panel was not yet in place and only few forecasts from statistical models were available. This explains the somewhat higher volatility of the early forecasts.

In 2012, with an increasing number of component forecasts available, Polly's forecast moved in a much narrower range, with a maximum of 52.6% at the beginning of April and a minimum of 50.9% in late October. Thus, range of her forecast was only 1.7 percentage points. 

One or more of Polly’s components will of course provide more accurate Election Eve forecasts than Polly, and it is easy to identify the most accurate forecast after the fact. However, at the time of making a prediction, it is difficult to pick the most accurate forecasts, in particular when making forecasts for long time horizons that involve much uncertainty.

The PollyVote is designed for situations that involve much uncertainty. This means that it is most useful months or years prior to the election, not for an Election Eve forecast, when it is usually sufficient to look at the latest poll average.

Polly has a perfect track record in predicting the popular vote winner, also for long time horizons. In 2004, starting eight months in advance, and in 2008 fourteen months ahead, Polly always predicted the correct winner. This time around Polly has continuously predicted Obama to win the two-party vote since her first forecast was released on January 1, 2011, almost two years before the election. With her final forecast of 51.0% for Obama, and all of her five components pointing to an Obama victory, Polly is confident that her perfect track record will remain.  

 

Politico reports about two new voter expectation surveys released by Gallup and Reuters. In both surveys, the majority of voters expected Obama to win. This result is in line with each voter expectation survey that has been conducted since early March. At no time, citizens expected Romney to win the election.

Voter expectation surveys are among the most accurate forecasting methods available when it comes to predicting who will win. In addition, they can be easily translated to valuable predictions of the popular vote. The chart below shows how such vote share predictions based on voter expectations have developed since early March. Note the small differences of the citizen forecasts to the PollyVote and Nate Silver's FiveThirtyEight.

  • Across the 232 days from March 14th to October 29th, the mean absolute deviation between the citizen forecasts and the PollyVote was only half a percentage point.
  • Across the 154 days since May 31st (the first release of the FiveThirtyEight forecast), the mean absolute deviation of the citizen forecast and FiveThirtyEight's popular (two-party) vote forecast was even smaller: 0.4 percentage points.

 

PollyVote 2012

 
Final forecast of the
popular two-party vote
Click here for more information about this forecast.

 

Fit to be president?

Compare your biography to Obama's and see your chances of winning the 2012 U.S. Presidential Election - or find out whether your favorite politician would have a chance.

The test is based on the PollyBio model, which correctly predicted 27 of the last 29 U.S. Presidential Election winners. Click here to access the test.