David Strömberg

WH 2008
Post-election evaluation

Phone: +46 8 16 43 77
Fax: +46 8 16 14 43
E-mail: info@iies.su.se
Webmaster:
www@iies.su.se

Last updated,

 October 1, 2008

The model can be evaluated in a number of dimensions. I will investigate three: how well did the model predict

• the vote outcomes,
• the uncertainty in the vote outcomes,
• and the distribution of campaign efforts?

I will get back to the campaign when I have collected the data, and now discuss first two.

How did the pre-election predictions do?

I will discuss two predictions, the last one I made, on November 3, and the one that Justin Wolfers and I discussed June 12 in the WSJ. The national vote prediction from November 3 was spot-on. It gave Obama 53 percent of the two-party vote, which is also what he got. The May 29 prediction was 2 percent off, giving Obama 51 percent of the two-party vote share.

The November 3 prediction was also accurate at the state-level. The standard deviation of the discrepancies in vote shares, excluding D.C. and Hawaii, is 2.5% (same as Nate Silver's model, according to the analysis of Adrew Gelman). The average absolute prediction error was 1.7 percent. If the present count holds, so that McCain wins Missouri and Obama wins North Carolina, then the model correctly predicted the winner in all states, except for Indiana and North Carolina.

The predictions from May 29 rely much less on opinion polls but are still pretty accurate, see Figure 2. One standard deviation in vote discrepancies in state vote shares is 3.5 and the average absolute prediction error is 2.2.

Figure 1. Prected and actual votes, Nov. 3 forecast 

Figure 2. Prected and actual votes, May 29  forecast 

The table below restates the forecasted democratic vote shares and win percentages as of November 3, it now also includes the actual vote shares.

How well did the model estimate
the uncertainty in the predictions?

While this may seem a nerdy comment, I'll still make it since it is important. Most of the discussion is about predicting vote outcomes. Still, to measure the probability of Obama winning, it is as important to correctly measure the uncertainty in the prediction. For example, in this election, more uncertainty would lower Obama's chances of winning since he was the front runner.

Something can be said about the independent state-level shocks. Using the out-of-sample prediction errors from previous elections, I estimated one standard devation of these errors to be 2.7% in the November 3 estimation. The measured standard deviation of the ex-post errors in 2008 was very similar to this, 2.5% (2.9% including Hawaii).

However, most of the uncertainty in who wins derives from national shocks. The distribution of these shocks is not precisely estimated since we have only a few observations. If we consider election post WWII similar, then we had 15 observations. Consequently, all measures of win probabilities are imprecise.