election forecasts 538 nate silver

Why most election forecasts are missing a key predictive element

Michael St. James Futures Studies, Political Trends Leave a Comment

We’re less than 100 days away from the U.S. general election. This is the Christmas, Passover, Ramadan, and Shark Week for political junkies around the world. Now is that magical time of the year when D.C. dorks and beyond start slavering over the latest poll data. Election forecasts become a mix of news item and religious talisman to be regarded, scrutinized, and hotly debated. This year is proving especially interesting, especially since one of the candidates is inviting Russian intelligence to hack our political parties and casually surmising that he might have to use tactical nukes in Continental Europe. This is one election that we all need to brace for.

Pollsters help feed the obsessives with their weekly data fix. Citizens are harangued in different ways to answer questions about candidates they scarcely know. The responses are dissected and massaged and held aloft in sodium lights. Then this is repeated in every state, and subjected to treatments by a variety of news sources. Nate Silver’s 538 is probably the most famous, ever since he called the 2012 election down to the last state. The site’s work is an extremely transparent and rigorous attempt to make concrete predictions, but also to show the methodology behind them. Pretty cool, and rather addictive. I myself am transfixed by the state of Georgia becoming more solidly blue, as I expect somebody slipped me a hallucinogen. That’s 2016 in a nutshell, I guess.

Still, while 538 no doubt represents some of the best quantitative analysis out there, I believe that the probabilities in the election forecasts are missing one critical element from the field of futures studies.

Election forecasts are missing analysis of wildcard events

Election forecasts are focused on predicting the future of the election, and for futurists this sets off red flags. Members of our profession will reflexively tell you that there isn’t a single future, but multiple futures that can result from a complex interaction of elements. And while these are complex, we believe that you can learn significantly more about potential futures by seeking information along these lines.

The methodology of different futurists can vary considerably (and cause heated, dorky arguments), but at Competitive Futures, our special sauce involves three main ingredients

Strategic trends

Trends in society, technology, economics, ecology, and politics (STEEP) that change the dynamics of the system in which we operate.

Actor analysis

Powers institutions or groups, and the (supposedly) rational decisions they would make in their interests


High-impact, low probability events that could change the future more significantly than if current trends were simply extrapolated.

Election forecasts are centered around quantitative data, past and present, which create a trend line that can be extrapolated under normal circumstances. We call these normative scenarios, basically the ones that make the most sense if things continue largely as they have in the past.

The thing is, in politics, wildcards are a frequent element, and they are rarely captured in orthodox election forecasts. When you see the “probability of winning” some state, it should have an asterisk that says *as long as things don’t get really weird.

But things do get really weird. I mean, Trump. Enough said.

The emergence of wildcard events can actually make the prior forecasts rather invalid in terms of methodology. For example, right now 538 has it marked that Trump has a 79.8% chance of winning the state of Utah. That’s a pretty precise number. And with all due respect to Nate Silver and crew, it’s bogus. The fact is there are a few reasons that this cannot actually measure the probability of the ultimate outcome. First, there has never been a candidate like Trump at the national level; he does not appear to have the same motivations as an actor, therefore a simple extrapolation of polling data does not tell the story. Second, there is insufficient data about the effect of third parties such as the Libertarian Gary Johnson to effectively extrapolate these trends. And third, Trump is almost guaranteed to say crazy shit that will alter all of these numbers. One snit with a Belgian diplomat followed by a joke about firebombing Liege, and we’re in brand new territory.

The dramatic impact of saying crazy stuff

A couple of races in in recent history convince me that candidates can now open their mouths and blow away election forecasts of “high certainty” inside of a few words.

First, let’s take my neighbor Todd Akin, Republican Congressman representing St. Louis County. Mr. Akin had aimed his sights on a U.S. senate seat, and things were going well. Akin was perceived as a solid, small-government, Constitution-focused conservative, and he was polling a reliable 7-11% ahead of moderate Democrat Claire McCaskill. This continued until this exact week in the 2012 cycle.

Then, he gave the interview.

todd akin rape shut that whole thing down
On local St. Louis television, Rep. Akin explained that he did not believe in any rape-exception in anti-abortion laws that would overturn Roe v. Wade, because rape did not really cause pregnancy. A woman’s body, he explained, had ways to shut down pregnancy from sperm present due to a rape.

This was not a popular theory.

Ultimately, despite polls showing a very comfortable route to victory, he lost by nearly 16% to McCaskill – a reversal of 25% in just a few weeks.

Another example further proves how fragile polling can be in the face of unpredictably dumb or offensive comments. Back in 2006 George Allen was running as a Republican against Democrat Jim Webb for U.S. Senate.  Allen was creaming him. June and July poll numbers showed Allen up 10 to 20%, an enormous gap.

Then he met S.R. Sidarth. A first-generation American from Virginia, Sidarth was at a rally for Allen, when the politician turned and recognized that the young man with dark skin probably wasn’t his normal constituency. Rather than let the minor intrusion slide, Allen pointed at the young man and said, “Let’s give a welcome to macaca, here. Welcome to America and the real world of Virginia.”

Of course, Mr. Sidarth was already at home in Virginia, and macaca is a type of monkey.

This was not a popular comment.

Ultimately, the reaction wasn’t as strong as the one against Akin, but Jim Webb did in fact take the race by less than a percentage point. Still, the forecasts ended up being off by 10 to 20 points. So again, the predictability of these quantitative forecasts are more fragile than we recognize.

Poll-based election forecasts are normative, but not definitive

In no way am I going to give up all the various poll trackers in this election. I lived in DC for more than a decade, and I am hit with the sickness, the obsession. But I just want to remind other junkies that the probabilities may be harder to predict than the quants tell us.

Look, if ever there were a candidate likely to throw off current forecasts, it’s Donald J. Trump. He’s said in past interviews that if poll numbers get too ugly, he might just quit. He’s yelling at parents whose child gave his life for our country. He’s got Scott Baio as a campaign proxy and Omarosa actually doing press for him around foreign policy.

Folks, we have to watch the wildcard element of this race, to stay rigorous with foresight methodology, if not then for sheer entertainment alone.

Now, excuse me, I’m going to go back to 538 and see if they have the latest numbers from Utah. If Gary Johnson surges, there a wildcard scenario where it becomes a blue state as Hillary wins with 36% of the vote.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.