Quote:
Originally Posted by GGG
That Huff post article is pretty terrible.
Essentially they are saying that if Nate as a state at 50.1% and the 49.9% wins then a model suggesting the 49.9% chance had an 80% chance of winning is right. When the model is predicting something will happen 50.1% of the time it should be wrong 49.9% of the time.
|
I don't think 538 is above criticism, but this article is an absolutely awful attempt to do so. I see two main differences between 538 and the other forecasters. One is, as this article says, is trend-lines. But the bigger difference, which this article only vaguely touches on, is uncertainty. As I believe 538 explained it early in the campaign, their model uses a longer historical sample of elections for its understanding of how polls can move. While Huffpost and Upshot's models are based on 1980s onward, while 538's go back into the 60s and 70s, when there were more polling misses and more volatility. So this is definitely a judgement call that forecasters need to make.
But there is reason to think 538 made the wrong call on that; with the probability calls they've making, you'd expect them to be wrong far more than they've actually been wrong. It's entirely possible that while their model is great in predicting results, by including a more volatile era of electoral history their model is essentially sandbagging it a bit. I'm not saying for sure it does, but that's what I think a legitimate criticism of 538's model might focus on.
edit: I should add, I think 538 has a solid defense to this, which is that most elections, the polls will be pretty accurate; but that when they miss, they'll miss in a lot of states. So you wouldn't necessarily expect to see them miss a couple every election; but you would expect that every now and then, there's an election where the polls missed or the race swung late in a lot of states at once.