All of this writing and data analysis is a lot of work! So after more than five years of posting, I've finally launched a Patreon to help pay the bills.

7/3/19

Daily polling drama isn't teaching you a thing

Seven polls of the Democratic primaries that have rolled out since the most recent debate:



What's going on here? If you want to, you can construct an elaborate narrative about how Biden and Sanders both saw their numbers drop after the debate, while Harris and Warren both saw significant jumps - and then, in a sharp reversal, everyone saw their polling return to their initial numbers (except for Biden, who is now ten points lower than he used to be). If you've been watching this play out in real time, then you were treated to a surge of triumphalism from opponents of Sanders, and then (starting this morning) a round of schadenfreude from his supporters during the rebound.

Now, let's look at this another way by comparing the average of these seven polls to an average of the seven polls before the debates:



What's changed? Mostly just a seven-point drop for Biden and a seven-point surge for Harris. There's a little bit of motion with the rest of the contenders, but entirely within the margin of error.

And doesn't this basically correspond with what you would expect? Harris's exchange with Biden was possibly the most dramatic moment of this election so far, and easily the most gripping episode in the debates, while the other three contenders put on (for them) pretty ordinary performances. If you were a so-called "low information voter" who only caught wind of the Biden-Harris exchange, that was probably more than enough to give you a good idea of how the numbers changed.



Here's the last month of polling - but instead of tracking every single data-point, these lines chart out an average of the last six polls from moment to moment. Predictably, this smooths out a lot of the turbulence in the polls - but notice how it captures all of the basic trends that everyone has covered for the last month. First we get some appreciable tightening between Sanders and Warren, and then at the end of the month everyone ends up at around 14% while Biden begins to drop.

Polls are turbulent. On average they tend to capture fundamental trends within the primaries, but individual polls tend to fluctuate within margins of error, or due to odd ephemeral dynamics that evaporate within a day or two. And most pollsters working in the media know this perfectly well - but they also know that turbulence creates drama, and that they can bait in viewers and readers by pretending that every little shift signals a new game-changing trend.

If you like infotainment based entirely on meaningless statistical fluctuations, then keep looking at graphs like the first one. But if you want to have a realistic sense of what's actually going on, you're better off looking at the second.