Pollster Ann Selzer ending election polling, moving 'to other ventures and opportunities'

[u/BastianMobile]

https://eu.desmoinesregister.com/story/opinion/columnists/2024/11/17/ann-selzer-conducts-iowa-poll-ending-election-polling-moving-to-other-opportunities/76334909007/

[u/justbuildmorehousing]

I think thats really diminishing that she had stuck her neck out from the pack in several elections previously and ended up basically nailing it. Its not a random number generator out there and she really did seem to have a good method for polling Iowa really accurately

[u/West-Code4642 → u/justbuildmorehousing]

I still don't get how her poll shifted so much in one month. She has trump up s month before. Guess she had loud Harris voters and shy trump voters

[u/Khiva → u/West-Code4642]

I would really love for her - or someone - to give a breakdown on this.

[u/eetsumkaus → u/Khiva]

Didn't she for her interviews after the results came out? Basically older women were driving the Harris +4.

[u/jaydec02 → u/Khiva]

Her sample was basically Biden +3. She polled a collection of Biden voters.

Pollsters began weighting recalled vote this year but she never had a more stringent methodology than dialing random people, and weighing by age, sex, and location. Her methodology was from the early 2000s and it finally bit her.

[u/ArcFault → u/jaydec02]

Like all polling methodology - it works until it doesn't.

[u/Khiva → u/jaydec02]

Her methodology was from the early 2000s and it finally bit her.

Wild that it worked even in this cycle until the very end though, no?

[u/PM_me_ur_digressions → u/Khiva]

She said something about how none of her initial respondents had changed their minds, but that the new additions to the poll were women who seemed very fired up, or something along those lines.

[u/ThePevster → u/Khiva]

Nate Silver talked about it on his blog. Basically she uses an older polling methodology where you randomly dial numbers, and that methodology tends to skew quite Democratic.

[u/Khiva → u/ThePevster]

Interesting. I remember reading that her unique skill came from her ability to assemble a sample set. If it was all just random then I got something wrong somewhere.

Still, wild that this methodology would be so right until it was unfathomably wrong.

[u/JeromesNiece → u/justbuildmorehousing]

"She got heads 7 times in a row, she must have the best coin flipping technique. The results speak for themselves."

[u/ddddall → u/JeromesNiece]

She wasn't just directionally correct, she was within a certain range of the final result too. That's not something you can model with a coin flip

[u/JeromesNiece → u/ddddall]

Add more dimensions to the coin, then! Roll a six-sided die, whoever rolls the most sixes is the most skilled!

The point is that her success was entirely indistinguishable from luck. With hundreds of local pollsters, one of them is going to look the best over the course of several election cycles by dumb luck alone. But that "best" pollster is not any more likely to be predictive going forward.

[u/cc_rider2 → u/JeromesNiece]

You say that methodology doesn’t matter when it comes to pollsters but do you have anything to back that up? Nate Silver and the folks at 538 do seem to think that certain pollsters have better methods and results. You’re just asserting it without giving any evidence or reasoning.

[u/Stanley--Nickels → u/cc_rider2]

If her methodology were exactly perfect with no response biases, she’d have had a 1 in 7 chance of missing the margin by 10 points. And even the best pollsters are nowhere near perfect.

[u/cc_rider2 → u/Stanley--Nickels]

Uh, no? Assuming a +/- 3% margin of error and 95% confidence interval, the odds of being off by 10 points is 1 in 1164.

[u/Stanley--Nickels → u/cc_rider2]

I’m out, but I’m guessing you applied to the margin of error to the difference between the candidates, but it’s the margin of error for a given candidate. The margin between them is double that.

[u/PersonalTeam649 → u/cc_rider2]

The truth is that it can be extremely hard to know whether a forecaster or pollster is getting lucky or is genuinely skilled, and I don’t think we have enough data to make an extremely confident call on how good Selzer is.

[u/JeromesNiece → u/cc_rider2]

I didn't say that methodology doesn't matter. It does matter. But methodological quality tops out very fast. Once you're taking representative samples of the population in question there's not much else to do. Which is why Silver and 538 and The Economist's weights for pollster quality are not very large. Whereas this subreddit was convinced that Seltzer had some magic sixth sense that meant her methodology was worth 10x more than other pollsters. Even after factoring in Selzer's poll and giving it an appropriately higher weight for quality, every modeler still only gave Harris a ~15% chance to win Iowa and ~50% chance to win midwestern swing states. Way lower than what this subreddit was supposing.

[u/IsNotACleverMan → u/JeromesNiece]

But I thought this sub was data driven.

[u/NeolibsLoveBeans → u/JeromesNiece]

Local man is loudly confident of his ignorance

[u/PleaseGreaseTheL → u/NeolibsLoveBeans]

Reminds me of people who still screech about nate silver even though he was right, and nobody could, or did, predict this electoli9n even remotely

Idk why some people hate him so hard.

[u/NeolibsLoveBeans → u/PleaseGreaseTheL]

Because he’s an asshole