Pollster Ann Selzer ending election polling, moving 'to other ventures and opportunities'

[u/BastianMobile]

https://eu.desmoinesregister.com/story/opinion/columnists/2024/11/17/ann-selzer-conducts-iowa-poll-ending-election-polling-moving-to-other-opportunities/76334909007/

[u/Goldenboy451]

Yeah I don't think it was at all unwarranted to think that she was on to something with her poll that other outlets had missed given her track record. Being off by 17 points is an astonishing break in form.

[u/JeromesNiece → u/Goldenboy451]

It was unwarranted.

Polling is a crapshoot. There is no such thing as a pollster that has some brilliant methodology or sixth sense that can lead to consistently better results than other pollsters. Her "track record" was nothing more than a series of dumb luck.

Every local pollster in the country has been engaged in a coin flipping competition, and you all fell for the "winner" that got the most heads in a row.

This is why you need to just throw every result into an average and not get too excited.

[u/justbuildmorehousing → u/JeromesNiece]

I think thats really diminishing that she had stuck her neck out from the pack in several elections previously and ended up basically nailing it. Its not a random number generator out there and she really did seem to have a good method for polling Iowa really accurately

[u/JeromesNiece → u/justbuildmorehousing]

"She got heads 7 times in a row, she must have the best coin flipping technique. The results speak for themselves."

[u/ddddall → u/JeromesNiece]

She wasn't just directionally correct, she was within a certain range of the final result too. That's not something you can model with a coin flip

[u/JeromesNiece → u/ddddall]

Add more dimensions to the coin, then! Roll a six-sided die, whoever rolls the most sixes is the most skilled!

The point is that her success was entirely indistinguishable from luck. With hundreds of local pollsters, one of them is going to look the best over the course of several election cycles by dumb luck alone. But that "best" pollster is not any more likely to be predictive going forward.

[u/cc_rider2 → u/JeromesNiece]

You say that methodology doesn’t matter when it comes to pollsters but do you have anything to back that up? Nate Silver and the folks at 538 do seem to think that certain pollsters have better methods and results. You’re just asserting it without giving any evidence or reasoning.

[u/Stanley--Nickels → u/cc_rider2]

If her methodology were exactly perfect with no response biases, she’d have had a 1 in 7 chance of missing the margin by 10 points. And even the best pollsters are nowhere near perfect.

[u/cc_rider2 → u/Stanley--Nickels]

Uh, no? Assuming a +/- 3% margin of error and 95% confidence interval, the odds of being off by 10 points is 1 in 1164.

[u/Stanley--Nickels → u/cc_rider2]

I’m out, but I’m guessing you applied to the margin of error to the difference between the candidates, but it’s the margin of error for a given candidate. The margin between them is double that.

[u/PersonalTeam649 → u/cc_rider2]

The truth is that it can be extremely hard to know whether a forecaster or pollster is getting lucky or is genuinely skilled, and I don’t think we have enough data to make an extremely confident call on how good Selzer is.

[u/JeromesNiece → u/cc_rider2]

I didn't say that methodology doesn't matter. It does matter. But methodological quality tops out very fast. Once you're taking representative samples of the population in question there's not much else to do. Which is why Silver and 538 and The Economist's weights for pollster quality are not very large. Whereas this subreddit was convinced that Seltzer had some magic sixth sense that meant her methodology was worth 10x more than other pollsters. Even after factoring in Selzer's poll and giving it an appropriately higher weight for quality, every modeler still only gave Harris a ~15% chance to win Iowa and ~50% chance to win midwestern swing states. Way lower than what this subreddit was supposing.

[u/IsNotACleverMan → u/JeromesNiece]

But I thought this sub was data driven.

[u/NeolibsLoveBeans → u/JeromesNiece]

Local man is loudly confident of his ignorance

[u/PleaseGreaseTheL → u/NeolibsLoveBeans]

Reminds me of people who still screech about nate silver even though he was right, and nobody could, or did, predict this electoli9n even remotely

Idk why some people hate him so hard.

[u/NeolibsLoveBeans → u/PleaseGreaseTheL]

Because he’s an asshole