Pollster Ann Selzer ending election polling, moving 'to other ventures and opportunities'

[u/BastianMobile]

https://eu.desmoinesregister.com/story/opinion/columnists/2024/11/17/ann-selzer-conducts-iowa-poll-ending-election-polling-moving-to-other-opportunities/76334909007/

[u/JeromesNiece]

It was unwarranted.

Polling is a crapshoot. There is no such thing as a pollster that has some brilliant methodology or sixth sense that can lead to consistently better results than other pollsters. Her "track record" was nothing more than a series of dumb luck.

Every local pollster in the country has been engaged in a coin flipping competition, and you all fell for the "winner" that got the most heads in a row.

This is why you need to just throw every result into an average and not get too excited.

[u/justbuildmorehousing → u/JeromesNiece]

I think thats really diminishing that she had stuck her neck out from the pack in several elections previously and ended up basically nailing it. Its not a random number generator out there and she really did seem to have a good method for polling Iowa really accurately

[u/West-Code4642 → u/justbuildmorehousing]

I still don't get how her poll shifted so much in one month. She has trump up s month before. Guess she had loud Harris voters and shy trump voters

[u/Khiva → u/West-Code4642]

I would really love for her - or someone - to give a breakdown on this.

[u/eetsumkaus → u/Khiva]

Didn't she for her interviews after the results came out? Basically older women were driving the Harris +4.

[u/jaydec02 → u/Khiva]

Her sample was basically Biden +3. She polled a collection of Biden voters.

Pollsters began weighting recalled vote this year but she never had a more stringent methodology than dialing random people, and weighing by age, sex, and location. Her methodology was from the early 2000s and it finally bit her.

[u/ArcFault → u/jaydec02]

Like all polling methodology - it works until it doesn't.

[u/Khiva → u/jaydec02]

Her methodology was from the early 2000s and it finally bit her.

Wild that it worked even in this cycle until the very end though, no?

[u/PM_me_ur_digressions → u/Khiva]

She said something about how none of her initial respondents had changed their minds, but that the new additions to the poll were women who seemed very fired up, or something along those lines.

[u/ThePevster → u/Khiva]

Nate Silver talked about it on his blog. Basically she uses an older polling methodology where you randomly dial numbers, and that methodology tends to skew quite Democratic.

[u/Khiva → u/ThePevster]

Interesting. I remember reading that her unique skill came from her ability to assemble a sample set. If it was all just random then I got something wrong somewhere.

Still, wild that this methodology would be so right until it was unfathomably wrong.

[u/JeromesNiece → u/justbuildmorehousing]

"She got heads 7 times in a row, she must have the best coin flipping technique. The results speak for themselves."

[u/ddddall → u/JeromesNiece]

She wasn't just directionally correct, she was within a certain range of the final result too. That's not something you can model with a coin flip

[u/JeromesNiece → u/ddddall]

Add more dimensions to the coin, then! Roll a six-sided die, whoever rolls the most sixes is the most skilled!

The point is that her success was entirely indistinguishable from luck. With hundreds of local pollsters, one of them is going to look the best over the course of several election cycles by dumb luck alone. But that "best" pollster is not any more likely to be predictive going forward.

[u/cc_rider2 → u/JeromesNiece]

You say that methodology doesn’t matter when it comes to pollsters but do you have anything to back that up? Nate Silver and the folks at 538 do seem to think that certain pollsters have better methods and results. You’re just asserting it without giving any evidence or reasoning.

[u/Stanley--Nickels → u/cc_rider2]

If her methodology were exactly perfect with no response biases, she’d have had a 1 in 7 chance of missing the margin by 10 points. And even the best pollsters are nowhere near perfect.

[u/cc_rider2 → u/Stanley--Nickels]

Uh, no? Assuming a +/- 3% margin of error and 95% confidence interval, the odds of being off by 10 points is 1 in 1164.

[u/Stanley--Nickels → u/cc_rider2]

I’m out, but I’m guessing you applied to the margin of error to the difference between the candidates, but it’s the margin of error for a given candidate. The margin between them is double that.

[u/PersonalTeam649 → u/cc_rider2]

The truth is that it can be extremely hard to know whether a forecaster or pollster is getting lucky or is genuinely skilled, and I don’t think we have enough data to make an extremely confident call on how good Selzer is.

[u/JeromesNiece → u/cc_rider2]

I didn't say that methodology doesn't matter. It does matter. But methodological quality tops out very fast. Once you're taking representative samples of the population in question there's not much else to do. Which is why Silver and 538 and The Economist's weights for pollster quality are not very large. Whereas this subreddit was convinced that Seltzer had some magic sixth sense that meant her methodology was worth 10x more than other pollsters. Even after factoring in Selzer's poll and giving it an appropriately higher weight for quality, every modeler still only gave Harris a ~15% chance to win Iowa and ~50% chance to win midwestern swing states. Way lower than what this subreddit was supposing.

[u/IsNotACleverMan → u/JeromesNiece]

But I thought this sub was data driven.

[u/NeolibsLoveBeans → u/JeromesNiece]

Local man is loudly confident of his ignorance

[u/PleaseGreaseTheL → u/NeolibsLoveBeans]

Reminds me of people who still screech about nate silver even though he was right, and nobody could, or did, predict this electoli9n even remotely

Idk why some people hate him so hard.

[u/NeolibsLoveBeans → u/PleaseGreaseTheL]

Because he’s an asshole

[u/WOKE_AI_GOD → u/JeromesNiece]

I agree, I always find it annoying when people attempt to demonstrate the predictive power of some signal by being like they got it right X times in a row. Like Lichtman and his keys. Think about how many people over that period of time had a similar idea - I'll come up with a series of factors, and it they go this way or that it predicts the election. How many of them have up after it failed on the first or second try? Somebody was going to get a win streak eventually.

[u/jquickri → u/JeromesNiece]

Seriously this. People really don't understand that polls have to be understood in aggregate and there's no person out there who is some kind of poll savant. Hell Nate silver has made an entire career off the fact that he called every state once despite being wrong many times before and after.

[u/blewpah → u/JeromesNiece]

you all fell for the "winner" that got the most heads in a row.

Did anyone say with certainty that she was going to be accurate? Did she even say that?

[u/MariaKeks → u/blewpah]

I don't think anyone claimed it was literally 100% certain, but that's asking too much. However, there definitely were a lot of comments hyping her up, implying she was a lot more accurate than other pollsters. A selection of quotes:

If I were a betting man, I would NOT be betting against the GOAT Ann Selzer. (475 points)

Look at the early voting number + the gender gap in Iowa…obviously Selzer is right (57 points)

The lady doesn’t miss. If she’s off 4 points this time around, then Blexas and Blorida are in play and Dems probably win a trifecta. (42 points)

It's pretty clear other pollsters care more about their perception than they do about being accurate. (57 points)

Go home and play with Fivey, Nate. The adults (key man and seltzer lady) are predicting the president. (17 points)

Note that a lot of these comments were downvoted after they turned out to be wrong, so before the election they would have been rated even higher.

Meanwhile:

The Ann Selzer Methodology:

Release an outlier poll favoring Democrats and enjoy the adulation for a few days. Once Kamala loses Iowa (duh?), who knows what she'll do, but I couldn't care less. (0 points)

[u/Khiva → u/MariaKeks]

there definitely were a lot of comments hyping her up, implying she was a lot more accurate than other pollsters

Yeah look at this absolute clown:

Ann Selzer Is The Best Pollster In Politics

Five thirty .... something or other. Never heard of them. What do they know?

[u/MariaKeks → u/Khiva]

Just because you read something on the internet doesn't mean it's true.

This is actually another great example of the cherry picking of evidence that is at the root of the issue. Let me explain.

You are citing a years-old fluff piece by relative nobody Clare Malone who put “best pollster” in the headline without actually quantifying that in the copy to justify your belief that the Selzer poll was right. The justification hinges on the fact that 538 published Malone's article, and this makes some sense, since 538 specializes in election predictions, so they have some credibility on gauging the accuracy of pollsters.

But if Clare Malone is credible through association with 538, then certainly that applies also to Nate Silver, the literal founder of 538, who wrote that “Selzer will probably be wrong”.

Why is it that you trusted Malone's old article more than Silver's analysis? My guess is that you wanted Selzer to be right, so you wanted Malone to be right, and Silver to be wrong. But wanting something to be true doesn't make it so.

And now you come here, posting sarcastically, implying that because you read Selzer is the best pollster on 538, you were totally justified in your belief that the Selzer poll would be correct. But what you didn't mention is that you chose to ignore all evidence to the contrary about Malone, just like you chose to ignore all evidence to the contrary about Selzer, and all evidence to the contrary about Harris.

[u/Pristine-Aspect-3086 → u/MariaKeks]

in the very nate silver article you're referencing he literally calls selzer and NYT/sienna the two best pollsters

[u/MariaKeks → u/Pristine-Aspect-3086]

The thesis of that article is “Selzer is great, but this time she is likely wrong”.

If you read that and your only take-away is “Selzer is great” then you're guilty of exactly the kind of selective reading I am talking about.

[u/Khiva → u/Pristine-Aspect-3086]

And the guy is complaining about "cherry picking."

The internet is immune is irony.

[u/Gooch_Limdapl → u/blewpah]

Sub-textually, yeah, every time her track record was mentioned to bolster the poll.