It’s said to be wrong to kick a person when he or she is down. If Monday’s New York Times/Siena poll were a person, it’s been stomped so severely that a compassionate observer would step in to stop the fight.

But even though the poll that launched a thousand headlines claiming the midterms are moving back toward Republicans, and that the so-called Dobbs effect – a shift to Democrats after the Supreme Court did away with a 50-year-old constitutional right to abortion – is subsiding, has been pretty thoroughly debunked by pollsters and progressive analysts, it still deserves attention (but no kicking here, folks).

It’s a case study of what even “good” polls can do wrong, and, maybe more important, of how journalists looking for a “new” story line hype outlier polls without understanding the first thing about what they mean – as well as the way voters should think about new polling as we get closer to the crucial election.

In case you were without a computer or television earlier this week, here’s the gist of the poll of 792 “likely voters.” In September, those polled by New York Times/Siena favoured Democrats on a “generic” congressional ballot, by one point. A month later, those polled back Republicans by four.

The big news, from the Times headline: “With elections next month, independents, especially women, are swinging to the GOP. despite Democrats’ focus on abortion rights.” The economy, the poll found, mattered much more to voters than abortion.

And despite the fact that a “gender gap” showing women favouring Democrats has been a defining feature of American politics since Ronald Reagan’s election in 1980, the poll showed women dividing their votes equally between the two parties.

“The poll showed that Republicans had entirely erased what had been an 11-point edge for Democrats among women last month in 2022 congressional races to a statistical tie in October,” the Times wrote.

The detail that got the most hype, though, from the Times write-up: “The biggest shift came from women who identified as independent voters. In September, they favoured Democrats by 14 points. Now, independent women backed Republicans by 18 points – a striking swing given the polarisation of the American electorate and how intensely Democrats have focused on that group and on the threat Republicans pose to abortion rights.”

Let me stop here and say: It’s entirely possible, maybe likely, that momentum has swung in Republicans’ favour over the last month. The Democrats’ summertime wins – a gun safety bill, the Inflation Reduction Act (and especially its provisions cutting Medicare costs for seniors), declining gas prices and the Kansas victory on abortion – may have faded in memory.

Persistent inflation, gas prices’ rising again, the declining stock market and other forms of economic pain are genuine concerns among voters. “We may be returning to a more traditional model,” where the president’s party almost always loses congressional seats in its first midterm election, admits Democratic pollster Celinda Lake, who is nonetheless a fierce critic of the latest Times/Siena poll.

TargetSmart CEO Tom Bonier, another poll critic, agrees. “There’s a plausible case to be made in this election that the GOP will do well,” Bonier says. But the Times/Siena poll doesn’t make it. “I break down its flaws to remind us: Never pay attention to a single poll. View these polls as a ‘what if’ scenario,” Bonier says. “They’ve constructed a ‘likely voter model,’ with a potential outcome that’s plausible.”

But that “likely voter model” is problematic in this election, some pollsters say. “We can’t build a likely voter model that says ‘Let’s go look at the last time half the population lost a fundamental right,’” Bonier notes.

The biggest flaw in the poll, which was sadly the fact most hyped by mainstream journalists, was that alleged 32-point swing among “independent women” to Republicans. It’s based on 95 women and its margin of error is at least 10 points. “Nobody should have reported that as truth,” Bonier told me.

Also, many journalists don’t seem to understand what’s behind “likely voter” models. Even strong polling organisations make choices about which demographic groups are likely to turn out, and in what numbers, based on shifting political winds.

To polling newbies: That means the Times/Siena did not poll the identical 792 voters in September and October to find those disturbing swings towards Republicans. Good pollsters and Times/Siena is considered one of the best, have to make judgments each time they poll about who’s “likely” to vote.

But that means month-to-month “swings” in voter preference like the Times-reported September Democratic advantage that tilted towards Republicans in October represent apples-to-oranges comparisons.

It’s true that many pollsters overestimated Democratic margins in 2020, and even predicted wins that turned to losses in 2016 (sorry, President Hillary Clinton). It’s widely accepted that they somehow missed a lot of Republican voters, whether because they didn’t reach them or their “likely voter” models were skewed.

“They don’t want to have a Democratic bias in their polls/likely voter models this year,” Bonier says, “so it’s possible they’re overcorrecting with Republicans.”

• The Nation report