Measuring American polling error, 2000–2024

Summary

This paper sets out how we measure polling error in the United States: which polls count as final polls, how the error is defined, and what the record since 2000 shows. Our database holds 4,688 usable final polls of 806 Senate, governor and presidential races by state from 2000 to 2025, 35 final generic-ballot polls at the 13 House elections from 2000 to 2024, and 166 final national presidential polls at the seven presidential elections.

The record has four features. First, the polls of a single race miss by a lot: the average of a race's final polls has missed the result by 4.6 points on the margin in a typical Senate or governor race. Second, misses are shared: in a bad cycle almost every race misses the same way, as in 2020, when the final Senate and governor polls overstated the Democrats by 6.9 points on average across 35 races. Third, the direction has leaned one way: the mean miss of the 12 cycles with enough races was +1.45 points towards the Democrats, positive in 8 of them, and the final generic ballot overstated the Democrats at 11 of 13 House elections. Fourth, the lean is strongest in the most Republican states: since 2018, polls of states at least 15 points more Republican than the nation have overstated the Democrats by 4.3 points.

Each feature has a use in the forecast. The size of a single race's miss sets the error of each poll average (State polls: the pollster-adjusted average and the blend); the shared part sets the national error in the simulation (Correlated error and the backtests of 2018, 2022 and 2024); the direction is what the half-correction tests (Forecasting the 2026 midterms: an overview, The generic ballot: the national vote behind every race); and the regional pattern is one reason the simulation draws errors by state and region.

What is measured, and against what

American elections are, with few exceptions, contests between a Democrat and a Republican, and the quantity that decides them is the margin between the two. We therefore measure error on the margin, Democrat minus Republican, in percentage points. This differs from the European papers, which measure each party's share (Measuring polling error in England, 2015–2024): with two main parties, the margin carries almost all the information, and it is what decides who wins.

Results are the official returns: the MIT Election Data and Science Lab's Senate and presidential files, J. Cha's governor file for 2006 to 2024 and Wikipedia's summary tables for 2000 to 2005, 2023 and 2025 (The US data: sources, grades and checks). A race enters the record only if a Democrat and a Republican each took at least 5% and finished first and second. That leaves out uncontested races, races where an independent was the main challenger, and California's and Washington's same-party general elections.

Definitions

Let \(v_r\) be the result margin of race \(r\) (the Democrat's share minus the Republican's share of all votes) and let \(y_{j,r}\) be the margin in final poll \(j\), one of the \(n_r\) final polls of the race. A poll's margin is taken on the decided vote:

where \(\text{decided}_j\) is the poll's total less its undecided share. A poll whose decided share is under 30% is not used. The error of the race's average is

The sign convention is the same everywhere on the site: error = poll minus result. A positive error means the polls overstated the Democrat. Each poll counts once, whatever its sample, and partisan polls count as published.

For a cycle \(c\) with races \(R_c\), the mean miss measures direction and the average miss measures size:

Odd-year governor races (Virginia and New Jersey, Kentucky, Louisiana and Mississippi) are counted with the even year before them, so the 2024 cycle includes the New Jersey and Virginia governor races of 2025.

The evidence: final polls

A final poll is the last poll by a given pollster in a given race, ending in the month before polling day. One poll per pollster per race is kept. Since a check of 13 October 2026, a past final poll whose figures, undecided included, add up to more than 103 is left out of the record: 31 such polls were found, most counting the same voters as both "other" and "undecided", and the rule is 103 rather than 100 because published figures are rounded.

That gives 4,688 final polls of 806 races from 2000 to 2025: 2,928 polls of 550 Senate and governor races, and 1,760 polls of 256 presidential contests by state. Of the 4,688, 567 were paid for by a campaign, party or allied group or come from a pollster that works mainly for one side; 405 of those were on the Republican side and 162 on the Democratic side. The number of races with final polls grew from a handful in 2000 to more than 60 in most midterm cycles after 2008.

Almost all past final polls come from Wikipedia's tables (grade C on our scale), checked against the pollster's own release where that could be read (The US data: sources, grades and checks). That is a real limitation, and we return to it at the end.

A worked example: Ohio's Senate race, 2024

Sherrod Brown, the Democratic senator, lost to Bernie Moreno by 3.6 points (Moreno 50.1%, Brown 46.5%). Twelve pollsters published a final poll in the last month:

Table: Final polls and result, Ohio Senate 2024. Margins on the decided vote by equation (1), points.

The average of the twelve margins is −0.74 and the result −3.62, so by equation (2) the race's error is +2.9 points. Eleven of the twelve polls overstated Brown. This is the pattern of a shared miss, and equation (6) below measures it: the square of the average's error is 57% of the average squared error of the individual polls. Choosing a different pollster would have helped little; most of the miss was common to all of them.

The record by cycle

The table applies the same calculation to every cycle of Senate and governor races, the races the model learns from.

Table: Senate and governor races with final polls, by cycle. Mean and average miss by equation (3), points of margin; positive means the polls overstated the Democrats. "Winner right": races where the average's leader won. 2000 (four races) is in the total only.

Four points stand out.

2020 was the worst cycle. The polls overstated the Democrats by 6.9 points on average, and the average miss (7.1) was almost the same size as the mean: nearly every race missed in the same direction. 2016 and 2014 were the next largest, both towards the Democrats.

The direction has not been constant. The polls understated the Democrats in 2006, 2010 and 2012, and were right on average in 2018. A forecaster in 2013, looking at the three previous cycles, would have expected the polls to understate the Democrats. The record since then has gone the other way in five of six cycles.

Size and direction vary independently. 2018 had a mean miss of almost zero and an average miss of 3.7 points: individual races missed in both directions. 2020 had both large. The model treats these separately: the average miss of single races sets the error of each race's average, and the mean miss of the cycle sets the shared error.

The polls usually pick the winner. The leader in the final-poll average won 509 of the 550 races (93%). Most races are not close, and a five-point miss changes the winner only where the race is within five points.

Senate and governor races

The two offices miss alike. Across all cycles, the 307 Senate races have a mean miss of +1.27 and an average miss of 4.68 points; the 243 governor races +1.29 and 4.40. In single cycles they can differ: in 2002 governor polls overstated the Democrats by 4.4 points and Senate polls by 1.7; in 2024 Senate polls by 2.9 and governor polls by 0.7. Some of that is the small number of races per office in a cycle.

Presidential races by state, and all races together

Adding the presidential contests by state gives the table on the site's accuracy page, 806 races in all:

Table: Senate, governor and presidential races by state with final polls. Points of margin.

Presidential races follow the same pattern, with a larger miss in 2020 and 2016 and a smaller one in 2008. The model's statewide correction and its shared error are fitted on Senate and governor races only, because those are the races it forecasts; the presidential races are used for pollster house effects, records and the partisan lean (Rating and adjusting American pollsters).

Where the polls miss: red, swing and blue states

We sort the 550 Senate and governor races by the state's presidential lean before the election (three parts its last presidential margin against the nation's, one part the one before; The fundamentals: partisanship, the national vote and incumbency).

Table: Final-poll miss by the state's presidential lean. Points of margin.

The pattern runs one way across the whole period and more steeply since 2018: polls of the most Republican states have overstated the Democrats, polls of the most Democratic states have slightly understated them, and the swing states lie between. Two cautions apply. Polls of very one-sided states are few per race (often one or two final polls), so their misses are large in both directions; the average miss of 5.75 points in the reddest states is the largest of any band. And the steeper slope since 2018 rests on four cycles, three of which (2018, 2020, 2022) carry most of it.

The model does not correct by state type. The correction in Forecasting the 2026 midterms: an overview is one number for every race, because only a rule that passes the leave-one-out test is applied and no regional correction has been tested that way. The regional pattern enters through the simulation instead, which draws shared errors for census divisions, states and demographic profiles (Correlated error and the backtests of 2018, 2022 and 2024). A site article sets out what a red-state correction would do to the Senate chances as a stress test, not as the forecast.

How much of the miss is shared

As in Measuring polling error in England, 2015–2024, the average squared poll error in a race splits exactly into the square of the average's error and the spread of the polls around their average:

Across races the same idea gives the model's error for a race average. If each poll's error is a part shared by every poll of the race, with variance \(\sigma_s^2\), plus its own, with variance \(\sigma_1^2\), the average of \(k\) polls has variance

We fit equation (5) by grouping races by their number of final polls (\(k\) from 1 to 8 or more) and regressing each group's mean squared error on \(1/k\), weighting groups by their number of races:

Table: Error of a race's final-poll average by the number of polls in it, Senate and governor races 2000–2025.

The fit gives \(\sigma_s = 4.53\) and \(\sigma_1 = 5.94\) points. A race with one final poll has an expected error of \(\sqrt{4.53^2 + 5.94^2} = 7.5\) points; one with ten polls, \(\sqrt{4.53^2 + 5.94^2/10} = 4.9\). The shared part does not fall with more polls, and it is larger than the part that does.

Part of the shared error is shared beyond the race. The root mean square of the twelve cycle means in the first table is 2.93 points: that is the part of a race's miss that every race of the same cycle shares. The simulation uses it as the national component of statewide poll error (Correlated error and the backtests of 2018, 2022 and 2024). For the Ohio example, equation (4) gives a shared share of 57%: an error of 2.9 points common to all twelve polls, and a further spread of 2.5 points around their average.

The generic ballot and the national presidential vote

Two national records sit beside the state races.

The generic ballot. Our final generic-ballot polls cover all 13 House elections from 2000 to 2024, from 35 polls by 10 pollsters. On the margin, the average of the final polls overstated the Democrats at 11 of them; the exceptions were 2008 and 2018. The largest misses came in 2002 (10.7 points), 2014 (9.8) and 2020 (9.3), each resting on one to four polls. The generic ballot: the national vote behind every race gives the table and explains why this record is corrected.

The national presidential vote.

Table: Final national presidential polls, decided vote, against the popular vote summed from the state results (MIT Election Data and Science Lab). Points of margin.

The national polls overstated the Democrat at the last three elections and understated him in 2000, 2004 and 2012. The miss in 2020 (5.0 points) is close to the statewide miss of that year.

Partisan polls

A poll paid for by a campaign, party or allied group, or from a pollster that works mainly for one side, can be compared with the independent polls of the same race. Across the 556 such final polls in races that also had an independent final poll, the partisan poll's margin sat 2.52 points towards its sponsor's side (median 2.33). Republican-side polls leaned more than Democratic-side ones (2.84 points across 401 polls against 1.70 across 155), and Senate polls more than governor and presidential polls (2.94, 2.57 and 1.71). The model takes the single figure, 2.52, off every partisan poll and halves its weight (Rating and adjusting American pollsters).

How the record feeds the model

The statewide correction. Leaving out each of the 12 cycles with five or more races and taking half of the others' mean miss off every race's average:

Table: The leave-one-cycle-out test of the statewide correction. Points of margin. "Level": the same to two decimals; the code counts a cycle as closer only if it gains more than 0.005 points.

The correction is closer in 2002 and in every cycle since 2014 except 2018, and further out in the four cycles from 2006 to 2012, when the polls had understated the Democrats. On average it gains 0.06 points, which passes the rule. Today it takes 0.73 points off every Democratic margin in a state or district average.

We also tested stricter versions. Weighting recent cycles more, or using only the nearest three, gained slightly more on the same test (as much as 0.12 points); a forward-only test, in which each cycle may learn only from earlier cycles, gained 0.02 points for the rule in use, closer in 4 of 10 cycles. The rule in use is the one every country uses; the alternatives are recorded, not applied.

The size of a race's error. Equation (5), with \(\sigma_s = 4.53\) and \(\sigma_1 = 5.94\), is the starting point for the error of every poll average (State polls: the pollster-adjusted average and the blend).

The shared error. The RMS of the cycle means, 2.93 points, is the national part of statewide poll error in the simulation (Correlated error and the backtests of 2018, 2022 and 2024).

The readings. Each of the twelve cycle means is replayed on today's races as a "miss like" reading (Forecasting the 2026 midterms: an overview).

Limits

Secondary sources. Most past final polls are transcriptions from Wikipedia, graded C. We check them where the release survives, and a 2026 check against an independent compilation found the 31 polls that add up to more than 103, but agreement between secondary sources is not the same as a pollster's own table.

Few cycles. Twelve cycles is a short record for a direction. The correction gains 0.06 points on average, and two or three cycles like 2006 to 2012 would reverse the test.

Uneven polling. Races in safe states have few polls, and those few miss by more. The cycle means weight every race equally, so a cycle's mean is pulled by thinly polled races.

Final polls only. These errors say nothing directly about polls taken months out; that is the subject of From a month out to election night.

References

Shirani-Mehr, H., Rothschild, D., Goel, S. and Gelman, A. (2018). Disentangling bias and variance in election polls. Journal of the American Statistical Association, 113(522), 607–614.

MIT Election Data and Science Lab. U.S. Senate 1976–2024 and U.S. President 1976–2024, Harvard Dataverse, CC0.

Figures as of 15 October 2026. The pages they come from are rebuilt with each new poll.