Andy Crosby is the guy you want at your dinner party in the next few months. With about six months until "Decision 2024," he's elbows deep in presidential polls, scouring their methodology, debunking them when necessary.
Crosby, who is an assistant professor in the School of Public Policy, became immersed with surveys and polls about 10 years ago. He teaches survey research at UCR, and he designs, consumes, or analyzes polls and surveys every week, even when it isn't a presidential election year.
"Although we rely on polling and survey research to inform us on a wide variety of public policy matters, relatively less energy has been dedicated to understanding how to better create and consume surveys," said Crosby, who joined UCR's faculty in fall 2023. "My goal is to work with students and the broader community to help people be more prepared to both design and consume surveys."
With the presidential primaries and elections approaching, the numbers of polls is increasing - which means Crosby is spending a lot more time comparing polls. Many leading polls have Donald Trump edging out Joe Biden. But are those polls any good? And is it too early to pay any attention to polls? It's a good time to check in with UCR's resident polling expert.
Stories in the past couple days have asserted Trump is leading Biden in five of six key states. How do you access the methodology behind these polls?
Siena College/New York Times is a trusted pollster and I trust the results of these polls. Their sampling methodology allows for more precise results (for example, the polls are stratified by party to make sure that the poll reaches voters from each party) and they are transparent with both their findings and methods. As a consumer of this poll, it should be reassuring to see the extensive disclosure of both their findings and methods.
What did the polls tell us about a Biden-Trump election in May 2020?
Most major polls had Biden leading Trump, although the margin varied somewhat by poll and by date. For example, a Washington Post-ABC News poll conducted May 25-28, 2020 showed Biden leading Trump, 53-43 percent.
In the May 19 USA Today, columnist Rex Huppke asserts: “Anyone using polling data to assert anything with even a modicum of certainty is a fool.” He is referring to the fact presidential elections are six months out. Is he right?
To some degree. Polls represent a snapshot in time. I generally view polls today as accurate for what they are trying to measure – that is, what voters think today. I would not yet use current polls to predict the outcome in November. Voters do sometimes change their minds between May and November. Part of the reason they may change their mind is due to events that occur during that time – for example, we will likely have at least two presidential debates that may help to inform voters on which candidate more closely represents their views. As such, polls closer to the election are likely to be more accurate.
In what proximity to presidential elections is it worth tuning in to polls?
Even though current polls will not necessarily predict the outcome of the November election, I view current polls as valuable. These polls tell us what voters think now – for example, what they feel the biggest issues are facing the country. That said, if I wanted to predict the election, the polls as we get closer to the election will likely be the most accurate. Recent research demonstrates this increase in accuracy over time and argues that the optimal lead time is approximately two to three months prior to the election, depending on how much error one is willing to accept. I also expect to see a large increase in the number of polls in the last 100 days prior to the election based on previous data.
Is there data that speaks to the historical accuracy of polls six months before a presidential election?
Polls six months prior to the election tend to have larger errors, with six months out still including a considerable amount of error. As I mentioned above, recent research suggests that two to three months out is an optimal lead time.
What do the polls tell us about the effect Robert F. Kennedy Jr. may have on the election returns?
The polls tell us that Kennedy certainly has support among the electorate, but I would use caution translating that into election returns at this point. Part of the reason for this is that Kennedy is not yet on the ballot in all 50 states, although he has an active effort to achieve ballot access in more states. Once we have a fuller picture of Kennedy’s ballot access, we can start to better gauge the potential effect of Kennedy’s candidacy on elections.
What are your favorite polls, and why?
I personally admire the work of NORC at the University of Chicago. More generally, polls that use probability sampling (that is, random selection) are generally considered among the highest-quality polls. These polls are generally considered more accurate than nonprobability samples (for example, approaching someone in a mall and asking them questions), which have become more popular in recent years. Transparency about polling methods (for example, knowing who sponsored the research) is also extremely important. The American Association for Public Opinion Research (AAPOR) Transparency Initiative is the gold standard for this type of disclosure, and I would trust polls that are a member of the AAPOR TI more than polls that are not.
Are there widely quoted polls for which you question the methodology?
Sometimes the answer to this question depends more on the method than the outlet. Even national media outlets (such as the Washington Post) sometimes report the results of polls that use different methods for different stories. I generally use extra caution with polls that do not use probability sampling and/or polls that disclose their methods. The difference in accuracy between randomly sampling registered voters and walking up to people in the mall can be stark. Similarly, if you as a consumer of a poll or survey cannot find basic information about how the poll was conducted (e.g., who was sampled, who funded the poll), I would be reluctant to trust the results of that poll.
Is it more difficult to poll voters accurately today than it once was? Easier?
In general, high-quality polls are quite accurate today, and have been for quite some time. The difference between today and past years is that response rates are going down – for example, telephone response rates today are typically less than 10%. Although lower response rates do not necessarily mean lower quality results, more effort and resources may be needed to reach the same quality estimate.
