A recent polling “primer” intended for journalists has some useful cautions for all of us being inundated with reports about the “latest polling results” in this weird campaign season.
We are always (usefully) reminded that even the best polls are but a snapshot of public opinion at the time the poll is fielded, so results depend upon what voters have heard and seen at that particular time. Subsequent campaigning can–and more often than not, does–change those perceptions.
It also should not be news that some polls are more equal than others: good polls are expensive, and a lot of what’s out there is at best unreliable and at worst, garbage. Composition and size of the respondent pool (the sample), design of the questions and a number of other flaws can make some surveys worse than useless.
But in addition to those standard cautions, recent changes in communications and the willingness of the public to answer questions cast further doubt on the accuracy of even the better-designed polls.
It should go without saying that “click on our link and tell us what you think” internet polls are worthless.
The increased use of mobile phones, especially, has challenged polling operations. That’s particularly true because there are significant differences in the populations that use cell phones and those who continue to keep their landlines, posing a huge challenge for the algorithms pollsters use to compensate for inability to reach mobile devices.
Further compounding the problem, the number of people willing to talk to a pollster when they are contacted has steadily declined; some estimates are that a mere 5% of those who answer their phones are willing to answer survey questions. Even if the number in the sample is increased in an effort to compensate, it is highly likely that the people who are willing to talk differ in some relevant ways from those who aren’t.
We saw the consequences of all this recently in the Michigan Democratic primary. The best polling has come a long way since “Dewey Beats Truman”–but most of what earns headlines isn’t the best polling.
The troubling aspect of this is that even garbage polls have the ability to affect people’s perceptions and ultimately, to affect election results.
Comments