The dearth of reliable, independent polling in Arizona elections is nothing new.
A long-term decline in independent polling by universities and media organizations has left IEs and other biased groups as the main source of publicly available polling for the state’s elections. Out-of-state groups pay little attention to Arizona, and when the state does get outside attention, it is generally from large auto-dial firms with partisan affiliations such as Public Policy Polling and Rasmussen Reports.
There has been no shortage of polls in the governor’s race. Since the beginning of July, at least 19 surveys of the race have been made public. But most have faced heavy criticism.
The biggest criticism of the polling in the gubernatorial race has been that so much of it has come from pro-Ducey organizations.
Polls commissioned by Conservative Leadership for Arizona, Veterans for a Strong America and political consultant Sean Noble have all shown Ducey in the lead. But the two groups have run independent expenditure ads for Ducey, and Noble is tied to both.
Experts say they generally don’t trust polls released by candidates or campaign-related groups, which only release polls when the numbers are favorable. And there are ways to make the numbers as favorable as you want.
Tucson pollster Margaret Kenski said people can manipulate the numbers by intentionally oversampling demographics that are supportive of a particular candidate, target respondents who are already known to support a particular candidate or ask loaded questions.
Good polls are supposed to rotate the order in which candidates are named because some respondents will simply choose the first name they hear, Kenski said. So a pollster can simply put the same person’s name at the top of every list to boost his or her numbers.
“We all know in the business of too many instances where people massaged the data to make it look better,” Kenski said.
Flaws of auto-dial polls
Some of the surveys of the governor’s race are independent but still questionable because they are auto-dial polls, not live-caller polls, which are more accurate.
The Arizona Automobile Dealers Association has commissioned weekly tracking polls over the past month, each of which had more than 1,000 respondents. Though some have questioned ties to Ducey by members of the association — the polls have shown Ducey’s lead growing considerably in the past several weeks — the association has not endorsed anyone in the race.
Republican political consultant Bert Coleman, who has conducted five polls of the governor’s race, is also unaffiliated with any of the gubernatorial campaigns. Coleman’s two most recent polls only surveyed voters who had already turned in their early ballots, and only in Maricopa County.
But both Coleman and the auto dealers, whose polls are conducted by Colorado-based Magellan Strategies, used auto-dial polling.
On Aug. 20, Remington Research Group, a polling firm based in Kansas City, Mo., released a poll showing Ducey with an 11-point lead over Smith. The poll of 502 voters was mostly auto-dial, though the firm used live interviewers for the 20 percent of respondents who were polled via cell phone, because pollsters are not allowed to call cell phones with automated polls.
Auto-dial polls are viewed as unreliable because it’s impossible to tell who is actually answering the questions. The respondent might be the voter targeted by the poll, or it might be a small child who is pushing buttons on a phone.
Kenski said one auto-dial pollster called her three times recently to ask the same questions, without ever confirming who was actually on the phone.
“It could be my seven-year-old grandson answering for the fun of it, and they wouldn’t know. He did that once,” she said.
The inability of automated polls to call cell phones casts further doubt on auto-dial polls. DeMenna said he distrusts any poll that isn’t about half cell phones, because so many people no longer have land lines.
Coleman defended auto-dial polls, saying many are fairly reliable. He said he takes some steps to increase the likelihood of getting reliable respondents, such as not calling households where political affiliations are split when polling primary races.
And Coleman said he doesn’t trust polls put out by groups with agendas. The only polls he trusts, he said, are the ones he conducts himself.
“The polls are released for one motivation, and that is to get people to think they’re winning. Polls that are done by third parties and independent parties, they’re just trying to get information out,” he said.
The polls that appeared to have the best methodology or came from the most credible pollsters were commissioned by groups with allegiances to one of the candidates.
The Arizona Free Enterprise Club, a free-market advocacy group, commissioned two polls, both of which were live-caller surveys conducted by the Tarrance Group, a Virginia-based firm. But the Free Enterprise Club has endorsed Ducey and spent $150,000 running attack ads against one of his top competitors, former Mesa Mayor Scott Smith.
Ducey led the group’s first poll, released on July 29. It showed Ducey ahead of former GoDaddy executive Christine Jones by 3 percentage points, while Smith trailed behind in third place with 15 percent of the vote. In the group’s Aug. 14 poll, Ducey extended his lead to 8 percentage points, while Smith had climbed into second place.
“They’ve endorsed Doug Ducey,” said Smith spokesman Drew Sexton. “They’ve spent hundreds of thousands of dark money dollars against Mayor Smith. So they’re going to do everything they can to show Doug in a better light.”
The only other live-caller poll released in the past two months was conducted by the lobbying and consulting firm HighGround, which is allied with Smith and had good news for his campaign. The Aug. 13 poll surveyed only Maricopa County voters who have already turned in their early ballots. The poll showed Smith trailing Ducey by 3.4 percentage points, within its margin of error.
HighGround’s poll, as well as two of Coleman’s polls, only covered people who have already turned in their ballots. But in that case, it’s at least easy to guarantee whether legitimate voters are being polled.
Some polls of the governor’s race have attracted criticism because of how they determine who is a “likely Republican voter” in the Aug. 26 primary.
Pollster Bruce Merrill, a professor emeritus at Arizona State University’s Walter Cronkite School of Journalism and Mass Communication, said people will lie when pollsters ask if they plan to vote in an upcoming election, much like they’ll lie about whether they go to church, beat their children or use pornography.
“If it’s a socially acceptable kind of a thing, then people are going to answer it in a socially acceptable way,” Merrill said.
That can skew results because it gives a false impression of who is voting, Merrill said, especially in low-turnout elections like primaries. Merrill and other experts say the most reliable way to choose respondents in a primary poll is only to call people with a documented history of voting in recent primary elections.
“The lower the turnout, and the people that have voted consistently … in the last three primaries are going to tend be much more conservative,” Merrill said.
A lack of ‘gold standard’ polling
If Arizona is lacking in high-quality, independent polls, it’s not alone.
Harry Enten, of the political polling and data website Fivethirtyeight.com, lamented the lack of “gold-standard polling” in campaigns across the country. To meet Enten’s gold standard, polls must meet several criteria, including two that are sorely lacking in the Arizona governor’s race: They must use live interviewers, and they must not come from campaigns or campaign groups that often release data only selectively.
In an article, Enten wrote that Gold standard polls not only provide better information, but often set the trend for other, “nontraditional” polls that tend to mimic more credible surveys. Nontraditional polls typically are more accurate in races that feature gold standard polls as well.
Enten surveyed 856 polls taken during the final three weeks of U.S. Senate races since 2006, and found that gold-standard polls had an error rate of 3.8 percentage points. In races with gold-standard polls, nontraditional polls had an error rate of 4.3 percentage points, compared to 6 percentage points without them.
“Gold-standard surveys appear to the LeBron Jameses of the polling world: They make everyone around them better,” Enten wrote.
They may still be right
Even if the polls are widely viewed as unreliable, some say useful information can be gleaned by following the trends. In the case of the 2014 GOP primary, that trend has consistently favored Ducey.
“As a general rule, you can get a sense of context of the race by looking at everything from a holistic perspective,” said Nathan Sproul, a Republican political consultant.
Some question the methodology of the polls released by the auto dealers and Coleman. But both have polled the race repeatedly using similar methodology. And the movement from poll to poll can be telling.
The first auto dealers poll, released on July 31, showed Ducey leading Smith by just 2 percentage points, within the margin of error. In subsequent weeks, Ducey’s lead jumped 8 and 9 percentage points.
Sproul said another good way to determine whether the polls are accurate is to simply watch the behavior of the campaigns.
In mid-July, when the polls showed Jones as Ducey’s main contender, Ducey and the independent expenditure groups supporting him trained all their fire on Jones, running multiple attack ads against her on television. But in recent weeks, Ducey and his allies have ignored Jones as she has sunk in the polls and shifted instead to attacking Smith, whose numbers have improved significantly.
“All you have to look at is how the Ducey IEs spent their money after Aug. 1,” Sproul said.
And while other campaigns have long questioned whether Ducey is really leading the pack, they have focused their attacks almost exclusively against him.