The Republican National Committee and networks want to limit debates to the top ten presidential candidates, based on averaging polling results from five different national surveys. In a prior column, I explained how this approach is unfair because debate performance drives polls, not the other way around. Also, early national surveys are especially volatile and hardly predictive of finalists. Finally, equally averaging the apples and oranges of surveys with different size (and limited) samples, and quite varied and even dubious methodologies. produces strange vegetables indeed. And in another column, I dissected one much publicized and deeply flawed specific national survey – the Wall Street Journal/NBC poll and showed how its results were unreliable, if not preposterous.
To participate in a debate, all candidates should at least (a) be announced and (b) agree to back the party’s nominee. But if the RNC will primarily use polling as the criterion, I concluded the best approach would be a first debate that consists of two equal status debates on consecutive nights, thus two news cycles. Night one would feature candidates 1,3, 5, 7, 9, 11, 13, and 15; and the second night would feature candidates 2, 4, 6, 8, 10, 12, 14, and 16. Besides the obvious fairness of offering all sixteen candidates equivalent exposure, this approach also means that Donald Trump could suck up the oxygen at only one debate.
Now comes a much-vaunted USA Today/Suffolk University Poll widely reported on networks that shows Trump “surging” to a first place lead of 17 percent, compared to a distant 2 percent in the last USA Today poll. Trump likely has increased his ballot strength, even dramatically, but this much? CNN has reported this story repeatedly, because its agenda is to feature Trump and divide the Republican Party.
The television pundits authoritatively and disingenuously say Trump’s 3-percent margin over Jeb Bush, at 14 percent, is “within sampling error.” But there’s much more to say about problems with this survey that is making national headlines.
This survey’s sampling distribution by region, race and age is based on the 2010 census, which is dated and also not even relevant, because it hardly is indicative of overall turnout, certainly not of Republican turnout. If you are not defining the universe correctly, the survey is suspect. A sample correctly drawn from the wrong universe obviously does not represent reality. But let’s pretend the universe is correct and look at the sample.
The sample is of 1000 “adults,” not voters, verified or not. At most, only 85 percent of those adults actually are registered to vote, and at most only 65 percent of those actual voters would vote in a primary. That means at most only 51 percent (.85 x. 65) of these one thousand adults, or any adult grouping within it, are relevant to the primary elections.
The survey of “adults” relies on self-identification of Republicans, not actual voter registration. Worse, it adds independent “adults” who merely claim they would vote in a Republican primary, even if in some states with a closed primary they would be ineligible to vote. Regardless, the total claimed is 349 adults who are supposed Republican primary voters.
Significantly, when we apply our conservative 51 percent assumption, that 349 becomes only 178 even possible Republican primary voters, with a margin of error of plus or minus about 7.5%. That’s sampling error, not errors arising from flaws. Remember, we don’t even know if the universe from which the sample is drawn is really appropriate.
Regardless, that “statistical” theoretical margin of error doesn’t tell the full story. The interviewers tried to minimize bias by rotating the order of the 16 candidates. Thus, for the 178 respondents, we have only about 11 per rotation pattern. That’s hardly enough to allow for unknown reaction to each sequence of candidates.
Finally, if all that were not enough, this survey was conducted from Thursday/ July 9 through Sunday, July 12. Friday, Saturday and Sunday generally are less reliable for polling and normally part of an extended study. And during these four specific days, Donald Trump dominated the news. For example, I heard him speak Saturday morning in Las Vegas at the Freedom Fest. He was entertaining and fun and well received. But in the hallways, the libertarians remained skeptical of his crony capitalism. But the media coverage of this event and his Arizona appearance was substantial.
In this USA Today survey, the ballot question includes only names, not senator or governor, so Trump – with high identification, was given a further edge over insufficiently identified elected officials. In reality, Trump may be gaining, possibly at the expense of some conservative candidates and thus ironically helping Jeb Bush. But if and when Trump collapses, those conservative candidates would benefit, but it’s still unclear whether one conservative would emerge against Jeb Bush.
Now if you think I’m too harsh on this particular survey, consider this: the USA Today interviewers asked whether the respondent had heard of several of the presidential candidates. But they asked these ID question after asking the ballot question that mentions the candidate names. Now, that’s oxymoronic.
Donald Trump has enough money to do a poll, and do it right. But why should he?
Arnold Steinberg, strategist and analyst, is the author of graduate texts on politics and media, and a court-recognized expert on quantitative research and media who has taught at Pepperdine’s Graduate School of Public Policy. He has conducted 2000 studies.
