Category Archives: The Great Poll Scam

Poll-arity

Posted on by
6

One outcome is certain tomorrow – the pollsters will finish last.

Give the pollsters of the 2012 cycle some credit – they’ve managed to straddle the fence, predicting a solid electoral victory for Barack Obama…and potentially a major popular vote win for Mitt Romney.

The top line of most of the recent polls has been easy enough to read.  The Real Clear Politics national average represents a statistical tie as Obama leads by 0.7% but the sheer numbers of polls showing slight edges to Obama in key states has the conventional wisdom pegging the President at somewhere around 290 to 303 electoral votes.  A step drop from 2008 but a large win by comparison to the recent histories of 2004 or 2000.

Yet the crosstabs of almost every pollster suggests a far different outcome as Mitt Romney holds a lead among unaffiliated/independent voters.  And the margins are anything but slight.  Romney leads independent voters by 7% with Fox News’ polling. By 9% with Rasmussen Reports.  12% according to two separate polls by NPR and the New York Times.  16% by Monmouth’s numbers.  And a jaw-dropping 24% by CNN.

The lead isn’t universal – Gallup has Obama up 1% among indies with Politico having a similar result…after deciding they would qualify more indies as Republicans following Romney’s 10% lead just two weeks earlier.  The trendline is obvious.  The question is how much does it matter to win independents?

Conventional wisdom in politics is like conventional wisdom about everything else – it’s right up until the point it’s wrong.  Whereas independent voters have been prized possessions in past elections, suddenly the value of these voters has been called into question:

It’s true that independents are a diverse group. But that’s mostly because the large majority of independents are independents in name only. Research by political scientists on the American electorate has consistently found that the large majority of self-identified independents are “closet partisans” who think and vote much like other partisans. Independent Democrats and independent Republicans have little in common. Moreover, independents with no party preference have a lower rate of turnout than those who lean toward a party and typically make up less than 10% of the electorate. Finally, independents don’t necessarily determine the outcomes of presidential elections; in fact, in all three closely contested presidential elections since 1972, the candidate backed by most independent voters lost.

Let’s look at that last statement in greater detail.

On the surface, it’s 100% correct.  Jerry Ford, John Kerry and George W. Bush all won the independent voter demographic and all three lost the popular vote (although not the election in all three cases).  Bush won indies by 2% and lost by 0.5% in an electorate that was 4% more Democrat than Republican.  Kerry won indies by 2% as well but lost by 3% in a tied partisan affiliation election.  And Ford, amidst a massive movement of Republicans to Independents post-Watergate, won that block by 4%…the largest margin for a losing candidate and done in an electorate with a 15% Democratic advantage.

The trendline here is simple as well – a narrow advantage among independent voters guarantees nothing other than perhaps a close election.  But compare Romney’s margin among indies to past performances.  Obama won indies by 7%.  Clinton won indies, despite an independent candidate on the ballot, by 8% in 1996 and 6% in 1992.  Bush Sr. won by 14% in 1988 and Reagan by 28% and 25% respectively in his two races.

Can Romney win independents and still lose the election?  Of course.  But only if a few other conditions arise.  The electorate has to be strongly Democrat.  Many pollsters are using D+8ish models ala 2008 even as 825,000 voters in eight key battleground states dropped their Democrat registration.  Or Romney could lose a key chunk of Republicans to offset his gains among indies.  That too seems unlikely as Democrats have held voter identification advantages every year since 1972 except in 2002 & 2004 – and the largest Republican advantage was 1% in ’02.

Some have argued that Romney’s lead among independents is simply a reflection of dissatisfied Republicans having left the party but whom will still vote conservatively.  It’s not a bad theory and it’s supported by some evidence.  Gallup has Republicans at 28% and Independents at 38%.  Pew has Republicans at 25% and Independents at 36%.  Yet neither Gallup or Pew reflect such a shift in their presidential polling.  Gallup has Obama up 1% among indies, as previously stated, and Pew has Romney up only 3%.  If Republicans just dropped the ‘R’ from their ID, someone forgot to tell them.

The end result isn’t actually about who wins on Tuesday.  Regardless of the  outcome, most of the pollsters have made a series of startling errors.  Either they’ve completely whiffed on properly defining party IDs within whatever likely voter model they’re using or they can’t accurately identify independent voters as a demographic.  Simply put, the numbers don’t match.  Obama can’t win if he loses the largest party ID block by high single or low double digits.  Conversely, Romney can’t lose if he wins independents by those kinds of margins.

The question in doubt tomorrow isn’t whether the pollsters erred but on which end of the spectrum.  We’ll find out for sure on Tuesday.  The pollsters will have to find out how they went wrong starting on Wednesday.

ADDENDUM:  Over at Mr. Dilettante’s, D pithily surmises the conundrum of the 2012 polls:

One thing will be decided this time — either polling is broken, or the time-honored tradition of reporting and observation is obsolete. It’s a fascinating question to resolve.

 

It’s Entirely Possible…

Posted on by
13

…that Barack Obama wins this election (more at noon).

But as the situation among reputable polls shifts ever more to Romney, and as more and more hitherto “likely Obama” states flip to “Leans Obama” and “Toss-up” and even “Lean Romney”, it’s interesting to watch Nate Silver doubling and tripling down on his prediction; he’s still giving The Light Worker a 75% chance of winning.

I’m not a statistician – but I can read and reason, and I’ve been dinging on Silver’s polling, methodology and predictions for a couple of years now.  My beef – and I’d suspect the beef of any rational person who isn’t one of the incurious low-information voters at which Silver’s polling is aimed – is that he calculates his results based on weighting existing polls based on some proprietary secret sauce known only to him.

Is the “sauce” valid?  I don’t know – nobody does, really – but as I showed in the 2010 Minnesota Governor’s race, it involved giving exaggerated weight to polls like the absurd “Minnesota Poll”, the so-bad-it’s-out-of-business Humphrey Institute poll and the frankly left-leaning PPP poll, while systematically shorting polls like Survey USA and Rasmussen.

Is Silver right?  Even if I could check his math, I probably couldn’t check his math, if you catch my drift.  Maybe Obama still is a near-sure thing, even after this past three weeks; maybe the Dems and Silver know something we don’t (like how many dead people will be voting).  We won’t really know until next week.

But while there will be many things about a Romney win that I’ll applaud, one of the big ones for me, personally, will be dancing – rhetorically, of course – on Nate Silver’s professional grave.

Shot In The Dark: Today’s News, Two Years Ago

Posted on by
5

Nate Silver at the NYTimes has been widely respected for his ability as a statistician.

His reputation, though, seems to stem largely from his facility at what amount to rhetorical parlor tricks (he once earned a bit of a living counting cards at poker, and he made a name for himself with baseball stats), and his calling of the vast majority of the 2008 election slate correctly (with the help of an epochal wave election and lots of access to Obama campaign internal polling), leading to his hiring at the NYTimes in time for the 2010 race.

Silver’s method at the NYTimes involves…:

  • Taking regional polls – from polling services as well as media polls – and…
  • “weighting” them according to some special sauce known only to Nate Silver, Registered Statistical Genius.

Now, I wrote about Silver’s method two years ago, when he spent much of the race predicting Mark Dayton would win by six points (with an eight-point margin of error).  As I pointed out, Silver’s “methodology” involved giving a fairly absurd amount of weight to  polls like the long-discredited Star Tribune “Minnesota” Poll and the since-discontinued Hubert H. Humphrey Institute poll (for whose demise I sincerely hope I deserve some credit, having spent a good part of the fall of 2010 showing what a piece of pro-DFL propaganda it has always been).  During the middle of th 2010 race, Silver gave the absurdly inaccurate-in-the-DFL’s-favor (especially in close elections) HHH and Minnesota Polls immense weight, while undervaluing the generally-accurate Rasmussen polls and, to a lesser extent, Survey USA.

I said Silver’s methodology was “garbage in, garbage out” – he uses bad data, and gets bad results.  I was being charitable, of course; his methodology, untransparent and proprietary as it is, processes bad data into worse conclusions.

That was in 2010.

Today?  NRO’s Josh Jordan reaches the same conclusion:

While many in the media (and Silver himself) openly mock the idea of Republicans’ “unskewing polls” (and I am not a fan of unskewedpolls.com by any means), Silver’s weighting method is just a more subtle way of doing just that. I outlined yesterday why Ohio is closer than the polls seem to indicate by looking at the full results of the polls as opposed to only the topline head-to-head numbers. Romney is up by well over eight points among independents in an average of current Ohio polls, the overall sample of those same polls is more Democratic than the 2008 electorate was, and Obama’s two best recent polls are among the oldest.

But look at some of the weights applied to the individual polls in Silver’s model. The most current Public Policy Polling survey, released Saturday, has Obama up only one point, 49–48. That poll is given a weighting under Silver’s model of .95201. The PPP poll taken last weekend had Obama up five, 51–46. This poll is a week older but has a weighting of 1.15569.

So it wasn’t just Minnesota!

And remember – PPP polls, while leaning a little left, are not generally flagrantly inaccurate in the sense that the Strib is and the HHH was.

And it’s not a fluke…:

The NBC/Marist Ohio poll conducted twelve days ago has a higher weighting attached to it (1.31395) than eight of the nine polls taken since. The poll from twelve days ago also, coincidentally enough, is Obama’s best recent poll in Ohio, because of a Democratic party-identification advantage of eleven points. By contrast, the Rasmussen poll from eight days later, which has a larger sample size, more recent field dates, but has an even party-identification split between Democrats and Republicans, has a weighting of .88826, lower than any other poll taken in the last nine days.

Jordan reaches a conclusion that even I didn’t:

This is the type of analysis that walks a very thin line between forecasting and cheerleading. When you weight a poll based on what you think of the pollster and the results and not based on what is actually inside the poll (party sampling, changes in favorability, job approval, etc), it can make for forecasts that mirror what you hope will happen rather than what’s most likely to happen.

Well, you can – if your goal isn’t so much to measure the nation’s zeitgeist (and report on it) but affect the election.

Which has, of course, been my contention all along.

Gallup-ing Towards The Finish

Posted on by
7

They don’t call it a horse-race for nothing.

As a rule in polling, outliers tend to get ignored.  Or you can choose to believe that Bush won Hawaii in 2004, Alf Landon won a 1936 landslide, or that Clinton v. Dole was a nail-biter.

But it becomes harder to ignore an outlier when it’s A) close to the election and B) one of the oldest and most respected polling outfits in the nation.  Thus as the media enters Campaign 2012′s home stretch, the narrative of a nip-and-tuck contest looks decidedly jeopardized by Gallup showing Mitt Romney with a 7% lead - and such an outcome apparently has to be challenged:

With a record of correctly predicting all but three of the 19 presidential races stretching back to 1936, Gallup is one of the most prestigious names in the business and its outlier status has other polling experts scratching their heads.

“They’re just so out of kilter at the moment,” said Simon Jackman, a Stanford University political science professor and author of a book on polling. “Either they’re doing something really wacky or the other 18 pollsters out there are colluding, or something.”

The caveats to Gallup’s polling (as with any pollster) are well-versed.  But to find an answer as to why Gallup posts a major Romney lead while the Real Clear Politics average of pollsters shows essentially a tie has nothing to do with credibility or collusion.  It has everything to do with turnout.

Take the recent IBD/TIPP poll as Gallup’s doppleganger with Obama leading by 5.7%.  Democrats are outsample Republicans by 7%.  The UConn Courant showing Obama up 3%?  The sample shows Democrats with an 8-point advantage.  Gallup plays their cards close to the vest, not showing the partisan affiliation of their likely voter model.  But their registered voter breakdown still shows a Romney lead, albeit of a modest 3% and is likely based on their party affiliation polls showing Democrats up 4 points.

Gallup says it determines its “likely voters” by asking whether they have voted in the past, if they know where their polling place is located, and other similar questions. The formula has been tweaked this year to take into account the increasing prevalence of early voting.

Gallup’s Newport pointed out that the firm’s likely-voter formula has more accurately predicted the election results than its wider poll of all registered voters going back to the 1990s and, in fact, the likely voter prediction tended to slightly favor Democratic candidates.

The idea of a single pollster being simply a part of a larger trendline is accurate, even if most media outlets tend to overlook that fact to trumpet their own poll to the exclusion of competitors and thus create news rather than report it.  Yet even if we exclude Gallup’s results, the trendlines have to be concerning for Obama’s camp.  Despite wielding turnout margins better than what propelled him into office four years ago, many polls show Barack Obama at best narrowly ahead – and more commonly tied or behind.

Gallup might be overstating Romney’s support, although the pollster’s worst estimations of support were in the 5-6 point range and happened in 1936 and 1948.  In the modern era, if anything Gallup has consistently overestimated Democratic support at the polls, giving Obama 2% more, Kerry 0.7% more and Clinton 2.8% and 5.7% more in his campaigns.  Which may mean that despite a 7% lead causing headaches among the media, Mitt Romney may…hold for dramatic effect…lead by more.

Chanting Points Memo: And We’re Back To The Fine Print

Posted on by
3

The left and media (PTR) was skipping and gamboling about like happy little meerkats yesterday; a new PPP poll showed – as PPP polls tend to do - nothing but good news for Minnesota Democrats.

In an automated phone survey of 937 likely MInnesota voters, they found…:

PPP’s newest poll on Minnesota’s amendment to ban gay marriage finds it running slightly behind, with 46% of voters planning to support it and 49% opposed.

That represents a 4 point shift compared to a month ago when it led for passage 48-47.

The poll claims that the major movement has been among indies and women.

“The marriage amendment in Minnesota continues to look like a toss up,” said Dean Debnam, President of Public Policy Polling. “Voters are very closely divided on the

issue.”

Well, we more or less knew that.

Then, they addressed the other Constitutional Amendment:

When we polled on it in June it was leading for passage by a 58/34 margin. By September that had tightened to a 56/39 advantage. And now it’s leading only 51/43. Democrats are now even more opposed to the voter ID amendment (23/71) than they are to the one on

marriage. And although independents continue to support it their 52/41 favor for it is down a good deal from 62/33 a month ago. This fight may end up a lot closer than people

initially expected.

Or it may not.

We’ll come back to that one.

They also put the DFL up substantially on a “generic legislative ballot”, which would be big news if voters voted for a generic legislature.  They don’t, of course.

As always, the devil is in the turnout model:

Here it is, buried deep in a set of crosstabs:

That’s D+9.  Not as far out as the D+13 we got from the Strib a while back, but it still higher than 2008.

That’s especially interesting compared to this other bit of crosstabbery:

So Democrats outnumber Republicans 38/29, but conservatives outnumber Dems 37/34?

At any rate – the polling services continue to put out (if you look hard enough for them) polls with turnout models that, when you ask them, they are are legitimately what they’re encountering out there…

…but do not in any way pass the sniff test.

And the media?

Well, they just shovel it on out there.  It’s just the topline number that really matters.  Right?

Chanting Points Memo: Puff

Posted on by
Reply

Who’s a better actor; Scarlett Johannson or Donald Rumsfeld?

Now, if you’re a sophisticated polling operation like the Star/Tribune “Minnesota Poll” – a Mason-Dixon joint – it’s an easy question; Ms. Johannson will likely outpoll the former Secretary of Defense.

Of course, you may respond “but measuring a Secretary of Defense in terms of acting skill is meaningless!”

And if you say that, then you’re already too smart to buy the latest Minnesota poll.

———-

The final Star/Tribune Minnesota poll of this long, nauseating week was released this morning – and it has two conclusions:

After two years of budget battles, vetoes and the longest state shutdown in Minnesota history, DFL Gov. Mark Dayton is winning the popularity battle with the GOP-controlled Legislature, a Star Tribune Minnesota Poll has found.

A slim majority of 53 percent of likely voters say they approve of Dayton’s job performance, while 31 percent disapprove. Another 16 percent say they are undecided.

For the majority leaders of the Legislature, the poll found 51 percent disapprove of the job they are doing. Another 21 percent approve and 24 percent are undecided.

This poll really needs two responses.

Damnation With Faint Praise

The poll notes that Governor Dayton scored a 53 percent approval rating.

Now, on its face, that’s not a good number.  Under 50%, says conventional wisdom, is trouble – and Dayton is a governor that’s done virtually nothing in two years but make odd, slurred pronouncements before scuttling away into his office under the cover of a fawning media.

But as always, you have to look below “the face”.  Of course, the poll has the same absurd, worse-than-Watergate-level turnout model – Democrats 41, Republicans 28 – as all the other polls this week.

However, for some reason the Strib doesn’t favor us with the full range of crosstabs; while reporter Jim Ragsdale notes the Governor’s geographic strengths and weaknesses, at no point in the article is the approval of Republicans or Democrats explained.

Is this an accident?  Or would showing it reveal something about the poll the Strib doesn’t want us to see?

Because if we assume Democrats are over-polled by 3%, and that they were the vast majority of the “approve” numbers, then Dayton’s approval drops down to right around 50% – and the “disapprove” numbers jump into the high thirties if we assume most Republicans disapprove of Dayton’s job.

Of course, it’s all guesswork until the Strib releases those crosstabs.

Damnation By Packing Peanuts

Of course, the numbers on the legislature are just plain nonsense

For the majority leaders of the Legislature, certain trouble spots stand out: Only 24 percent of voters in the metro suburbs outside of Hennepin and Ramsey — which include strong GOP areas — approve of their leadership, while more than half disapprove. In southwestern Minnesota, 57 percent disapprove. They had slightly stronger showings in Rochester, where 30 percent approve, and in the northwest corner of the state at 33 percent.

This is just a stupid thing to measure.

Grading a legislature, as an entire body, all together, is like asking what a football fan thinks of the NFC Central Division.  You will get a dog’s breakfast of opinions, or no opinion – because the division is not the focus (except for bracketing playoffs).

Put another way?  Nobody is going to vote for “The Legislature” this fall.   They will vote for or against candidates.  I will be voting for Rick Karschnia for State Senate and Dan Lipp for House; not “for the legislature” or even “for the House GOP caucus”.  And I”m a wonk! 
Indeed, this next paragraph sums up the absurdity of the whole question:

One startling figure is that the GOP-controlled Legislature only broke even among Republican voters: 31 percent approve, 32 percent disapprove and 37 percent are undecided. Undecided numbers are higher throughout the Legislature’s poll, suggesting many voters do not have a clear opinion on the topic.

Well, dug.

I disapproved of the Legislature’s job!   The GOP caucuses gave away too much money in 2011 and caved on the stadium last year without getting anything useful in return!  I give them a “D”.

Does that mean I’m going to support Mark Dayton?  Not if you held a gun to my head (not to give the SEIU any ideas).

It’s a meaningless number.

So Why Run A Poll With A Meaningless Number?

That one’s easy; it gives the Strib a nice tidy number – 53-21 approval ratings! – to toss in front of people who don’t pay much attention to what the numbers actually mean.  These voters – the “Low-Information Voters”, people who retain headlines from the Mainstream Media and believe things that appear in the Strib – are the target for this sort of polling, or as it’s known in the world of logic, a “non-sequitur”.  The Strib is comparing apples and axles, just like comparing Donald Rumsfeld with Scarlett Johannsen.

While the individual numbers might be valid (they’re not – remember, the turnout model is absurd), at least in terms of math used to generate the numbers we see – but even if both numbers were in fact dead-nut accurate, what the Strib has done is created a phony horserace.

And why would the Strib splash a phony, non-sequitur comparison on its front page when it only serves to show Mark Dayton with a decisive (if phony) lead…

…oh, wait.

Never mind.

More Monday.

Chanting Points Memo: The Rigger’s Dilemma

Posted on by
4

It’s my contention that the Star/Tribune “Minnesota” poll is, and has been for two and a half decades, less a “public opinion” poll and more an instrument of DFL propaganda.  I’ve supported that contention with a raft of circumstantial evidence; proof that the Minnesota Poll underestimates GOP turnout – especially in races that are perceived to be close; it showed Mark Dayton with an absurdly huge lead over Tom Emmer, and Al Franken with a four point lead over Norm Coleman, while guessing the Klobuchar/Kennedy race fairly accurately.

It’s my contention that this is to leverage the “Bandwagon Effect” – to discourage Republicans and conservatives from going to the poll.

But this year’s race presents a dilemma for the editors who – I’m being half-hyperbolic here [1] – plan the results of these polls.  On the one hand, you have the Voter ID initiative which is likely to win in a blowout.  To skew that poll enough to encourage Democrats and opponents of the amendment, the Strib would need to skew the poll to an absurd extent.  As in, assume conditions that are the same as in 1976, after Watergate.

On the other hand, you have a Senate race between Amy Klobuchar and Kurt BIlls that is widely perceived to be a pretty safe race for the incumbent.   Skewing the sample too far to the left would make the results look completely implausible.

The answer, if you’re the Strib?  You see it in this weeks’ Minnesota Polls; this is their old buddy Jim Klobuchar’s daughter we’re talking about here!  Of course they’ll do what it takes to make her re-election as epic as possible – why, everyone on Editor’s Row remembers Amy when she was just this tall, dagnabbit!

Beyond that – and more germane to the propaganda organ – they know that the voters the DFL needs are the “low-information” voters. The ones that rarely get past the headline, much less the lead – forget about looking at partisan breakdowns.  The ones that still believe the Strib is anything but DFL shills, or don’t care either way.

The Strib is showing a 57-28 lead for Klobuchar, with 15 percent either undecided or voting for someone else.  As we’ve been showing every day this week, this is based on a sample that includes 41% Democrat/28% GOP turnout.

Now, if we assume it’s more like 38/34 – which is more in line with Rasmussen’s figures, which have been traditionally vastly more accurate – and multiply the changes by the support each candidate gets within their party…

…well, that’s bad for Bills, if you believe the Strib.  While 90-odd percent of Dems say they’ll vote for A-Klo, the Strib claims only 2/3 of Republicans will vote for BIlls.  While the nomination battle was a bruising one in the GOP, and left a lot of bad blood, Republicans are much more suck-it-up-and-support-our-guy than that.  This strikes me as dubious – the “stink test” is crying “BS!” – but I’ve got no hard evidence to the contrary just yet.  Absent that, let’s run with the Strib’s numbers.

So if we subtract 3% from the Democrat split, multiplied by 90%, we get a net loss of 2.7% for Klobuchar, taking her down to 54.3%.

Adding 6 to Bills – times the 66% support in the party, naturally – leads to a four point rise, to 32.

54-32 still isn’t close.  But it’s not the 2:1 humiliation…

…that, I contend, the Strib wants Republicans to believe is coming, on top of all the other “bad news” they’ve brought us this week.

So what does all this mean?

More tomorrow.

Continue reading →

Chanting Points Memo: Tie Manufacturing Is Way Up!

Posted on by
Reply

The Star Tribune ”Minnesota Poll” has declared Minnesota tied on the Obamacare issue:

About 46 percent of the state’s likely voters say they support keeping the Affordable Care Act, whose main tenets were largely upheld by the U.S. Supreme Court this summer, while 47 percent believe the law should go.

As always when dealing with any polls – especially polls with records of dishonesty as long as the Strib’s -http://www.startribune.com/local/171271561.html.  The Strib’s, in case you’ve forgotten,  is 41% Democrat, 28% Republican – which, as I showed yesterday, is more Democrat-leaning than the 1976 post-Watergate, post-Nixon-pardon election, the post-war nadir of GOP fortunes.

As a bipartisan sampling – liberal Hamline poli-sci professor Dave Schultz and I – both agree,  the sample is more like 38% DFL, 34% GOP.   Since 80% of DFLers (according to the Minnesota Poll) support the amendment, that means you deduct 80% of 3 points – 2.4 – from the “support” column.  Likewise, 90% of the six additional points of Republicans – 5.4% – support repeal.

With those assumptions – a more realistic turnout model and those levels of support – the “repeal” case is really more like 52-44.

It looks like the MInnesota Poll is building up to the Senate Race.

Chanting Points Memo: That D+13 Split

Posted on by
7

As I wait for the latest “Minnesota Poll” to release its results for the Senate race, I’ve been turning the poll’s D+13 (their sample of respondents was 41% Democrat and 28% Republican) number around in my head.

After all, as the Strib tells us, “Brad Coker, managing director of Mason-Dixon Polling & Research, which conducted the poll for the Star Tribune, said those numbers are consistent with what he has seen over the years”.

But as we noted yesterday, the 2008 election – an epic Democrat win – was D+6 or so.  The 2010 election had turnout of D+2, roughly, and turned out to be a GOP rout nationwide and in the MN Legislature.

So what about the worst election in the past 50 years for the GOP – the post-Watergate presidential election of 1976?    Where the GOP got shredded in DC and in Saint Paul, sending the MNGOP running to their “Independent Republican” label?

I can’t find the partisan split – but does it seem unreasonable that in a year when Jimmy Carter beat Gerald Fold by 12 points in Minnesota that the partisan split was, maybe in the neighborhood of D+12?

In other words, maybe somewhere around the D+13 number the Strib would have you believe today?

The Great Poll Scam: A Blast From The Past

Posted on by
Reply

As we look at the abusive travesty that is the Minnesota Poll – in this case, the ludicrously skewed, 3:2 pro-DFL partisan breakdown in this weekend’s polling on the two Constitutional Amendments – let’s take a trip back through history.

Frank Newport, the president of the broadly-respected Gallup Polls, savaged the Minnesota Poll in the immediate aftermath of the 2010 midterm and gubernatorial elections.

Let’s make sure we’re clear on this.  Pollsters attacking pollsters in public is a little like magicians publicly heckling other magicians.

Is there any evidence the Strib has polished up their methodology?

No.  Indeed, the two polls released over the weekend on the Marriage and Voter ID amendments show quite the opposite – or that some copy editor took a poll of Ramsey County voters and mislabeled it a poll of Minnesotans.

Chanting Points Memo: Camouflaging The Battleground

Posted on by
17

The Strib “Minnesota Poll” is doing what it’s paid to do:  create a pro-DFL bandwagon effect, and suppress GOP voter turnout.  It’s calling Minnesota at Obama with 48% and Romney with 40%.

But the poll uses the same absurd D41/R28 breakdown that the Marriage and Voter ID polls.  This polling would have you believe that while in 2008, with a messianic media darling running against an unpopular two-term candidate (McCain was irrevant) and the war the DFL had a six point advantage in partisan turnout (D39 R33), this year, mirabile dictu, we have a 13 point Democrat advantage in this state?

If you use turnout numbers from somewhere in between 2008 and 2010 – say, D36 R34 – and multiply the changes by the percent of each party that the poll itself says plan on voting for their candidate (93% of Democrats plan to vote for Obama, vs 96% of Republicans), then you wind up lopping off roughly .3% of Obama’s numbers, and adding a whopping 5.8% to Romney’s.

That makes the real split 47.7% Obama, 45.8% Romney.  

Question – especially for you libs in the audience:  In what way is a widely (one might say “lavishly”) publicized poll using a partisan split that this state hasn’t seen since Watergate to be interepreted as anything other than an elaborate voter-suppression scam?

Chanting Points Memo: “Minnesota Poll” Orders Material For A Narrative-Building Spree

Posted on by
8

If you take the history of the Minnesota Poll as any indication, yesterday’s numbers on the Marriage Amendment might be encouraging for amendment supporters:

The increasingly costly and bitter fight over a constitutional amendment to ban same-sex marriage is a statistical dead heat, according to a new Star Tribune Minnesota Poll.

Six weeks before Election Day, slightly more Minnesotans favor the amendment than oppose it, but that support also falls just short of the 50 percent needed to pass the measure.

Wow.  That sounds close!

But as always with these polls, you have to check the fine print.  And the “Minnesota Poll” buries its fine print in a link well down the page; you don’t ever actually find it in the story itself.  And it contains the partisan breakdown (with emphasis added):

The self-identified party affiliation of the random sample is: 41 percent Democrat, 28 percent Republican and 31 percent independent or other.

That’s right – to get this virtual tie, the Strib, in a state that just went through photo-finish elections for Governor and Senator, and has been on the razor’s edge of absolute equality between parties for most of a decade, sampled three Democrats for every two Republicans to get to a tie.

If you believe – as I do – that the “Minnesota Poll” is first and foremost a DFL propaganda tool, intended largely to create a ‘bandwagon effect” to suppress conservative turnout (and we’ll come back to that), then this is good news; the Marriage Amendment is likely doing better  than the poll is showing.

What it does mean, though, is that they are working to build a narrative; that the battle over gay marriage is much more closely-fought than it is.

And the narrative’s players are already on board with this poll.  The Strib duly interviews Richard Carlbom, the former Dayton staffer who is leading the anti-Amendment

Actually, here’s my bet; the November 4 paper will show a “surge of support” that turns out to be much larger than any that actually materializes at the polls.

More At Noon.

UPDATE:  I wrote this piece on Sunday.  Monday morning, all of the local newscasts duly led with “both ballot initiatives are tied!”.

If you’re trying to find a construction job in Minnesota, you can get a job putting siding on the DFL’s narrative.

UPDATE 2:  Professor David Schultz at Hamline University – no friend of conservatism, he – did something I more or less planned to do on Wednesday; re-ran the numbers with a more realistic partisan breakdown:

Why is the partisan adjustment important? The poll suggests significant partisan polarization for both amendments, with 73% of DFLers opposing the marriage amendment and 71% of GOPers supporting. Similar partisan cleavages also exist with the Elections Amendment. If this is true, take the marriage Amendment support at 49% and opposition at 47%. If DFLers are overpolled by 3% and GOP underpolled by 6%, and if about 3/4 of each party votes in a partisan way, I would subtract about 2.25% from opposition (3% x .75) and add 4.5% to support (6% x .75) and the new numbers are 53.5% in support and 44.75% against. This is beyond margin or error.

If one applies the correction to the Elections Amendment there is about an 80% DFL opposition to it and a similar 80% GOP support for it. Then the polls suggest approximately 56.8% support it and 41.6% oppose.

Which brings us very nearly back to the 3:2 margin  for the Voter ID amendment, and the tight but solid lead for the Marriage Amendment that every other poll – the reputable ones, anyway – have found.

The Bandwagoneers

Posted on by
11

Have you noticed something?

No “Minnesota Poll” yet this cycle.  Ditto the Humprey Institute.

Usually by this point in an election cycle, they’ve run a poll showing the Republican candidate down by some absurd amount that turns out to be many times greater than the eventual margin of victory (or defeat) for the DFLer.

Now, I’ve been writing about the HHH and Strib “Minnesota” polls for quite some time.  I noted that since 1988, the Strib Minnesota Poll has consistently shorted Republicans by a consistently greater margin than Democrats in their pre-election polls – and that the discrepancy is even greater in elections that end up being closest.  I noted that the HHH poll is even worse – but that in polls where the DFLer appears to be in no danger, their polls end up being more accurate.

It is my contention that the Strib and the Humphrey Institute are allied – at least at the executive level – with the DFL, and use their polls to further the DFL’s ends; everyone involved is certainly aware of the “Bandwagon Effect” – the phenomenon by which voters who believe their candidates have no chance of victory will stay home.

So we’ve seen no “Minnesota” poll so far this cycle; Amy Klobuchar – perhaps the greatest beneficiary of media bias in the history of Minnesota politics, as the daughter of a former Strib columnist – seems to be in no great danger, so the polls say, from Kurt Bills (not to say I won’t do everything I can, personally, to fix that).  I’ll bet dimes to dollars the Strib polls wind up pretty darn close to the election totals, in fact!

———-

But the “Bandwagon” effect is going nationwide; Minnesota in 2008 and 2010 showed that it can keep juuuuuuuuust enough people home, if it’s relentless enough, to tip a close election.

And so you see the mainstream media already declaring the election over, based entirely on polling that is entirely based on the Democrats getting turnout they didn’t even get in 2008.

It is, in fact, the flip side of the “Low Information Voter” strategy they’ve run on their own side – convincing the ill-informed, the querulous and the not-bright that there’s a “war on women” and Obama “stands with the 99%” and “the economy was Bush’s fault but it’s almost back, any day now”; trying to convince people, especially independents, who might be sick to death of Obama and possibly thinking of voting GOP that it’s all hopeless and they should stay home.

Think about it.  Why else would they run polls that are transparently false?  That rely on assumptions that probably didn’t even occur during the post-Watergate election in 1976, much less 2008, much less today?

Because only the high-information voters either dig into the partisan breakdowns (or read the bloggers who do), and the record in Minnesota shows there are just enough incurious, too-busy, ill-informed, and just plain un-bright people to sway the matter if it’s close enough.

The media at all levels – bald-faced cheerleaders like the LATimes and the Strib and the supposedly-ethical ones like NPR alike – are going to be beating the “it’s over” drum constantly ’til the election.

The well-informed people know it’s baked wind.

But it’s not aimed at them.

All That’s Silver Does Not Glitter

Posted on by
3

While the national polls show the presidential race a statistical toss-up, Nate Silver points out that polls conducted in swing state show Obama with an actual lead of sorts – around three points:.

While that isn’t an enormous difference in an absolute sense, it is a consequential one. A one- or two-point lead for Mr. Obama, as in the national polls, would make him barely better than a tossup to win re-election. A three- or four-point lead, as in the state polls, is obviously no guarantee given the troubles in the economy, but it is a more substantive advantage.

Here’s the part that caught my attention; I’ve added emphasis:

The difference isn’t an artifact of RealClearPolitics’s methodology. The FiveThirtyEight method, which applies a more complicated technique for estimating polling averages, sees broadly the same split between state and national polls.

On the one hand – well, doy.  Obama’s an incumbent elected in a wave, protected by a media that serves as his Praetorian Guard.  Of course he’s going to be polling well.

On the other hand?  My real point in this article is the abovementioned “FiveThirtyEigtht Method”.

I addressed this two years ago – when Silver, who is generally acknowledged to be a moderate Democrat, spent most of the 2010 campaign predicting a 6+ point Mark Dayton victory.

How did he arrive at that number?

  1. By taking an assortment of polls from around MInnesota, conducted by a variety of polling operations, and…
  2. Applying a weighting to each poll, the “538 Poll Weight”, which came from an unexplained formula known, near as I can tell, only to Silver.  Which is not to say that it’s wrong, or statistically, intellectually or journalistically dishonest, per se - merely that it’s completely opaque

But let’s take Silver’s methodology at face value – because he’s a respected statistician who works for the NYTimes, right?

The fact remains that, at least here in Minnesota, two of the polls that were given great weight in Silver’s methodology – the Star Tribune “Minnesota” poll and the Hubert H. Humphrey Institute poll, are palpably garbage, and should be viewed as DFL propaganda at best, calculated election fraud at worst. 

We went through this in some detail after the 2010 election: there’s an entire category on this blog devoted to going over the various crimes and misdemeanors of Twin Cities media pollsters.  ,Long story short – since 1988, the Strib “Minnesota” poll has consistently shorted Republican support in polls, especially the polls closest to the elections, especially in close elections.  The “Minnesota” poll’s only redeeming point?  The Humphrey Institute poll is worse.  In both cases, they tended – moreso in closer races – to exaggerate the lead the Democrat candidate for Governor, Senator or President had.   For example, in 2010 both polls showed Mark Dayton with crushing, overwhelming, humiliating leads over Tom Emmer on election-eve.  It ended up the closest gubernatorial race in Minnesota history.  The “Minnesota” poll was so bad, Frank Newport of Gallup actually wrote to comment on its dubious methodology. I suspect that the results are less mathematical background noise or methodological quicks – which would, if truly random, show distortions that would even out between the parties over time.  While it’s not provable without a whistle-blower from inside either or both organizations, I suspect the results shake out the way they do, if you are inclined to believe people have integrity, due to selection bias in setting up survey samples (and, if you don’t have much faith, in systematic bias working to achieve a “Bandwagon Effect” among the electorate.  Count me among the cynics; an organization with integrity would have noticed these errors long before a guy like me who maxed out at Algebra I in college and fixed the problem.  I’m willing to be persuaded, but you’ll have to have a much better argument than most of the polls’ defenders). 
The point being, this is the quality of the raw material that leads Nate Silver to his conclusions.  
And that should give Silver, and people who pay attention to him, pause.
I don’t know if the other state polls are as dodgy as Minnesota’s local media polling operations.  That’d be a great subject for a blogswarm.  

Where Used Car Salespeople Fear To Tread

Posted on by
5

Say what you will about the Minnesota Poll and the Hubert H. Humphrey poll.  As bad, inaccurate, DFL-biased and seemingly-rigged as both are, they both actually release their cross tabs – such as they are.

So far.

With the WaPo’s new practice of sitting on the data for their polls - which, naturally, show that Barack Obama has bounced back – I don’t expect that to last for long.

Ed Morrissey wrote about the new practice:

More importantly, though, the poll series has dropped its reporting of partisan identification within their samples.  It’s the second time that the poll has not included the D/R/I split in its sample report, and now it looks as though this will be policy from this point forward.  Since this is a poll series that has handed double-digit partisan advantages to Democrats in the past (for instance, this poll from April 2011 where the sample only had 22% Republicans), it’s not enough to just hear “trust us” on sample integrity from the Washington Post or ABC.

One cannot determine whether Obama’s improvement in this series is a result of the State of the Union speech, as Dan Balz and Jon Cohen suggest, or whether it’s due to shifting the sample to favor Democrats more so than in previous samples.  The same is true for the Post’s report that Obama “for the first time has a clear edge” over Romney head-to-head.  One would need a poll of registered or likely voters to actually make that claim (one has to register to cast a vote, after all), and one would need to see the difference in partisan splits between this and other surveys in the series to determine whether the movement actually exists or got manufactured by the pollster.

Expect the effort to get Obama re-coronated to result in the extinction of whatever passes for “Journalistic Standards” in the polling industry.

He’s Baaaaaack

Posted on by
6

The lefties were all atwitter yesterday over a poll in the MinnPost that purported to show that Minnesotans blame the Minnesota GOP for the shutdown:

By a whopping 2-1 margin, Minnesotans blame the Republicans who control both houses of the Legislature for the recent government shutdown more than they blame Gov. Mark Dayton, according to a poll taken this week for MinnPost.

 

Predictably, most Republicans blamed Dayton more (by 56 to 10 percent, with the rest saying both sides were to blame or holding no opinion). DFLers blamed the Republicans by an even more overwhelming majority (68 percent to just 2 percent of DFLers who blamed DFLer Dayton).

 

But the key swing group of self-identified independents was also much more likely to blame Republicans than to blame Dayton. Among independents, 46 percent “blamed” the Republicans, 18 percent blamed Dayton and 25 percent both.

Hm. That sounds bad!

It also sounded familiar – indeed, it sounded right in line with a prediction I made in this space mere weeks ago.  Go ahead and read it; Prediction 1 was a month late, and it appeared in the MinnPost rather than the Strib; the piece is written by Erik Black and Doug Grow, former Strib staffers, so the feeling of deja vu was so overwhelming…

…that when I first read this post, I practically predicted the bit that is emphasized in the quote below:

Based on other questions in the poll, it was difficult to say whether the fallout from the shutdown will give DFLers a significant advantage heading into the 2012 elections, as Republicans seek to retain their majorities. Projecting current attitudes onto an election 16 months in the future would be folly.

 

Also, this poll, conducted for MinnPost by Daves & Associates Research, was designed to take the pulse of the state in the aftermath of the shutdown, not to predict the next election. No likely voter screen was used and sample surely includes non-voters.

And there you have it.  The MinnPost gets its polling from “Daves and Associates”.  That’d be Rob Daves – the guy who ran the Minnesota Poll for 21 years – the poll whose election-eve polls on Gubernatorial, Senate and Presidential races *always* showed the GOP doing worse – usually much worse – than it ended up doing.

And if it’s a post on politics in Minnesota by Strib alums Black and Grow, who else just has to show up?

Humphrey Center Political Scientist Larry Jacobs said the results of the new poll were “basically bad news for the Republicans.”

 

“They have to think about this fact,” said Jacobs.”The principles that they ran on in 2010 — that they would advocate for cuts only and would refuse to go along with any tax increase — may still be the principles that appeal to the most enthusiastic base of support they have. But that position seems to be pretty unpopular not only with two-thirds of Minnesotans, but with half of their own party, all of whom prefer a mix of significant spending cuts and at least some tax increases.”

Yep, Dr. Jacobs, whose Hubert H. Humphrey Institute Poll is even worse, and whose methodology was openly and publicly savaged by Frank Newman of Gallup last year after the Humphrey Institute polls were not only grossly wrong (predicting a 12 point Dayton blowout in the gubernatorial race which ended up about a .4% race) but were shown to have systematically oversampled strongly DFL areas of the state.

Both Daves’ and Jacobs’ polls, as I showed last year, shared an interesting trait: if the final result of an election ended up being really close, like the ’08 Senate and ’10 Governor’s race (as opposed to blowouts, like the ’06 Senate race), the Minnesota and HHH Polls *both* shorted Republicans *even more*:

The reason? Well, it’s a known fact that voters are prone to the “Bandwagon Effect”; they do tend to go along with what polls tell them, positively or negatively.  My theory – while it’s conceivable that the Strib, Rob Daves, the Minnpost, the HHH Institute and Larry Jacobs are unaware of the “bandwagon effect”,  I’d be a lot more convinced if Daves didn’t have a 24 year record of shorting the GOP on controversial, loaded polls when the chips were down (and Jacobs’ polls even worse for seven years).

The poll canvassed less than 600 random adults – not registered, much less likely, voters – and, as usual, it heavily-sampled identified DFLers and unspecified “independents”.

Fearless Predictions

Posted on by
22

I have a couple of predictions for you.

Prediction 1:  Polled To Death - Take this to the bank:  sometime before July 1, the Strib will run another “Minnesota Poll” in re the shutdown.

The poll’s headlines will be within one rhetorical standard deviation of  ”65% of Minnesotans Favor Compromise On Budget Impasse”.

The crosstabs, carefully buried, will show that DFLers are oversampled by 50%; those trying to investigate the faint whiff of metrocentrism in the polling will be frustrated by the absolute lack of crosstabs showing geography.

Prediction 2: Dead Silence – Despite the avalanche of evidence coming out of the state bureaucracy that Dayton is not only pushing for the shutdown, but actively trying to make it “hurt” as much as possible, there will be not one word on the subject from the Strib, WCCO, the PiPress, the KARE Bears (whose John Cronan is rapidly shaping up to be an Esme Murphy-grade stealth-DFL  propagandist), or MPR.

Place your bets.

Or make your own predictions, in the comment section.

Strib Poll: Empowering The Powerful, Gulling The Gullible

Posted on by
18

The poll was as drearily predictable as the annual stadium extortion-fest; notwithstanding last November’s electoral GOP legislative sweep, yet another Star/Tribune “Minnesota Poll” shows that the public is, mirabile dictu, entirely on board with the DFL agenda:

Sixty-three percent of respondents said they favor a blend of higher taxes and service reductions to tackle the state’s $5 billion projected deficit. Just 27 percent said they want state leaders to balance the budget solely through cuts.

The poll comes [with utter predictability - Ed.] as the Republican-led Legislature and the DFL governor head into the final week of a legislative session still dug in on their vastly different approaches to balancing the budget.

Dayton said the results show the public backs his position. Republicans said the results run counter to last fall’s election and what they are hearing from Minnesotans.

Predictable?  Absolutely.  Whether through editorial perfidy or lazy methodology, the Strib/”Minnesota” Poll has a long history of releasing “news” the DFL needs, exactly when it needs it.  Especially when the issue is especially close-fought; the harder-fought the issue, the more absurdly lopsided the  Strib poll, like the “Humphrey Institute” Poll run for many years by the U of M and MPR polls, seem to be.  Right when the DFL needs it.

My theory; the DFL knows full well how the “bandwagon effect” in polling works for manipulating public perception; the Strib serves the DFL, wittingly or not.

And, sure enough, the poll’s methodology was as predictable as the Strib’s smug headline; emphasis is added by me:

Today’s Star Tribune Minnesota Poll findings are based on 565 landline and 241 cellphone interviews conducted May 2-5 with a representative sample of Minnesota adults. Interviews were conducted under the direction of Princeton Survey Research Associates International.

Results of a poll based on 806 interviews will vary by no more than 4.7 percentage points, plus or minus, from the overall population 95 times out of 100.

The self-identified party affiliation of the random sample is 33 percent Democrat, 23 percent Republican and 37 percent independent. The remaining 7 percent said they were members of another party, no party or declined to answer.

Results for the question about the best approach to solving the budget deficit — primarily through service reductions or through a combination of tax hikes and spending cuts — are based on interviews with 548 of the 806 respondents. The question was reasked in follow-up calls to all respondents because of a problem in the original wording of the question, and 548 of the respondents were reached. Results of a poll based on 548 interviews will vary by no more than 5.7 percentage points, plus or minus, from the overall population 95 times out of 100.

In other words, a group which self-reports its political leaning, whose geographical weighting and mix are unknown (remember the Humphrey Institute’s overweighting of Minneapolis respondents? Which they didn’t bother to report until after the election, even though their actual poll, which indicated a 12 point blowout for Mark Dayton, went out on schedule, right before the election?), and where the “independents” are given no known context, and which gives the DFL a completely unearned 50% head start, shows the public solidly behind Mark Dayton.

Just like it needed to.

I doubt the Twin Cities media will ever admit that the “Minnesota Poll” and the “Humphrey Institute” polls are, intentionally or not, pro-DFL propaganda. But it’s gotten to the point where the evidence doesn’t support any other conclusion.

Trump, The Media, and Bandwagons

Posted on by
28

For background, I’ll refer you to…:

The Huckabee Corollary the McCain Corolloary To Berg’s Eleventh Law: The Republican that the media covers most intensively before the nomination for any office will be the one that the liberals know they have the best chance of beating after the nomination, and/or will most cripple the GOP if nominated.

If you’re like me, you looked at the polls “showing” Donald Trump “leading” the GOP field and thought “Huckabee Corollary!”.

Nate Silver – fresh from playing a role in engineering the DFL’s “Bandwagon Effect” in the Minnesota gubernatorial election last year – notices the media blitz on Trump without, I suspect, getting the “Why“:

One of the few pieces of statistical evidence that we can look toward at this early stage of the presidential campaign is the number of media hits that each candidate is receiving. Apart from being interesting unto itself, it’s plausible that this metric has some predictive power. At this point in 2007, Barack Obama and John McCain were receiving the most coverage among the Democratic and Republican candidates respectively, and both won their races despite initially lagging in the polls.

In contrast to four years ago, however, when the relative amount of media coverage was fairly steady throughout the campaign, there have already been some dramatic shifts this year. Sarah Palin’s potential candidacy, for instance, is only receiving about one-fifth as much attention as it did several months ago.

In the past, I’ve usually used Google News to study these questions, but I’ve identified another resource — NewsLibrary.com — that provides more flexibility in search options and more robustness in its coverage. (One problem with counting things on Google is that the number of hits can vary fairly dramatically from day to day, for reasons I don’t entirely understand.)

(Another downside to Google News: it seriously overweights the left).

I’ve counted the number of times on NewsLibrary.com in which the candidate’s name appeared in the lead paragraph of the article, and a select combination of words appeared down in the article body. In particular, I’ve looked for instances in which any combination of the words “president”, “presidential” or “presidency” appeared, as well as any of the words “candidate”, “candidacy”, “campaign”, “nomination” or “primary.”

The idea is to identify cases in which a candidate was the main focus of the article (as opposed to being mentioned in passing) and when the article was about the presidential campaign itself (as opposed to, say, Mr. Trump’s reality show). The technique isn’t perfect — there are always going to be a few “false positives” from out-of-context hits — but it ought to be a reasonably good benchmark for the amount of press attention that each candidate is getting.

And the results?

So far this month, however, Ms. Palin has accounted for just 124 hits out of 1,090 total, or roughly 11 percent. Instead, her place has been taken by Mr. Trump, who has accounted for about 40 percent of the coverage.

The decline in media coverage for Ms. Palin tracks with a decline in her polling numbers. Whereas she was pulling between 15 and 20 percent of the Republican primary vote in polls conducted several months ago, she’s down to about 10 percent in most surveys now.  Mr. Trump, meanwhile, whose media coverage has increased exponentially, has surged in the polls, and is essentially in a three-way tie for the lead with Mitt Romney and Mike Huckabee over an average of recent surveys.

Hm.  What do you suppose the odds were that the mainstream media would pump the hell out of a buffoonish cartoon like Trump at the expense of the serious GOP candidates?

After the MN Gubernatorial election we noted that  the “Bandwagon Effect” is known to have an effect on election turnout,  shown in academic studies on the subject.  As studied, it’s a negative effect – people are less likely to turn out for candidates that the media says are getting drubbed in the polls (like the Humphrey Institute’s polling last fall, which showed Emmer near-tie race as a 12 point loss with all-too-convenient timing.

So why would the media not be building up Trump as a “force to be reckoned with”?  It’s a win/win for the Media and the Democrats (pardon the redundancy); as long as Trump is pictured as a contender, GOP candidates have to waste time and money fighting the strawman with the bad combover.  And if by some freak of fate he gets the nomination (he won’t, because he’s no conservative, but let’s run with it) the media will tear him down promptly, because – let’s be honest – that’s what he’s there for.

This blog will be watching the libs/media and their bandwagonning over the next year and a half.  It’ll be a growth industry.

The Great Poll Scam, Part XIV: Fool Me Ten Times…

Posted on by
3

You’ve heard the old saying – “the definition of insanity is repeating the same thing over and over again and expecting a different result. 

The joke writes itself.  Nearly every election season, Minnesota’s media runs the results of the Star-Tribune Minnesota Poll and the Humprey Institute/MPR Poll on its front pages; front and center on its 6 and 10PM newscasts; up-front in its hourly news bites; in the New York Times; prominently on that big news crawl above Seventh Street in downtown Saint Paul.   To those who don’t dig into the numbers – and that’s probably 99 percent of Minnesota voters – that’s all there is to it.  “Hm.  Looks like Dayton’s winning big!”.

In most elections- especially the close ones - both polls (along with their downmarket stepsibling, the SCSU Poll) show numbers for GOP candidate that beggar the imagination.  The media – the Strib, the TV stations, MPR – run the polls pretty  much without any analysis.  The job of actually fact-checking the polling falls to conservative bloggers – myself, MDE, Ed Morrissey, Scott Johnson and John Hinderaker, Gary Gross, the Dogs, Sheila Kihne and others; poll after poll, election after election, we shout into the storm “the numbers are a joke! Democrats are oversampled to an extent that is not warranted by electoral results we’ve seen in this state in nearly a generation!  Would someone look into this?”

The elections take place.  There is hand-wringing about the inaccuracy of the polls.  Two years pass.  Larry Jacobs and the  Strib release still more polls, repeating precisely the same pathologies, over and over and over.  Forever and ever, amen.  Lather, rinse, repeat.

Now, “journalism” is supposed to be about accuracy and clarity.  About telling the story, and telling it in a way that your sources reinforce your credibility and clarity.  If you are a reporter, and you report a story based on a source’s information, and that information turns out to be wrong, it’s a bit of a vocational black eye.

This morning I asked, rhetorically, “do you think that if a source burned Tom Scheck or Pat Doyle or Rochelle Olson or Rachel Stassen-Berger over and over, year in and year out, by feeding them laughably inaccurate information, not just once or twice but on nearly every story on which they are a key source, would they keep using them as sources?”  Without really serious corroboration, if indeed it could be found?  Ever?  

And yet the regional media not only continues running the Strib and HHH polls, election after election, without any serious question – until after the election, anyway.  Notwithstanding the fact that the Strib’s Minnesota Poll has been very regularly wrong for a generation now.  Notwithstanding the fact that the Humphrey Poll has been even more consistent in its systematic shorting of GOP candidates.  The polls are still treated not only as useful news, but front-page material.

This would prompt a curious person to ask a whooooole lot of questions:

Why do the pollsters continue to generate such a defective product?:  While I focused heavily over the past few days specifically on Gallup’s Frank Newport’s critique of the Humphrey Institute poll, that gives the impression that this is a one-time issue.  And yet both the major media polls have had nearly the same problems, election in, election out, for a generation (or in the case of the Humphrey Institute Poll,  in every major election since 2004).   It’s gotten to the point where I want to stand outside 425 Portland, or outside the Humphrey Institute’s building at the U, and wave a sign about; “It’s the same thing, every time!”.

Why do the media continue to present such a routinely defective product as newsworthy?:  Scott Johnson has been lighting up the “Minnesosta Poll’s” shortcomings for a solid decade now; the Strib’s poll is rarely even close, and performs worse in close elections than in blowouts.  And at the risk of repeating myself, let me repeat myself; the Humphrey Institute poll has underpolled Republicans by an average of nine points.  This past election was distinguished from the previous years’ ineptitude only in degree, not in concept.

Does it never occur to our “watchdogs” and “gatekeepers” to look into this?  Wasn’t “insatiable curiosity” once a pre-requisite for being a reporter?

Do the editors at the Strib, the PiPress, KARE, MPR, WCCO and the rest of the regional mainstream media genuinely consider “polls are a snapshot in time” an excuse for decades worth of a pattern of inaccuracy, not only in polling technique but in their own coverage of elections?

If a city councilman is caught cashing checks to herself, would saying ”it’s just a snapshot in time!” get the Strib to call their dogs off?

Appearance Of…Something?: I’ve said it before; I’m not a fundamentally conspiracy-minded person.  I don’t necessarily believe that the media is involved in a conscious, considered conspiracy to short conservative candidates in close elections.

Still – given that…:

…I’ll ask again: if the Humphrey Institute (whose institutional sympathies lean definitively left-of-center) and the Strib (ditto) wanted to create a system that would help tip close-call contests toward the DFL, how would it be any different than the system they’ve developed?

Not accusing.  Just asking.

The Great Poll Scam, Part XIII: Reality Swings And Misses

Posted on by
5

Contrary to the impression some wrote about on various blogs, I never worked for the Emmer campaign.  Oh, I did a fair amount of writing about Emmer’s bid for governor – I thought he had what it took to be the best governor we’ve had in a long time, and I was a supporter from long before he actually declared his intent to run.  I volunteered a lot of time, and a lot of this blog’s space, to fight against the sleaziest, most toxic smear campaign in recent Minnesota electoral history, and I do believe the better man lost this election.

But I never got any money for it.

What I did get – although not to an extent that would make a Tom Scheck or a Rachel Stassen-Berger in any way jealous – was a certain amount of access.  I heard things.

One of the things I heard from sources inside the Emmer campaign, especially during the long, dry, advertising-dollar-free summer before the primaries, when all three DFL contenders curiously spent their entire ad budgets sniping at Emmer, and the media played dutiful stenographers for Alliance for a Better Minnesota’s smear campaign, was that the Emmer campaign had its work cut out for it.  In late July and early August, a source inside the Emmer campaign, speaking on MI-5-level deep background, told me the internal polls showed Emmer trailing by 12 points.  It wasn’t good news, certainly – but it was early in the race, it was a byproduct of being outspent roughly 16:1 to that point, and it was just part of doing business.   ”We gotta pick up six points, and Dayton’s gotta lose six”, the source told me, as the campaign dug its way out of “Waitergate”.

I observed to the source that that should have been nothing new for Emmer; he’d come back from a bigger margin in the previous nine months or so, from being way back in the pack at the Central Committee straw poll about this time last year, where Marty Seifert won by a margin many considered insurmountable.

The source expressed confidence it could be done.

He was, statistically, exactly right. Emmer brought the race back from a 12 point blowout to a near-tie, with numbers that pretty steadily improved – according to the party’s own internal polling.

Steadily?

On October 11, I held a “Bloggers For Emmer” event at an undisclosed location in the western subs.  It had been ten busy weeks since my off-the-record conversation with my source in the campaign.  An Emmer functionary told me – off the record – that it was now a four point race.  

A week later, within ten days of the election, the same internal poll said the race was a statistical dead heat.

Then came the last-minute hit polls from the Humphrey Instititute, the Strib and Saint Cloud State – after which Emmer released his internal polling, which was reinforced by a Survey USA poll that more or less reinforced the internal polls’ results.

And then came the election.

Last week, David Brauer at the MinnPost interviewed Emmer campaign manager Cullen Sheehan.  As part of the piece, he graphed the respective polls: Emmer’s internal polling (orange), the Strib poll (wide dashes) and the HHH poll (dots), showing the indicated size of the Dayton lead.

Graph used by permission of the MinnPost

Graph used by permission of the MinnPost

Brauer:

Although “internal numbers” often become propagandistic leaks, Sheehan insists the data was not for public pre-election consumption. Though he wound up releasing the most favorable result during the campaign, it proved prescient, and two independent pollsters subsequently showed similar results.

And while Brauer points out that internal numbers “aren’t holy” – and many leftybloggers openly guffawed when Sheehan released them – the GOP’s internal numbers have a long record of accuracy, in my experience.  In 2002, when the Strib poll had Roger Moe measuring the drapes in the mansion, a GOP source leaked me internal polling showing that Pawlenty was tied and rising.  And internal polling released to a group of bloggers a month before the election showed Chip Cravaack pulling close to Jim Oberstar; numbers that the campaign asked be kept off the record showed that with “leaners”, Cravaack was actually leading.

So for all the leftyblogs’ caterwauling about “push polling”, the GOP’s internal polls – as seen both publicly and behind the scenes – called things as they were.  There’s a reason for that; parties need to accurate polling to help them allocate scarce resources effectively.  The DFL has not released their internal polling – but the Dayton campaign’s behavior indicates to me that they also saw Emmer’s late surge, leading them to re-roll-out the “Drunk Driving Ad” (the closest the Dayton campaign ever came to a coherent policy statement, with full irony intended).

But neither sides’ internal polling is affiliated with a major media outlet.  The Strib, Minnesota Public Radio and MinnPost all have symbiotic relationships with Princeton, the Humphrey Institute and Saint Cloud State, respectively (though to be accurate the MinnPost only paid for three questions in the SCSU poll, and those were, according to Brauer, on ranked-choice voting).  Those relationships, presumably, exist so that the news outlets can get “their” results out to the public first.

No matter how they’re arrived at, or so it seems.

Brauer confirms after the fact what my sources in the campaign told me, off the record, at the time; it was a real numerical rollercoaster ride:

Although “internal numbers” often become propagandistic leaks, Sheehan insists the data was not for public pre-election consumption. Though he wound up releasing the most favorable result during the campaign, it proved prescient, and two independent pollsters subsequently showed similar results.

“It really is, internally, a compass,” Sheehan says of the campaign’s polling.

Emmer’s own numbers show a candidate trailing — sometimes badly — for nearly the entire race.

On July 28 — three weeks after Emmer’s interminable “tip credit” debacle — the Republican trailed Dayton by 11 points. Ironically, the Star Tribune poll — which Republicans say overstates DFL support — had it closer: Dayton plus-10.

It was a demonstrable fact that the Strib poll oversampled DFL voters by a big margin – but that’s a poll-technique discussion to be held some other time.

In the wake of the double-digit gap, Sheehan took over as campaign manager. But by early October, the internal numbers had barely budged: Emmer was still down 7. A Strib survey taken a week or so earlier showed the Republican down 9 — again, pretty close to what the campaign was seeing.

Finally, on Oct. 13, Emmer got his first great inside news: he was only down 1. But the next media poll (SurveyUSA/KSTP) had him down 5, and an Oct. 18 internal poll repeated that number. It was two weeks before Election Day.

And then came the Big Three media polls, one after the other – the Strib, SCSU and the Humphrey polls – showing Emmer 9, 10 and 12 points down, respectively.  At which point Sheehan opted to release the internal numbers – which were shortly reinforced by SUSA.

Sheehan:

“At that point [right before the election - the polls on which I've focused throughout this series], undecided voters are making up their minds and supporters are getting anxious, having seen 7 down, 10 down and 12 down,” Sheehan says. “It impacts fundraising and volunteers. It’s definitely not the only factor, but it is a factor.”

Sheehan, now the Minnesota GOP Senate caucus chief of staff, is a Republican, but Democratic Senate Majority Leader Harry Reid’s pollster feels similarly. Reid’s internal numbers proved better than media polls predicting his opponent would win.

Says Sheehan, “The point I am making is that outside public polls have an impact on campaigns — ultimately, some impact on eventual outcome of campaigns, especially in close races.”

At least one media outlet agreed even before the results were known. This year, the Star Tribune declined to do its traditional final-weekend poll. A key reason, editor Nancy Barnes told me, is that “a poll can sometimes influence the outcome of an election.”

Sheehan’s plea? Withhold questionable numbers. “I’m under no illusion that public polls will cease, but I do think news organizations have a responsibility to ask themselves, when they get their results, if they really believe they’re accurate,” he says.

I’ve met Sheehan not a few times.  Great guy.  Big future in politics.  Now, I’m not sure if he’s ever read this series; if he has, I’m sure he needs to be diplomatic.  He’s gotta get along with the regional media.

But the fact remains that the closer the race got, the farther off-the-beam the Strib and HHH polls swerved.

Just the same as they do in practically every election, especially the close ones.

So Sheehan has a point; the news media should treat suspicious polls as they would a source that’s burned them. 

Seriously – can you imagine Erik Black or Bill Salisbury or David Brauer putting a story on the front page (or “page”) based on the uncorroborated word of a source that had burned them, over and over again?  As in, not even close, but really, really embarassingly burned?

And the Strib and Humphrey Polls have burned the regional media – over and over and over again.

Presuming, of course, that accuracy is what they’re shooting for.

More later today.

The Great Poll Scam, Part XII: The Dog Ate Their Homework

Posted on by
17

Writing in defense of the Humphrey Institute Poll – which indicated our tie governor’s race was headed for a 12 point blowout – Professor Larry Jacobs says:

Careful review of polls in the field conducting interviews during the same period indicates that the MPR/HHH estimate for Emmer (see Figure 2) was within the margin of sampling error of 3 of the 4 other polls but that it was also on the “lower bound” after accounting for random sampling error. (Its estimate was nearly identical to that of MinnPost’s poll with St. Cloud.)

Which showed the race a ten point blowout for Dayton.

Jacobs is, in effect, saying “yeah, our poll was a hash – but so was everyone else’s”.

This pattern is not “wrong” given the need to account for the margin of sampling error, but it is also not desirable. As part of our usual practice, the post-election review investigated whether there were systematic issues that could be addressed.

Research suggests three potential explanations for the MPR/HHH estimate for Emmer; none revealed problems after investigation.

Indeed.

Here are the three areas the Humphrey Institute investigated:

Weighting: First, it made sense to examine closely the weighting of the survey data in general and the weighting used to identify voters most likely to vote. Weighting is a standard practice to align the poll data with known demographic and other features such as gender and age that are documented by the U.S. Census. (Political party affiliation is a product of election campaigns and other political events and is not widely accepted by survey professionals as a reliable basis for weighting responses.)

Our own review of the data did not reveal errors that, for instance, might inflate the proportion of Democrats or depress that of Republicans who are identified as likely to vote. To make sure our review did not miss something, we solicited the independent advice of well-regarded statistician, Professor Andrew Gelman at Columbia University in New York City, who we had not previously worked with or personally met. Professor Gelman concluded that the weighting was “in line with standard practice” and confirmed our own evaluation.

“And an expert said everything’s hunky dory!”

Our second investigation was of what are known as “interviewer effects” based on research indicating that the race of the interviewer may impact respondents.11 (Forty-four percent of the interviewers for the MPR/HHH poll were minorities, mostly African American.) In particular, we searched for differences in expressed support for particular candidates based on whether the interviewer was Caucasian or minority. This investigation failed to detect statistically significant differences.

And the third was the much higher participation in the poll from respondents in the “612″ area code – Minneapolis and its very near ‘burbs.  Jacobs (with emphasis added by me):

When analyzing a poll to meet a media schedule, it is not always feasible to look in-depth at internals.

It’s apparently more important to make the 5PM news than to have useful, valid numbers.

With the time and ability that this review made possible, we discovered in retrospect that individuals called in the 612 area code were more prone to participate than statewide — 81% in the 612 area as compared to 67% statewide in the October poll.13 Given that Democratic candidates traditionally fare well among voters in the 612 area code, the higher cooperation rate among likely voters in the 612 area code may explain why the estimate of Emmer’s support by MPR/HHH was slightly lower than those by other polls conducted at around the same time. This is the kind of lesson that can be closely monitored in the future and addressed to improve polling quality. 

Except we bloggers have been “closely monitoring” this for years.  It’s been pointed out in the past; on this very blog, I have been writing about this phenomenon since 2004 at the very latest.  Liberals looooove to answer polls.  Conservatives seem  not to.

That Jacobs claims to be just discovering this now, after all these years, is…surprising?

Frank Newport at Gallup critiques Jacobs’ report:

The authors give the cooperation rate for 612 residents compared to the cooperation rate statewide. The assumption appears to be that this led to a disproportionately high concentration of voters in the sample from the 612 area code. A more relevant comparison would be the cooperation rate for 612 residents compared to all those contacted statewide in all area codes other than 612. Still more relevant would be a discussion of the actual proportion of all completed interviews in the final weighted sample that were conducted in the 612 area code (and other area codes) compared to the Census estimate of the proportion of the population of Minnesota living in the 612 area code, or the proportion of votes cast in a typical statewide election from the 612 area code, or the proportion of the initial sample in the 612 area code. These are typical calculations. The authors note that residents in the 612 area code can be expected, on average, to skew disproportionately for the Democratic candidate in a statewide race. An overrepresentation in the sample of voters in the 612 area code could thus be expected to move the overall sample estimates in a Democratic direction.

That Jacobs finds an excuse for failing to weight for higher participation in a city that is right up there with Portland and Berkeley as a liberal hotbed would be astounding, if it weren’t the Humphrey Institute we’re talking about.

The authors do not discuss the ways in which region was controlled in the survey process, if any. The authors make it clear that they did not weight the sample by region. This is commonly done in state polls, particularly in states where voting outcomes can vary significantly by region, as apparently is the case in Minnesota.

Summary:  The HHH poll is sloppy work.

The Great Poll Scam Part XI: Weasels Rip My Results

Posted on by
6

Professor Larry Jacobs – by far the most-quoted non-elected person in Minnesota – defends the Humphrey Institute Poll:

Differences between polls may not be substantively significant as illustrated by the case of MinnPost’s poll with St. Cloud State, which showed Dayton with a 10 point lead, and the MPR/HHH poll, which reported a 12 point lead.

The “margin of sampling error,” which is calculated based on uniform formulas used by all polling firms, creates a cone around the estimate of each candidate’s support, reflecting the statistical probability of variation owing to random sampling.2 The practical effect is that the results of the MinnPost poll with St. Cloud State and MPR/HHH are, in statistical terms, within range of each other. Put simply, the 2 points separating them may reflect random variation and may well not be a statistically meaningful difference.

What might be a “statistically meaningful difference” is that Survey USA and Rasmussen all came much, much closer – as in, one-third to one-quarter of the Strib, HHH and St Cloud polls – to getting the actual election right, and tracked much closer to the GOP’s internal polling, which turned out to be dead-nut accurate (as we’ll see tomorrow).

Figure 2 creates a zone of sampling error around estimates of support for Dayton and Emmer by the five media polls completed during the last two weeks of the campaign.3 In terms of the estimates of Dayton’s support, the MPR/HHH poll is within the range of all four other polls. Take home point: its estimate of Dayton’s support was consistent with all other polls.

Well, no.  It was consistent with the other polls who have developed a reputation for inaccuracy that inevitably favors the DFL.  The other polls – Survey USA, Rasmussen, Laurence – were not consistent with the Humphrey poll at all.

Frank Newport of Gallup responds to this:

It is unclear from the report how much the write‐up of results from the October 21‐25 MPR/HHH poll emphasized the margin of error range around the point estimates. Although this is not part of their recommendation, if the authors feel strongly that the margin of effort around point estimates should be given more attention, future reports could include more emphasis on a band or range of estimated support, rather than the point estimates.

In other words, if the Humphrey Poll is really a range with no particular confidence in any particular number within the range, publicize the range.

But that’s not what the Humphrey Institute, or the media, led with just before the election.  It was “DAYTON LEADS BY 12″.  Not “Dayton leads by 8 to 16, maybe, sorta”.

The distinction might make a difference.

This is generally not done in pre‐election polling, under the assumption that the point estimate is still the most probable population parameter. Any education of the public on the meaning of margin of errors and ranges and comparisons of the margins of errors surrounding other polls is an admirable goal. It does, however, again raise the question of the purpose of and value of pre‐election polls if they are used only to estimate broad ranges of where the population stands. This topic is beyond the scope of this review.

In other words – if you take Jacobs at his word, then there’s nothing really newsworthy about the HHH poll.

Do you suppose they’ll stick with that line in the runup to the 2012 election?

The Great Poll Scam, Part X: Weasel Words

Posted on by
12

I’ve been raising kids for a long time.  Before that, I grew up around a bunch of them.  Indeed, I was one myself, once.

And I know now as I knew then the same thing that every single person who watches Cops knows, instinctively; if you think someone did something, and their response is “you can’t prove it”, it’s the same as an admission of guilt.

Oh, it doesn’t stand up in court – and it’s probably a good thing.

And in the rarified world of academics – and its poor, profoundly handicapped accidental offspring, political public opinion polling – I’m going to suggest it works the same way.

If there is a poll that is, year in and year out, just as ludicrous as the Humphrey and Strib polls, it’s the Saint Cloud State University poll.  I haven’t heretofore included it in my “Great Poll Scam” series, because it’s sort of out of sight and out of mind.

But in David Brauer’s interview with Emmer campaign manager Cullen Sheehan, the director of the SCSU poll – which is done in conjunction with the MinnPost – a fellow named Stephen Frank, tips us off; he concludes…:

Frank says. “Campaign managers like to find excuses rather than looking at their candidate or performance. Do you think if we stopped [publishing results] others would — or the candidates would and the latter won’t go public or only partially public?”

True, to a point.

But he began the statement by saying:

“Please show me one credible study that shows people change their mind on the basis of a poll,”

On the one hand:  “You can’t proooooooooove we did it!”

On the other hand – allow me to introduce you to Dr. Albert Mehrabian, who published a study entitled “Effects of Poll Reports on Voter Preferences”

From the abstract summary, with emphasis added:

Results of two experimental studies described in this article constituted clear experimental demonstration of how polls influence votes. Findings showed that voters tended to vote for those who they were told were leading in the polls; furthermore, that these poll-driven effects on votes were substantial.

How substantial?  I don’t know.  As I write this, it’s 5AM, and I have no way of getting to the University of Minnesota library to find a copy of Journal of Applied Social Psychology (Volume 28).  But I will.

But Mehrabian noted a decided “bandwagon effect” in voter responses to poll results.

Effects of polls on votes tended to be operative throughout a wide spectrum of initial (i.e., pre-poll) voter preferences ranging from undecided to moderately strong. There was a limit on poll effects, however, as noted in Study Two: Polls failed to influence votes when voter preferences were very strong to begin with.

Bingo.

I’d have voted for Tom Emmer even if he did finish 12 points back, as the Humphrey Institute suggested.  Or ten points out of the game, as Frank’s survey (which I ridiculed in this space), or thirty points back.  But then, nobody really doubted that.

But people who don’t live and breathe politics?  That’s another story – says Dr. Mehrabian.

Additional findings of considerable interest showed that effects of polls were stronger for women than for men and also were stronger for more arousable (i.e., more emotional) and more submissive (or less dominant) persons.

Which would be important, in a year when the DFL was worried about women flaking away from Dayton, and moderates being drawn (successfully!) to the Tea Party.

Wouldn’t it?

Especially noteworthy is my discussion of similarities and differences between the study methods and real- life political campaigns beginning with the middle paragraph on page 2128 (“Overall, results …).

I’ll dredge up a copy of Mehrabian’s study (unless any of you academics out there can shoot me a pointer…).

Mehrabian was cited in this study of the subject – “Social information and bandwagon behaviour in voting: an economic experiment“, by Ivo Bischoff and Henrik Egbert, a pair of German economists; the paper isn’t about the bandwagon effect – but it touches on it pretty heavily (all emphases are added by me):

The political science literature contains a number of empirical studies that test for bandwagon behaviour in voting. A first group of studies analyses data from large-scale opinion polls conducted in times of upcoming elections or on election days. The evidence from these studies is mixed (see the literature reviews in Marsh, 1984; McAllister and Studlar, 1991; Nadeau et al., 1997). One essential shortcoming of these studies is that it is very difficult to disentangle the complex interrelations between voting intentions, poll results and other pieces of information that drive both of the former simultaneously (Marsh, 1984; Morwitz and Pluzinski, 1996; Joslyn, 1997). Avoiding these difficulties, a second group of studies are based on experiments. Mehrabian (1998) presents two studies on bandwagon behaviour in voting. In his first study, he elicits the intended voting behaviour among Republicans in their primaries for the presidential election in 1996. He finds that the tendency to prefer Bob Dole over Steve Forbes depends on the polls presented to the voters. Voters are more likely to vote for Dole when he leads in the opinion poll compared to the situation with Forbes leading. The second study involves students from the University of California, Los Angeles. These are asked to express their approval to proposals for different modes of testing their performance: a midterm exam or an extra-credit paper. Mehrabian (1998) uses bogus polls in his studies. Results show that bogus polls do not influence the answers when subjects have clear and strong preferences. However, bogus polls have an impact when preference relations are weak. In this case, bandwagon behaviour in voting is observed. Next to Mehrabian (1998), there are a number of others experimental studies that find evidence for bandwagon behaviour in voting (Laponce 1966; Fleitas 1971; Ansolabehere and Iyengar 1994; Goidel and Shields, 1994; Mehrabian 1998).

It’s not an open-and-shut, according to Bischoff and Egbert – but there is evidence to suggest that the “Bandwagon Effect” exists, and that polling drives it.

Is it possible that the learned Professors Larry Jacobs or Stephen Frank are unaware of this?  Certainly.

Given both polls’ lock-step consistency, especially at under-polling GOP support in close elections, where people with weak initial preferences – people whose “preference relations are weak”, as Bischoff and Egbert put it, which might well be as good a good description for “independents” and “swing voters” as I’ve seen -  it’s worth a look, though.

More from Dr. Mehrabian in the near future.