Who is going to win?

[dannubis]

Reputation for the larger companies is quite important, but there will always be low cost providers.
Working for one of the largest research companies in the world, reputation is quite important

[Wezil]

How much revenue are you likely to get long term if you are a widely recognized polling firm who is consistently polling inaccurately?

I've made this point before. A conservative on the boards accused a polling firm of essentially lying about a poll result in our last election (the pollster was obviously a dirty liberal out to suppress conservative votes). I questioned the wisdom of such a business move.

Turns out the poll numbers were actually pretty close....

[Jaguar]

This thread has rightly circled around the important issue. The polling data is bimodal. Gallup and Rasmussen say one thing, and other prominent pollsters say something else. The difference between Gallup and Rasmussen and, say, NBC/Marist or PPP or whomever, is so large that we can say with statistical certainty that they aren't sampling from the same distribution. (Incidentally, it's not cellphones. Gallup samples cellphones, and they're far more Romney-leaning than Rasmussen is.)

If Gallup and Rasmussen are right, Romney will win. If the other ones are right, Obama will win. If it's somewhere in between, Obama will probably win but it will be very close.

The difference is in their model of the electorate - particularly, with respect to party affiliation. Here's Gallup's prediction. They believe that self-identified Democrats and self-identified Republicans exist in approximately equal numbers, as they did in 2004. Here's Rasmussen, who believes the same. In such an election, whoever cobbles together a lead among independents should be the victor. Gallup and Rasmussen both have Romney with a lead among independents, and therefore, a lead overall.

In contrast, look at a typical unfavorable state poll. Here's a PPP poll (PDF) from Ohio. It clearly disagrees with Gallup and Ras about the state of the race. If Romney has a clear popular vote margin, he would almost certainly win Ohio outright. Let's see why it disagrees. First check out the crosstabs on page 14. Each candidate wins his own party by 90-10 margins. Romney wins independents by 4. These seem consistent with the story we heard above. But then, check out question 16. PPP's poll finds that, of its likely voters, 43% self-identify as Democrats, and 35% self-identify as Republicans. This is the major - and gigantic - area of disagreement between the Obama-leaning results and the Romney-leaning results. It's nothing like the numbers in the models put forward by Gallup and Rasmussen.

I only showed you one state poll, but just trust me that the discrepancy is similar for all of them. Here's one more. Different pollster, different state, same issue. Democrats lead 31-27 among likely voters in party identification, and therefore, lead the poll.

This disagreement has persisted for months now. Nate Silver has talked about it since July - though usually framed as a difference between the national polls and the state polls. Really it's a difference between Gallup/Rasmussen and everyone else. Gallup doesn't do individual state polls, and the large numbers of state polls by other firms tend to overwhelm Rasmussen in polling averages.

What it boils down to is that either Gallup and Rasmussen have correct models of the electorate, or they don't. Either we have a simple story where independents break for Romney and give him the win, or we have a story where Obama has built a party-ID advantage large enough to make up for losing the independent vote.

So who do we believe?

Beats me. Normally we'd happily believe both, because people mostly agree. This year, they don't. It hasn't happened before. We don't have a good idea of what predictions to make in this situation.

Someone has made an honest mistake here. (Let's stop assuming hackery - this job is really difficult because poll responses are not actually random samples from the ballot box.) Either Gallup and Rasmussen have a good handle on national trends, and are right not to expect a turnout advantage for self-identified democrats - or their models are wrong and need to be redone.

Romney's re-election relies on the state polls being wrong, across the board, by enough that he wins. Nate Silver's model is state-poll-based but accounts for polling error being correlated, which is why he doesn't call the election for Obama. He gives it a 28% chance or so that they're wrong by 2.1% in the swing states and 1.5% nationwide, giving Romney the win.

If you think the weird discrepancy described above makes the state polls look a little bit less reliable than usual, then you should bump up that 28% probability, and believe something more like 38%. That's where Romney trades on Intrade, and I think that price is reasonable.

[DinoDoc]

How accurate was Gallup in 08 & 10?

[Ben Kenobi]

Ras was off by about 2 points, Gallup the same. The people who are the professionals who care about this stuff already know this, Asher.

Thanks for the smackdown Jag.

[giblets]

That wasn't a "smackdown". You are retarded.

[MRT144]

Actually, based on gun purchase records from December 2008, Obama is one of the most effective gun salesmen of all time. Slowwhand's attitude is far from unique.

That and doomsday faux economist quacks hawking gun toting gold standard fantasies.

[Asher]

Ras was off by about 2 points, Gallup the same. The people who are the professionals who care about this stuff already know this, Asher.

Thanks for the smackdown Jag.

God, you're so special.

[pchang]

Ras was off by about 2 points, Gallup the same. The people who are the professionals who care about this stuff already know this, Asher.

Thanks for the smackdown Jag.

Mr. Rivers wants his license fee

[Ben Kenobi]

That wasn't a "smackdown". You are retarded

Ahahaha, If it were me - you'd be going on your usual shtick about how I'm obviously wrong.

[Ben Kenobi]

Mr. Rivers wants his license fee

Given his recent performance it should be the other way around. Gah.

[Jaguar]

Further information on pollster bias:

Partisan "lean" isn't very consistent:
You might remember this election. It was kind of important at the time. PPP was the first pollster to show a lead for Scott Brown, and doubled down with another poll before the election. Rasmussen was the penultimate poster to show a lead for Martha Coakley. PPP was right.

The partisan-leaning polls might be the accurate ones:
In 2010, Rasmussen was a far outlier in the Republican direction. Very early on, his tracker was massively Republican-leaning on the generic congressional ballot. People incredulously wrote that if you believed Rasmussen's numbers, the Republicans would gain sixty seats, and everyone knew that was impossible. Eventually, other pollsters saw the same trends happening and fell in line with Rasmussen. The Republicans ended up gaining 63 seats. Another example: CNN/USA Today/Gallup and CBS/NYTimes were far outliers in the Republican direction in the 2002 generic ballot, and they were proven correct.

Nonetheless, there is statistical merit in adjusting for partisan lean:
Gallup gave Romney a five point lead yesterday. NPR released a national poll today. A naive model would expect NPR to show Romney +5. A smarter model would notice that Gallup has been to the right of NPR this cycle, and expect NPR to show something more like the actual result - Romney +1. The naive model would think the race had changed, while the smarter model would mostly take NPR's result in stride. This is basically what Nate Silver does, and it's one of the reasons he's better than the RCP average.

[regexcellent]

One thing to consider is that Nate Silver doesn't change his model during the election season for consistency's sake. So any problems he may have identified based on novel polling circumstances aren't going to show up until next election.

[Jaguar]

One thing to consider is that Nate Silver doesn't change his model during the election season for consistency's sake. So any problems he may have identified based on novel polling circumstances aren't going to show up until next election.

For this, he deserves praise. It's a big statistical no-no to change your experimental design in the middle of the experiment.

Here's why he doesn't do it:

First, it would allow him to fish for the results he wants, introducing personal bias. Second, if he's allowed to change the model when he doesn't like the results, it's no longer clear what model he's actually testing. And third, if he wants to redesign his model based on new information, he should have just designed a model that incorporates that information in the first place.

[regexcellent]

Oh of course. What I'm saying is not meant to be criticism.