Daily Kos goes to war with Research 2000

[The Mad Monk]

Internet Goes to War as Daily Kos Renounces Its Long-Time Polling Firm
More By Max Fisher on June 30, 2010 9:49am For a year and a half, liberal website Daily Kos commissioned Research 2000 to conduct political polling that was well respected and used by everyone from political operatives to mainstream media outlets to The Atlantic Wire. Daily Kos chief Markos Moutlitsas recently announced he was dropping Research 2000, sometimes called R2K, in search of a new polling firm. Now he's explained why.


He wrote on Tuesday, "I have just published a report by three statistics wizards showing, quite convincingly, that the weekly Research 2000 State of the Nation poll we ran the past year and a half was likely bunk." Moulitsas promised a lawsuit, adding, "We were defrauded by Research 2000, and while we don't know if some or all of the data was fabricated or manipulated beyond recognition, we know we can't trust it." What does this mean for Daily Kos, the statistics world, and the polling that drives so much of politics?

A 'Bombshell' for Political World Talking Points Memo's Josh Marshall writes, "To say that this is a bombshell in the polling and politics geek world is no overstatement. And remember, R2K didn't start out as Kos's pollster. They've been around for some time and had developed a pretty solid reputation." Yahoo News' John Cook adds, "Research2000's polls are widely cited; the prospect that they might be 'bunk,' as Moulitsas calls them, means that a significant amount of bad data could have been entering the political pipeline."
The Evidence of R2K Fraud The Atlantic's Megan McArdle explains, "To summarize crudely, there doesn't seem to be any random statistical noise in the data; there are no outliers, and there's an extremely odd tendency for the male and female crosstabs to both end in either an odd, or an even number; since these numbers vary pretty randomly, there's no reason that they should consistently be both odd or both even." Pollster's Mark Blumenthal adds, "the most damning information for the layperson about Research 2000 provided in today's announcement -- and is certainly most troubling to me -- is their apparent reluctance to share raw data with their own client." Nate Silver explains his own long-held objections to R2K.
This Blog War Won't End Soon CBS News' Charles Cooper writes, "For the record, Research 2000's president denied Moulitsas' allegations. Until the facts get sorted out, we're left with a he-said, she-said scenario. ... But with the story already taking on a life of its own, this much is clear: the Kos folks are going to eat a fair amount of public crow for an indeterminate period."
R2K Owner Has Shady Financial Record Yahoo News' John Cook reports, "Del Ali, the firms's president, has been sued numerous times in his home state of Maryland for nonpayment of debt and has been hit with several tax liens, according to court records." Cook details the debts and charges.
R2K Threatens Lawsuits Against Critical Blogs FiveThirtyEight's Nate Silver writes that he received a cease and desist letter, which also threatens a lawsuit, alleging "a campaign to discredit and damage R2K by posting negative comments regarding Mr. Ali, the Company, and its work products on the Daily Kos blog."
Will This Scuttle Kos's Forthcoming Book? National Review's Jonah Goldberg notes, "A friend points out that Research 2000 was behind the 'Republicans are crazy wingnuts' poll of not too long ago. Here's Markos writing about it." Goldberg quotes a February post where Moulitsas links the thesis of his soon-to-be-released book to some high-profile R2K polling.
Controversial R2K Poll on GOP Still Looks Reliable The Washington Post's Garance Franke-Ruta writes, "One of the most provocative findings of the firm was that a substantial portion of Republicans did not believe President Obama was born in the United States. A July 2009 poll for Daily Kos found that 28 percent of Republicans were 'birthers,' as those who question Obama's birth place are frequently called. Washington Post polling in April 2010 found similar results, with 31 percent of Republicans saying Obama was born in another country." Public Policy Polling's Tom Jensen concurs:
I'm seeing a lot of conservatives, referring to the emerging possibility that Research 2000's polls for Daily Kos were fradulent, say that means all of the findings from the infamous poll about crazy things Republicans think are bogus. While I'd agree that Research 2000's poll may have been bogus, I would not agree it means Republicans don't think those things. We have polled on many of the same issue over the last year and in some cases found even more Republicans buying into some of those conspiracy theories.
The Debate
Shady Practices Megan McArdle, The Atlantic
Ruin Kos' Book? 34Jonah Goldberg, National Review
Bombshell 29Josh Marshall, Talking Points Memo
We Matched Results Garance Franke-Rita, The Washington Post
R2K Owner's Financial Dealings John Cook, Yahoo News

http://www.theatlanticwire.com/opinions/view/opinion/Internet-Goes-to-War-as-Daily-Kos-Renounces-Its-Long-Time-Polling-Firm-4170

Lot of made up polls, I guess...

[Ben Kenobi]

Least surprising news. This is daily Kos after all.

[Dinner]

If a polling company isn't doing the job it is paid to do then why not fire them and switch to another polling company? In any event it is their money so they can decide who they hire with it.

[The Mad Monk]

Research 2000 Called into Question Permalink
The Daily Kos Website has been commissioning pollster Research 2000 to run polls for it. Now the founder, Markos Moulitsas, has repudiated Research 2000 and told everyone to delete all its tracking polls from their data bases due to a statistical analysis that suggests they have been curbstoning (making up the data). Kos, as he is known, didn't say anything about Senate polls, but if Research 2000 is unreliable with tracking polls, it is probably unreliable about everything.

This leaves us with a big problem. While we are not going to start daily posts of the polling data until September, when the primaries are largely over and people begin paying attention to the election, at that point we will be faced with the problem that very few pollsters are left and of those, some have other issues. SurveyUSA and PPP are probably pretty solid but the largest "nonpartisan" pollster is Rasmussen, who now works for Fox News. While Fox is theoretically a news organization, it has a definite slant on the world, that say, ABC News does not have. There is no evidence that Rasmussen makes up numbers, but there are other issues here. In particular, most pollsters, including Rasmussen, have a model of the electorate and normalize their polls to it. Very briefly, suppose Rasmussen believes that the set of likely voters (which is very different from the set of registered voters and very, very different from the set of people over 18) is 40% Republican, 30% Democratic, and 30% independent. If an actual poll turns up 200 Republicans, 200 Democrats, and 200 independents, he will weight each Republican respondent by 40/33, each Democratic respondent by 30/33 and each independent by 30/33 to compensate for the bad mix in his small sample.

In principle, such weighting is not only legitimate, but a good idea. But if the model has too many of one party and too few of the other, it will affect the results of course since the people in the underrepresented groups will count more. If the pollsters released the raw data, then others could undo this effect, but most pollsters do not release the raw data. In the case of Research 2000, it wouldn't even release it to Kos, its own client, which is very fishy. Weighting is also done for gender, age, and other demographic characteristics. Again, doing this is actually a good thing, but a lot depends on how good the model is and whether that has been fudged at all.

One might think that it is easy to tell which pollsters are accurate and which are not by comparing their polls to the final election results, but such is not the case. For the final polls a few days before the election, this is true, but for general election polls in June (which are probably not worth much anyway), there is no way to check. It is not inconceivable that a polling organization with an ulterior motive could issue biased polls most of the year (to encourage supporters of its favored party and discourage the opposition) and then as the election approached, gradually remove the bias and try to be as accurate as possible, to acquire a good reputation for next time.

Research 2000 has hired a law firm to try to intimidate the media (including the blogs) into not talking about this issue. Nate Silver of 538.com has already gotten his. I'll check my mailbox later today expectantly.

Charles Franklin, a professor of political science at the University of Wisconsin at Madison, is worried that scandals like this may hurt the polling industry as a whole. His proposal is that everyone commissioning a poll should write into the contract that the pollster must deposit the raw data in the Roper Archive at the University of Connecticut, which collects them. This would allow third parties to analyze the raw data, do statistical checks on them, like using Benford's Law, and renormalize them using different models of the electorate.

We'll keep an eye out for further developments in the Research 2000 affair. For the time being, they have been marked as suspect in the internal data base.

http://www.electoral-vote.com/evp2010/Senate/Maps/Jun30-s.html#1

[Ogie Oglethorpe]

Interesting the attempt by association to discredit the pollster Rasmussen. I never seem to recall a similar call to alarm previously towards R2K despite their obvious relationship with 'nonpartisan' Moulitsas. I do give Moulitsas credit for having the intellectual honesty to call it quits with R2K if he believes the polling data fabricated.

Rasmussen if I am not mistaken has a differing methodlogy that looks at likely voters rather than a simple adult polling methodlogy.