Is the New Washington Post Poll "Out of Whack?"

Update - According to Blair Lee with the Gazette of Business and Politics the Washington Post weights it polls according to Census data and not actual turnout - as a result the poll is overstating turnout among key constituent groups and heavily Democratic Baltimore City.

Original Post with revised analysis based on material provided by the Washington Post:

Much news has been made in the past few days since the Washington Post released a new poll in the Maryland governors race that showed incumbent Democrat Martin O'Malley opening a huge 11 point lead over former governor Bob Ehrlich. No other poll has found this race to be anything but a dead heat and news that the O'Malley campaign is bringing President Obama to the state certainly suggests that this is a close race.

When asked about the new poll, Ehrlich described it as "out of whack." In a story about Ehrlich's comments, Washington Post reporter John Wagner noted that Ehrlich did not identify any problems with the polls methodology. Having read through the data made available by the Post I can see no obvious methodological problems, but I am left wondering about the Post's turnout assumptions, like some recent polls in California I think the Post may be overstating likely Democratic turnout.

The Post does not provide a breakdown of party ID among likely voters, but polling director Jon Cohen provided that data to me today. Party ID was 50% Dem, 26% Rep, and 23% Ind. The Post data also lacks cross tabs showing the level of support for each candidate by party ID among likely voters, Cohen provided that as well - O'Malley is winning 87% of Dems, 5% of Reps, and 31% of Inds, Ehrlich wins 10% of Dems, 93% of Reps, and 54% of Inds.

This is crucial information and suggests that the Washington Post is overstating Democratic turnout and understating Republican turnout in 2010.  Consider - the 2004 Maryland turnout (a decent Republican year)was 48% Democrat, 30% Republican, and 22% Independent; the 2008 turnout numbers (an incredible Democratic year) were 51% Democrat, 28% Republican and 21% Independent.

So the Post sees a 2010 electorate that every bit as Democrat as was 2008 and less Republican than either the bad year of 2008 or the good year of 2004. In other words, in a year marked by a tremendous enthusiasm gap between Democrats and Republicans the Post thinks that Democrats will rival their 2008 turnout and Republicans will not even be able to reach their's - this is just not a reasonable assumption.

Those differences may seem slight, but based on the cross tab data provided by Jon Cohen it is possible to model the Post's data based on a turnout dynamic like 2004 and 2008. I've done this in Tables One through Four below.

If 2004 turnout numbers are applied to the Post poll (Table One) the results become 50% O'Malley to 45% Ehrlich - a statistical tie in line with all other polls.

Using the 2008 turnout assumptions (Table Four) the results of the poll become 54% O'Malley to 44% Ehrlich - essentially the margin reported by the Post. This leads me to believe that the Post is assuming/modeling a turnout dynamic similar to 2008 - and that seems highly unlikely in the current political environment. This primary season has been marked by historic levels of GOP voter turnout and in key races in VA, NJ, and MA Democratic turnout was well off of the 2008 levels while GOP turnout was steady.

For these reasons I would argue that 2004 is a much more likely scenario - and that would make this a 4 point race and within the margin of error.

The Post poll also found that Ehrlich is winning just 10% of Democrats. That seems low, in the Democratic primary just two weeks ago fully 14% of Democrats voted against O'Malley and for two unknown candidates.  If one assumes that Ehrlich wins 14% of Democrats (a number well below the roughly 22% he won in 2002 and the 15% he won in 2006) the poll results become 51% to 49% (Table Three).

Of course, if Ehrlich did receive 20% of the Democratic vote and the turnout is like 2004 then he's ahead 52% to 48% (Table Five). Even using a 2008 dynamic the race would be tied at 50% to 50% (Table Six).

In short, I suspect that the Washington Post poll actually confirms that the race between Martin O'Malley and Bob Ehrlich is a toss-up. I don't think that the poll is "out of whack" I just think that their turnout assumptions are wrong. Jon Cohen explained to me that the post weights "our adult sample to information we have good reason to believe is true, not an estimate about what a future population (voters might look like." That is of course their right, but it makes far more sense to base turn-out on actual past turnout numbers and not on a simple breakdown of the adult population - in other words a model should be based on actual voters not just all people.

I'd like to thank John Wagner and Jon Cohen for being so willing to speak with me about the poll and for being so accessible.

Stay tuned....