Update - The Washington Post poll has Hogan ahead by 20 points, 58% to 38% with a MoE of +/- 4.5%. Factor in the upper and lower bounds of the MoE (as detailed below) and the Post poll is seeing the same race as are the prior polls in Maryland.
Across all 4 polls, Hogan has an average lead of 18.25 points. I had estimated his lead to be between 16 and 18 points in the original post.
Original Post:
In the span of five weeks, 3 separate polls have been released in the Maryland governors race between Larry Hogan and Ben Jealous. The head to head numbers in each poll looked like this:
Gonzales: Hogan 52% / Jealous 36% - a 16 point gap
Goucher: Hogan 54% / Jealous 32% - a 22 point gap
Mason Dixon: Hogan 52% / Jealous 37% - a 15 point gap
I'm a professor of Political Science and in addition to Public Policy, American, and Maryland Politics I teach statistics, probability, random sampling, and survey design. Before becoming a professor I worked for the state of Maryland and conducted survey research of Maryland residents. One of my studies even helped to convince the Census Bureau to change the way in which they measure the number of folks receiving publicly financed health insurance in their annual Current Population Survey.
After seeing the Mason Dixon poll I took to social media and wrote that the Goucher, Gonzales, and Mason Dixon polls all saw a similar race. That there really wasn't a difference among them. Needless to say, this observation didn't sit too well with folks who don't study polls or probability. So I thought I'd spend a little time explaining why these surveys are seeing essentially the same race.
It all comes down to the margin of error (MoE) and the level of reported confidence in each survey. Because polls rely on a random sample of the population, researchers can never be 100% certain of the results. The MoE tells us that were the poll to be conducted 100 times the observed results would fall with +/- 3.5 of the results reported in the poll in 95 of the polls. The remaining 5 would fall outside that range. The other way to think of this is that researchers are 95% confident that the true results, were they to survey everyone, would fall with the MoE of the results reported from the random sample. A common way of expressing that is to say that we're 95% confident that the true value falls within +/- 3.5% of the reported result.
The first poll, released by Gonzales Research and Marketing found that Larry Hogan led Ben Jealous 52% to 36% - a difference of 16 percentage points in a poll with a margin of error (MoE) of +/-3.5%. Again, were the survey to be conducted 100 times, the results would fall with a range of +/- 3.5% in 95 of the surveys.
Two weeks ago, Goucher College released a poll reporting a Hogan lead of 54% to Jealous' 32% - a gap of 22 percentages points, with an MoE of +/- 4.5%.
This week, Mason Dixon released a poll of the race and found Hogan ahead of Jealous by 52% to 37% - a 15 percentage point gap with an MoE of +/- 4%.
The three polls caused some confusion. Some questioned how the polls could show a lead ranging from 15 points to 22 points. As one would expect, the Jealous campaign pointed to the Mason Dixon poll as evidence that the candidate had gained 7 points on Hogan in the time since the Goucher Poll had come out.
That's good spin, but the truth is a bit more complicated.
The Gonzales Poll had Hogan at 52% and Jealous at 36%. That 3.5% MoE applies to each candidate's reported level of support. So in 95 out of 100 surveys we'd expect the level of support for Hogan to fall within an upper bound of 55.5% and a lower bound of 48.5% (just add and subtract 3.5%) and Jealous would have an upper bound of 39.5% and a lower bound of 32.5%. That means, it's possible the true gap between Hogan and Jealous could be as high as 23 percentage points or as low as 9 percentage points - that's because the MoE for each individual candidate (3.5% in the Gonzales poll) must be doubled when applied to the distance between the candidates, so it becomes an MoE of +/- 7%. Add 7 percentage points to 16% and you get 23%. Subtract 7 percentage points from 16% and you get 9%.
We can easily apply these basic rules to the Goucher and Mason Dixon polls and then see how it is that they are all in the same universe. Goucher had an MoE of +/-4.5% among likely voters. So Hogan's actual possible range was 58.5% at the upper bound to 39.5% at the lower bound. Jealous had a possible range of 36.5% at the upper bound to 27.5% at the lower bound. The MoE for the distance between them would be +/- 9% so that 22 percentage point gap could be as high as 31% or as low as 13%.
Mason Dixon had an MoE of +/- 4% so Hogan's actual possible range was 56% at the upper bound and 48% at the lower bound. Jealous had a possible range of 41% at the upper bound and 33% at the lower bound. The MoE for the distance between them would be +/- 8% so that 15 point gap could be as high as 23% or as low as 7%.
The standards for whether or not poll results are significantly different from one another are high in survey research - typically requiring that 95% level of confidence to declare what we call a statistically significant difference. In order to state that the Gonzales, Goucher, and Mason Dixon polls show statistically significant different results there would need to be no overlap in the results after applying the MoE to get the upper and lower bounds of each poll. The Mason Dixon and Gonzales polls had nearly identical results, so it's easy to see that there wasn't a significant difference. It's the Goucher poll that seems to be confusing people. But consider Goucher's 22 point gap and Mason Dixon's 15 point gap. The upper bound of Mason Dixon is 22 percentage points, the lower bound of Goucher is 13 percentage points. They quite clearly overlap. As a result, we cannot say with 95% certainty that the results are statistically different. Obviously, the ranges of the upper and lower bounds of the polls show that were we to repeat these surveys 100 times, there would be times when the upper and lower bounds would not overlap. Because of that we can say that the results may be different, but we'd need to lower our level of certainty or confidence below 95% to a level not considered to be an acceptable test of statistically significant difference within survey research.
As a result of these basic rules of survey research, the best conclusion is that these 3 polls - spread across the past 5 weeks - are seeing essentially the same race. As a matter of simple politics, it would be wise of the Jealous campaign to tout the Mason Dixon poll as evidence that Hogan's lead is narrowing. Politics is about spin. But as a matter of math, the mathematics of surveys and random sampling, it would be more spin than reality. Based on the results of these 3 polls, and the regions of overlap, I'd place the the likely gap between them in the neighborhood of 16-18 percentage points.
Rumor has it that Gonzales Research and Marketing has a new survey in the field. Once we know those results, we can throw the new numbers into this mix.
Across all 4 polls, Hogan has an average lead of 18.25 points. I had estimated his lead to be between 16 and 18 points in the original post.
Original Post:
In the span of five weeks, 3 separate polls have been released in the Maryland governors race between Larry Hogan and Ben Jealous. The head to head numbers in each poll looked like this:
Gonzales: Hogan 52% / Jealous 36% - a 16 point gap
Goucher: Hogan 54% / Jealous 32% - a 22 point gap
Mason Dixon: Hogan 52% / Jealous 37% - a 15 point gap
I'm a professor of Political Science and in addition to Public Policy, American, and Maryland Politics I teach statistics, probability, random sampling, and survey design. Before becoming a professor I worked for the state of Maryland and conducted survey research of Maryland residents. One of my studies even helped to convince the Census Bureau to change the way in which they measure the number of folks receiving publicly financed health insurance in their annual Current Population Survey.
After seeing the Mason Dixon poll I took to social media and wrote that the Goucher, Gonzales, and Mason Dixon polls all saw a similar race. That there really wasn't a difference among them. Needless to say, this observation didn't sit too well with folks who don't study polls or probability. So I thought I'd spend a little time explaining why these surveys are seeing essentially the same race.
It all comes down to the margin of error (MoE) and the level of reported confidence in each survey. Because polls rely on a random sample of the population, researchers can never be 100% certain of the results. The MoE tells us that were the poll to be conducted 100 times the observed results would fall with +/- 3.5 of the results reported in the poll in 95 of the polls. The remaining 5 would fall outside that range. The other way to think of this is that researchers are 95% confident that the true results, were they to survey everyone, would fall with the MoE of the results reported from the random sample. A common way of expressing that is to say that we're 95% confident that the true value falls within +/- 3.5% of the reported result.
The first poll, released by Gonzales Research and Marketing found that Larry Hogan led Ben Jealous 52% to 36% - a difference of 16 percentage points in a poll with a margin of error (MoE) of +/-3.5%. Again, were the survey to be conducted 100 times, the results would fall with a range of +/- 3.5% in 95 of the surveys.
Two weeks ago, Goucher College released a poll reporting a Hogan lead of 54% to Jealous' 32% - a gap of 22 percentages points, with an MoE of +/- 4.5%.
This week, Mason Dixon released a poll of the race and found Hogan ahead of Jealous by 52% to 37% - a 15 percentage point gap with an MoE of +/- 4%.
The three polls caused some confusion. Some questioned how the polls could show a lead ranging from 15 points to 22 points. As one would expect, the Jealous campaign pointed to the Mason Dixon poll as evidence that the candidate had gained 7 points on Hogan in the time since the Goucher Poll had come out.
That's good spin, but the truth is a bit more complicated.
The Gonzales Poll had Hogan at 52% and Jealous at 36%. That 3.5% MoE applies to each candidate's reported level of support. So in 95 out of 100 surveys we'd expect the level of support for Hogan to fall within an upper bound of 55.5% and a lower bound of 48.5% (just add and subtract 3.5%) and Jealous would have an upper bound of 39.5% and a lower bound of 32.5%. That means, it's possible the true gap between Hogan and Jealous could be as high as 23 percentage points or as low as 9 percentage points - that's because the MoE for each individual candidate (3.5% in the Gonzales poll) must be doubled when applied to the distance between the candidates, so it becomes an MoE of +/- 7%. Add 7 percentage points to 16% and you get 23%. Subtract 7 percentage points from 16% and you get 9%.
We can easily apply these basic rules to the Goucher and Mason Dixon polls and then see how it is that they are all in the same universe. Goucher had an MoE of +/-4.5% among likely voters. So Hogan's actual possible range was 58.5% at the upper bound to 39.5% at the lower bound. Jealous had a possible range of 36.5% at the upper bound to 27.5% at the lower bound. The MoE for the distance between them would be +/- 9% so that 22 percentage point gap could be as high as 31% or as low as 13%.
Mason Dixon had an MoE of +/- 4% so Hogan's actual possible range was 56% at the upper bound and 48% at the lower bound. Jealous had a possible range of 41% at the upper bound and 33% at the lower bound. The MoE for the distance between them would be +/- 8% so that 15 point gap could be as high as 23% or as low as 7%.
The standards for whether or not poll results are significantly different from one another are high in survey research - typically requiring that 95% level of confidence to declare what we call a statistically significant difference. In order to state that the Gonzales, Goucher, and Mason Dixon polls show statistically significant different results there would need to be no overlap in the results after applying the MoE to get the upper and lower bounds of each poll. The Mason Dixon and Gonzales polls had nearly identical results, so it's easy to see that there wasn't a significant difference. It's the Goucher poll that seems to be confusing people. But consider Goucher's 22 point gap and Mason Dixon's 15 point gap. The upper bound of Mason Dixon is 22 percentage points, the lower bound of Goucher is 13 percentage points. They quite clearly overlap. As a result, we cannot say with 95% certainty that the results are statistically different. Obviously, the ranges of the upper and lower bounds of the polls show that were we to repeat these surveys 100 times, there would be times when the upper and lower bounds would not overlap. Because of that we can say that the results may be different, but we'd need to lower our level of certainty or confidence below 95% to a level not considered to be an acceptable test of statistically significant difference within survey research.
As a result of these basic rules of survey research, the best conclusion is that these 3 polls - spread across the past 5 weeks - are seeing essentially the same race. As a matter of simple politics, it would be wise of the Jealous campaign to tout the Mason Dixon poll as evidence that Hogan's lead is narrowing. Politics is about spin. But as a matter of math, the mathematics of surveys and random sampling, it would be more spin than reality. Based on the results of these 3 polls, and the regions of overlap, I'd place the the likely gap between them in the neighborhood of 16-18 percentage points.
Rumor has it that Gonzales Research and Marketing has a new survey in the field. Once we know those results, we can throw the new numbers into this mix.
