Friday, June 23, 2006

On Margins of Error

The Star's blog reports a poll showing Talent up over McCaskill by 5:
An online Zogby Interactive poll for The Wall Street Journal has Jim Talent leading Democrat Claire McCaskill by 5 points in Missouri's U.S. Senate race.

Conducted June 13 thru Monday, the survey showed Talent, the GOP incumbent, with 49 percent to McCaskill's 44 percent.
Which is interesting and puts Talent ahead of his past ratings. But then the Star goes off the rails:

The poll's margin of error was 3.4 percent, meaning that each number could rise or lower by that amount. The race is within the margin, or still too close to call.
The last sentence is true, but the previous one is a blood-soaked mess of innumeracy and incompetence. What it means for the margin of error to be 3.4% is that if you did this poll a lot of times, let's say millions, you'd get a number within 3.4% of the true percentage backing each candidate in about 95% of the cases. And by inference, if we're usually within 3.4% of the true number, the true number ought to be within 3.4% of the estimate we got from the poll.

It does not (and this is important) imply a hard limit on how large the variation could be. By bad luck, in a sample of 800 people, you might happen to get the entire population of Talent supporters in the state. The true support could be less than one percent, but your sample would be 49% favorable. The point is that the odds of that happening are vanishingly small, and the odds of each number in the sample being more than 3.4% away from the true value is pretty small.

And since the margins of error overlap, you can't exclude the possibility that McCaskill is still ahead of Talent, so a better title for the Star would be "Talent and McCaskill are neck-and-neck."