Friday, June 02, 2006

Writing the right poll question

I think I made it clear in my post on the Eagle's survey that I didn't like the way the evolution question was phrased, and RSR's response makes a valid point:
We think the question -- Would you be more likely to support a candidate who supports teaching only the theory of evolution? Or a candidate who supports teaching alternative theories to evolution? -- doesn't fully capture what voters are thinking about the current school board and its right-wing majority.

We would have preferred a series of questions that probe the issue more deeply. We think voters have a more nuanced view than this particular question is capable of getting at.
So, collective wisdom of the blogocracy, what would be a better question/set of questions. And if we get a good set of questions, can we get someone to actually run the poll?

Here are my proposed questions, in no particular order, followed by points to ponder in writing your own.

  1. (Open-ended) Can you name any scientific alternative theories to evolution?
  2. Do you think it is possible to scientifically test the activities of God or other deities, or are such proposals inherently unscientific?
  3. Would you be more likely to support a candidate who supports only teaching accepted scientific theories in science class? Or a candidate who favors teaching unscientific ideas alongside scientific theories?
  4. Which of the following do you think is the best scientific explanation of human origins: evolution from a pre-hominid ancestor, evolution guided by a divine power, direct creation by a divine power, or are you not sure? (this question might follow something like what Harris asks: "Which of the following do you believe about how human beings came to be? Human beings evolved from earlier species. Human beings were created directly by God. Human beings are so complex that they required a powerful force or intelligent being to help create them.")
  5. Which of the following accounts of human origins (or you could ask about the diversity of life) should be taught in science class: evolution by natural processes, evolution guided by a divine power, direct creation by a divine power, some combination of these, or are you not sure?
  6. Should discussion of the role of God or other deities in the universe be the responsibility of a student's parents and spiritual advisors or should the student's teachers also discuss that topic?
  7. Should a student's science teacher discuss that topic or should that conversation happen elsewhere?
  8. Should a student's history teacher discuss that topic or should that conversation happen elsewhere?
  9. Should a student's social studies teacher discuss that topic or should that conversation happen elsewhere?
  10. Should a student's English teacher discuss that topic or should that conversation happen elsewhere?
  11. Would you support an elective comparative religion class in high schools?
  12. Would you support a required comparative religion class in high schools?
  13. Should a teacher of such a course discuss the role of God or other deities in the universe or should that conversation happen elsewhere?
The point I'm trying to get at is that poll questions about evolution ought to disentangle people's natural and praiseworthy desire for all sides of an issue to be presented from the issue that not all of the alternatives being offered are actually scientific, and presenting non-science as science is dishonest.

The question Harris asks about human origins (cited above in question 4) reads:
"Which of the following do you believe about how human beings came to be? Human beings evolved from earlier species. Human beings were created directly by God. Human beings are so complex that they required a powerful force or intelligent being to help create them."
Only 22% think humans evolved from earlier species, a whopping 64% chose that humans were created directly by God, and 10 think a powerful force or intelligent being helped to create them.

The problem is, theistic evolutionists could honestly choose any of the three answers. Theistic evolutionists do hold, after all, that God created humanity, but used evolution to do so. They regard God's action as a theologically important point, but scientifically untestable. Asking what they believe confuses the issue. I expect that if people were asked what they felt the best scientific explanation was, you'd get noticeably different results.

Questions 6-13 are meant to dig into another question often asked, and the only issue that really matters as far as school boards. Harris asks:

"Regardless of what you may personally believe, which of these do you believe should be taught in public schools? Evolution only. [READ IF NECESSARY: Evolution says that human beings evolved from earlier stages of animals.] Creationism only. [READ IF NECESSARY: Creationism says that human beings were created directly by God.] Intelligent design only. [READ IF NECESSARY: Intelligent design says that human beings are so complex that they required a powerful force or intelligent being to help create them.] All three."
A majority – 55% – chose that public schools should teach all three, while 27% chose creationism (including ID) only and 12% chose evolution only. But even if creationism ought to be taught, people may not want it taught in science class. It's a meaningful component of 20th century American history, there are relevant philosophical, historical and comparative religious discussions to be formed around the topic of the creationism battles, and I'd expect that many people who think creationism should be presented in public schools might well prefer to see it taught outside of science classes.

In thinking about other interesting questions, try to think about avoiding open-ended questions (question 1, for instance). Those questions tend to give fewer useful results as far as gauging opinion, but can give a sense of whether something important is being missed entirely. Also think about balance. I've tried to turn yes/no questions into either/or questions, and to keep the options roughly parallel.

I admit that I did violate parallelism in one question. Question 3 asks "Would you be more likely to support a candidate who supports only teaching accepted scientific theories in science class? Or a candidate who favors teaching unscientific ideas alongside scientific theories?" I'm asking people to compare scientific and unscientific and also to compare "ideas" to "theories." I don't know how to keep the question accurate while establishing parallelism, but this question would probably give people misleading results.

Another good topic to address would be multiple choice style questions to gauge the public's appreciation of what exactly people know about the evidence for evolution, the evidence (or lack thereof) for "alternatives," and the actual content of a high school class which discusses evolution. Questions about people's views on the social implications of evolution would be very interesting.

Another topic that's always intrigued me is the much higher support for creationism among women. A while back I discussed that with some pollsters at the Pew Center, who said that religion and educational status did not explain this correlation. Questions that dig into this topic would be fascinating.

Please leave your questions and suggestions\ in the comments.