How to Write Good Multiple Choice Poll Questions

The most common mistakes in multiple choice questions

Leading questions, ambiguous options, unequal option lengths, and overlapping choices are the most common errors in multiple choice poll questions. Each one compromises the quality of the data you collect. A question that seems clear to the writer can produce inconsistent or uninterpretable results because different respondents read the options differently.

The good news is that these errors are avoidable with a simple checklist. Running your questions through a review process before deploying them — even just a quick self-review — catches most problems before they affect your data.

Clarity and neutrality in question stems

The question stem should contain exactly one idea. "Which of the following best describes your experience with X and would you recommend it?" contains two ideas and will produce inconsistent responses because different respondents will answer one or the other. Separate them into two questions.

Avoid emotionally loaded language in question stems. "Which of these harmful approaches..." primes respondents toward the answer you expect rather than the answer they actually hold. "Which of these approaches..." is neutral; "harmful" is leading. Apply the same neutrality test to your answer options.

Writing good answer options

Mutually exclusive options don't overlap. If a respondent could reasonably choose two options for the same reason, the options aren't mutually exclusive. "Less than 1 hour" and "30–60 minutes" overlap, which will produce ambiguous results. "Less than 30 minutes" and "30–60 minutes" don't overlap.

Exhaustive options cover all realistic answers. If you use "always," "sometimes," and "rarely," respondents who never do the thing have no appropriate option and will either skip or choose incorrectly. Add "never." Better yet, define the frequency ranges numerically so there's no ambiguity about what "sometimes" means.

Testing your questions before going live

Read each option and ask whether any two options are indistinguishable in practice. Compare option lengths — if one option is dramatically longer than the others, respondents will associate length with correctness. Verify that the question makes sense without the answer options (many multiple choice questions only make sense in context of the options, which indicates the stem is too narrow).

When you create a poll on rifts.to, take 60 seconds to answer your own questions as if you were a respondent who didn't write them. Often this exercise reveals ambiguities that weren't visible from the writer's perspective.