Writing Good Surveys
Phrase questions clearly, order them to minimize bias, choose the right response scales, and lay out your instrument for accurate answers.
The wording, order, and layout of your questions shape the answers you get. Small changes can move results more than the effect you are trying to measure. This page covers how to write items that capture what participants actually think.
Phrasing Questions
Each item should measure one thing, in plain language, without nudging the participant toward an answer.
| Avoid | Why | Better |
|---|---|---|
| Double-barreled: "Was the app fast and easy to use?" | Bundles two things; you cannot tell which one the answer is about | Ask about speed and ease separately |
| Leading: "How much did you enjoy the new feature?" | Presumes enjoyment | "How did you feel about the new feature?" |
| Double negatives: "Do you disagree that it should not be removed?" | Hard to parse, easy to misread | Rephrase positively |
| Loaded or jargon terms | Different people read them differently | Use neutral, common words |
| Vague quantifiers: "often", "sometimes" | Mean different things to different people | Give concrete anchors ("2 to 3 times a week") |
A good rule: if a colleague who has not seen your study could misread an item, rewrite it. Test every item on someone before launching.
Ordering Questions
Earlier questions prime later answers. A few principles:
- Open with easy, non-sensitive questions to build comfort and momentum.
- Group related questions so participants are not switching context constantly.
- Place sensitive questions later, once some trust is established, but before fatigue sets in.
- Watch for carryover: asking about a topic can change how someone answers a related question. If order might matter, randomize question or block order so any effect is spread evenly rather than confounded with content.
Response Scales
Likert Scales
A Likert scale measures agreement or intensity along an ordered set of options, typically five or seven points (for example, "Strongly disagree" to "Strongly agree"). They capture nuance that yes/no items miss.
- Five points are easy to answer; seven capture more gradation.
- Decide deliberately whether to include a neutral midpoint. A midpoint is honest when "neither" is a real position, but it can also be a place for low-effort participants to hide.
- Label every point, not just the ends, so the scale means the same thing to everyone.
- Keep the scale direction and labeling consistent across your survey.
Choosing a Format
Match the response format to the construct: ordered options for intensity, multiple choice for categories, numeric entry for counts. Mismatched formats (for example, forcing a continuous quantity into three buckets) throw away information you cannot recover later.
Sensitive Questions
For topics people may be reluctant to answer honestly:
- Emphasize anonymity and explain how responses are used.
- Normalize the behavior in the phrasing so the question does not feel accusatory.
- Provide an opt-out ("Prefer not to say") rather than forcing an answer.
- For potentially distressing topics, signpost support resources in your debrief.
Visual Design
How a survey looks affects how carefully it is answered:
- Readability first: legible font size, strong color contrast, generous spacing.
- One idea per screen for complex items reduces errors.
- Progress bars help, especially on longer or repetitive studies, by setting expectations and reducing dropout.
- Mobile matters: many participants answer on phones, so check your instrument on a small screen.
Attention and Quality Checks
Well-designed attention checks help you identify low-effort responses, but they must be fair: they should test attention to instructions that matter for the task, not trick participants with ambiguous wording. Build them in deliberately and pilot them. See Avoiding Bias for more, and Screening for how qualification logic works on Terac.