Research Design
Decide whether you are exploring or confirming, define your target population, and avoid common design traps like HARKing.
Before you write a single question, be clear about what kind of study you are running. The biggest threats to credible findings are introduced here, at the design stage, not during analysis.
Exploratory vs Confirmatory Research
Every study leans one of two ways, and conflating them is one of the most common mistakes in applied research.
| Exploratory | Confirmatory | |
|---|---|---|
| Goal | Discover possible relationships and generate hypotheses | Test a specific prediction stated in advance |
| Mindset | Open, flexible, descriptive | Pre-committed, falsifiable |
| Output | Hypotheses worth testing later | Evidence for or against a hypothesis |
| Risk if mislabeled | Treating a chance pattern as a confirmed result | Discarding genuine discoveries because they were not predicted |
Exploratory work is valuable. Problems only arise when an exploratory finding is reported as though it were confirmatory.
HARKing
HARKing stands for Hypothesizing After the Results are Known: running an analysis, finding a pattern, and then writing it up as though you had predicted it all along. It inflates false positives because a hypothesis built to fit the data will almost always fit the data.
If you only formed a hypothesis after seeing the results, say so. Report it as exploratory and, ideally, confirm it in a fresh study before drawing strong conclusions.
The cleanest defense against HARKing is to write your hypotheses down before you collect data. See Preregistration.
Defining Your Target Population
Your target population is the group you want your conclusions to apply to. Define it concretely in terms of observable criteria, because those criteria become your recruitment settings on Terac.
Work through three questions:
- Who are you studying? Age range, location, profession, behavior, or experience level.
- Why those people? Tie each criterion back to your research question. If a criterion does not change who should qualify, drop it.
- How will you reach them? Map each criterion to a filter or a screening question.
Filters narrow who can see your opportunity before they apply. Screening questions qualify people during the application flow. Use filters for hard demographic or professional requirements, and screening questions for anything that needs a judged answer.
Operationalizing Your Constructs
A construct is the abstract thing you care about (for example, "brand loyalty" or "comfort with AI tools"). Operationalizing means turning it into something you can actually measure with concrete items. State, in advance, exactly how each construct maps to the questions you will ask. If two researchers would measure your construct differently, your definition is not specific enough yet.
A Pre-Build Checklist
Before moving on to building your instrument, you should be able to answer:
- Is this study exploratory or confirmatory?
- What is my exact research question or hypothesis?
- Who is my target population, in observable criteria?
- How does each criterion map to a filter or screening question?
- How will I measure each construct?