Designing screening questions for surveys

Screening questions that reveal their purpose to respondents inflate eligibility rates and contaminate the sample. Two design principles prevent this.

How to design screening questions that identify eligible respondents without telegraphing eligibility criteria - covering non-leading question design and screener placement.

Ben White, 09.07.2026

The screening problem

Most surveys restrict participation to respondents who meet specific eligibility criteria - a diagnosis, a behaviour, a demographic characteristic. Screening questions identify eligible respondents before they enter the main survey. When screening questions reveal what the eligible answer is, respondents who want to participate but do not qualify can simply give the eligible answer.

This is not a theoretical risk. In online panel research, where respondents are accustomed to being screened out and losing the survey incentive, the motivation to give eligible answers to screener questions is real. Studies targeting specific patient populations or professional groups are particularly vulnerable.

The cost of a compromised screener

A compromised screener produces a sample that appears to meet eligibility criteria but does not. WTP estimates from a contaminated health preference sample are not valid for the target patient population. The study cannot be replicated or submitted for regulatory review.

Screener placement also has a direct cost implication. Panel providers charge different rates for early screen-outs (respondents who are excluded in the first few pages) versus late screen-outs. Placing screener questions late in the survey means paying for ineligible respondents to complete a significant portion of the questionnaire before being excluded.

TLDR Quick links

Confirmation of diagnosis in patient preference studies

Incidence rate monitoring during fieldwork

Designing screeners that work

Do not telegraph eligibility criteria. If you are screening for female respondents aged 50+ in California, do not ask gender, age, and location in sequence with only eligible options highlighted. Present the questions in a neutral context with a full range of response options. The eligible respondent gives the eligible answer; the ineligible respondent has no signal about which answer to give.

if you plan to screen respondents according to a self-reported question, it is best to not expose the purpose of the question. So, if you just wanted females from California aged over 50.

So this is better

Than this

Use knowledge-based eligibility questions for clinical populations. Rather than asking 'Have you been diagnosed with X?', ask about specific symptoms, treatments, or clinical procedures that only genuine patients would know. Combine self-reported diagnosis with knowledge-based verification.

Place screener questions at the very beginning of the survey - before any survey content, instructions, or incentive information. This minimises the cost of ineligible respondents and reduces the information available to respondents trying to determine the eligibility criteria.

For complex eligibility criteria, use a sequential screener. Ask the most restrictive criterion first. If 5% of the population meets criterion A and 20% meets criterion B, screen on A first to reduce the number of respondents who need to answer criterion B questions.

Screening for a rare disease study

A patient preference study targets adults with non-metastatic castration-resistant prostate cancer (NM-CRPC) who have received androgen deprivation therapy but not chemotherapy. A leading screener would ask directly about these criteria in sequence.

A non-leading screener presents a broad prostate cancer diagnosis question followed by a treatment history grid covering all major treatment types. Eligible respondents self-select by marking ADT without chemotherapy. Knowledge-based questions about ADT side effects and monitoring procedures provide a second eligibility signal. The combined approach reduces fraudulent eligibility claims relative to direct questioning.