Inattentive respondents in health preference DCE studies

Inattentive patient respondents bias WTP estimates in ways that are invisible to standard model fit statistics - and can invalidate regulatory submissions.

This article explains how to detect inattentive respondents in health preference DCE studies, the thresholds used by leading researchers, and how to report exclusions transparently.

Ben White, 07.07.2026

Inattention in patient populations

Patient populations present specific challenges for response quality monitoring. Patients may have genuine cognitive or fatigue-related reasons for completing tasks quickly - a cancer patient experiencing cognitive effects of treatment may genuinely process information more slowly, but also more quickly in terms of survey engagement. The cognitive burden of a DCE may be too high for some patient populations.

At the same time, health preference studies are targeted by the same panel fraud rings that affect all online research. The stakes are higher: inattentive or fraudulent responses in a patient preference study intended to support an FDA submission can invalidate the evidence.

Why inattention detection is more important in health research

Regulators reviewing patient preference studies scrutinise data quality more closely than academic reviewers. The FDA's patient preference guidance explicitly expects researchers to describe data quality controls and report the proportion of respondents excluded for quality reasons.

ISPOR's Good Research Practices for conjoint analysis in health (Bridges et al., 2011) specifies data quality checks as a required element of health preference study reporting. Studies that do not report inattention checks are increasingly rejected by peer-reviewed journals.

SurveyEngine's health preference study template includes a pre-configured suite of data quality checks: consistency check (dominant alternative), response time monitoring, speeder detection, and straightliner detection. These checks are documented in the study metadata for inclusion in regulatory submissions.

TLDR Quick links

Fraud detection toolkit

SurveyEngine health resources

Open a free account

Implementing and reporting inattention checks in SurveyEngine

Step 1: Configure the consistency check. Embed a dominant alternative at choice set 4–6 of your design (non-prominent position). Flag respondents who do not select it.

Step 2: Set response time thresholds for your patient population. For patient populations, use a more conservative speeder threshold than general population studies - set the minimum time per task at one-quarter of the median rather than one-third, to account for genuine cognitive slowness in some patients.

Step 3: Apply data quality flags in sequence. First exclude speeders (response time below threshold), then straightliners (same position in 80%+ of tasks), then consistency check failures. Document the number excluded at each stage.

Step 4: Run sensitivity analyses. Estimate utility models with and without each excluded group. Report the WTP estimates and their confidence intervals for the main analysis and each sensitivity scenario.

Step 5: Report transparently. In your methods section, describe all data quality checks applied, the threshold used for each, and the number of respondents excluded at each stage. This transparency is required for regulatory submissions and peer-reviewed publication.

Worked example - oncology patient preference data quality

An oncology patient preference study with 320 recruited patients applies four data quality checks: completion time below 5 seconds per task (n=14 excluded, 4.4%); same alternative position in 7+ of 8 tasks (n=8, 2.5%); consistency check failure (n=22, 6.9%); and open-ended response flagged as copy-paste or nonsense (n=6, 1.9%). Total exclusions: 47 respondents (14.7%), leaving a final analysis sample of 273.

Sensitivity analysis comparing main results (n=273) with the full sample (n=320) shows WTP estimates within 11% for all attributes, confirming that exclusions do not materially affect the conclusions. Both sets of estimates are reported in the regulatory submission.