Scope insensitivity as a data quality indicator in environmental DCEs

Scope insensitivity is not just a validity threat - it is a data quality signal that can identify subgroups of respondents whose choices are not driven by genuine preferences.

This article explains how scope insensitivity can be used as a data quality indicator in environmental DCEs, identifying respondents who are not genuinely engaging with the choice tasks.

Ben White, 07.07.2026

Scope insensitivity as a quality signal

Scope sensitivity - WTP increasing with the scope of environmental improvement - is both a validity criterion for the study as a whole and a data quality signal at the respondent level. A respondent who states the same WTP for a 10% and a 60% improvement in water quality is almost certainly not engaging genuinely with the choice tasks.

This respondent-level scope insensitivity is distinct from study-level scope insensitivity. Even if the aggregate WTP estimates are scope-sensitive, individual respondents who show scope insensitivity may be contaminating the distributional estimates - particularly for latent class models that estimate preference heterogeneity.

Why scope insensitivity correlates with other quality issues

Research on scope insensitivity shows it is highly correlated with other indicators of low engagement: short completion times, status quo bias (possibly protest-related), and warm glow motivation. Respondents who show scope insensitivity are more likely to also show other data quality problems.

This correlation allows scope insensitivity to be used as a composite quality indicator alongside response time and protest response measures. Respondents flagged by multiple indicators - scope insensitivity, short completion time, and protest motivation - have a much higher probability of producing invalid data than respondents flagged by any single indicator.

SurveyEngine's environmental DCE template includes scope sensitivity check attributes and debriefing questions, providing the data needed for respondent-level scope sensitivity analysis.

TLDR Quick links

Environment resources

Scope sensitivity testing article

Open a free account

Analysing scope sensitivity at the respondent level

Step 1: Include scope sensitivity check attributes in your design. Design two versions of a key attribute at very different levels - for example, water quality improvement of 10% and 60% - and ensure both appear in the choice sets each respondent sees.

Step 2: Calculate respondent-level implied WTP for each scope level. For each respondent, use their SP choices to estimate an implied WTP for the small improvement and the large improvement. This can be done using a simple regression on the individual respondent's choices.

Step 3: Flag scope-insensitive respondents. Flag respondents whose implied WTP for the large improvement is not significantly greater than for the small improvement. The threshold should be pre-specified.

Step 4: Cross-reference with other quality indicators. Combine the scope insensitivity flag with response time, protest, and consistency check flags. Respondents flagged by multiple indicators are strong candidates for exclusion.

Step 5: Report scope sensitivity results at both aggregate and individual levels. In the methods section, report the aggregate scope sensitivity test result and the proportion of respondents flagged as scope-insensitive.

Worked example - river quality scope sensitivity analysis

An environmental DCE includes two scope versions of the water quality improvement attribute: 15% and 55% improvement. Respondent-level WTP analysis identifies 18% of respondents as scope-insensitive (implied WTP for 55% improvement within ±10% of implied WTP for 15% improvement).

Cross-referencing with other quality indicators: 64% of scope-insensitive respondents are also flagged as short completers; 41% gave protest debriefing responses; and 28% chose the status quo in 7+ tasks. Only 12% of scope-insensitive respondents show no other quality issues. The study excludes respondents flagged by 2+ quality indicators (including scope insensitivity) in the main analysis and presents full-sample results in sensitivity analysis.