Detecting speeders and straightliners in DCE data

Detecting straightliners and speeders in Survey data

Straightliners and Speeder are a common forms of inattentive responding research. Detecting them early prevents contaminated data from entering your analysis.

This article explains how to detect in regular surveys and DCE and how to handle flagged respondents.

Knowledge Base -> Data Quality -> Methods & Academic

What are straightliners?

Along with speeders, straightliners are the among most common forms of inattentive responding research.

A straightliner is a respondent who selects the same alternative in every choice set - always Alternative 1, always Alternative A, always the cheapest option. This response pattern indicates the respondent is applying a simple non-compensatory rule rather than making genuine trade-offs between attributes.

In the same way Speeders, who are often the same - employ efficient heuristisc to "just get through" the survey.

They are essentially the same class of respondent. Bored, disengaged or fraudsters.

Why they matter and why DCE's can pick them up

While adhoc analysis may raise suspicions in standard ranking and rating scale, DCE's can be relatively immune from their effects; and can measure them.

In an unlabelled experiment - where the alternatives have no intrinsic difference, only the attributes, modelling the alternative will reveal any systemic bias - usually left.

However - an alternative specific experiment may suffer, with larger estimates for the lexical preference, unless this is controlled for through randomisation.

A major benefit of DCE's is that random effects end up in the error term - preserving the 'signal' of tradeoffs.


Detecting straightliners (and the inattentive)

Unless you have the automated SurveyEngine Quality Report - you'll have to do this the hard way - which is to correlate time spent per page against actual time.

Step 1: Export response time data for each page. Your survey tool should record the time spent on each survey page and each choice task. Export this data alongside the choice responses. This is the 'Actual' time per page

Step 2: Export the word counts for each page, or manually calculate it. Then apply a reading speed appropriate to your audience (200 wpm is typical). and generate an 'Expected' time per page. Alternately use a known 'good' subsample to develop an Expected representative data set

Step 3: Correlate the two - with side by side page time profiles and scatter plots. An R squared of less than 0.3 indicates problems with your study overall.

Step 4: Perform the same analysis per respondent to discover the individuals who's page time profiles bear no relation to the content presented. These should be excluded from any data.

Worked example - speeder detection in an environmental DCE

An environmental DCE with 8 choice tasks produces response time data with a median time per task of 48 seconds. The speeder threshold is set at 16 seconds (one-third of median). 42 respondents (8.4%) fall below this threshold.

Straightliner analysis identifies 19 respondents (3.8%) who selected the same alternative position in 7 or 8 of 8 tasks. 11 respondents are flagged by both criteria. Total flagged: 50 respondents (10%).

Models estimated with and without flagged respondents produce WTP estimates within 8% of each other, confirming robustness. The sensitivity analysis is reported in the supplementary materials.


References


Want to implement response quality monitoring in your DCE? Log in to SurveyEngine to access response time data.

Monthly. Annual. Procurement Friendly.
No obligation · 30-minute call

Or Contact us at support@surveyengine.com — we're glad to help.


Related articles


Scroll to Top