Variable direction consistency in survey design

Mixing the direction of response scales across variables confuses respondents and analysts. Agreeing direction before coding prevents a systematic error that is invisible in the data.

Why consistent response scale direction matters, how mixing direction biases responses and complicates analysis, and how to establish and enforce a direction convention before coding starts.

Ben White, 09.07.2026

What variable direction means and why it matters

Variable direction refers to which end of a response scale represents the highest value. If 'satisfaction' is coded 1=very dissatisfied, 5=very satisfied, and 'price sensitivity' is coded 1=very sensitive, 5=not sensitive, the two variables have opposite directions. A respondent who is highly satisfied and highly price sensitive will have high scores on both - which is consistent. But a respondent who is highly satisfied and not price sensitive will have a high score on one and a low score on the other - which is inconsistent with the labelling.

Inconsistent direction causes two problems. First, respondents who read the scale labels carefully and those who anchor on position (always choosing '1' for the 'worst' option) will give systematically different responses to the same underlying attitude. Second, analysts who do not check direction before computing composite scores or running regressions will produce results that mix the two scales without realising it.

Direction errors are silent

Unlike missing data or out-of-range values, direction inconsistencies produce no error messages. The data looks clean. The problem only becomes apparent when results are counter-intuitive - when higher satisfaction is associated with worse outcomes, for example - and by then the data has often been shared with clients.

Agreeing direction before coding costs nothing. Reversing direction in analysis requires re-coding every affected variable and re-running every affected analysis - and the risk of missing one is real.

TLDR Quick links

Variable and survey naming conventions

Cognitive interviewing for DCE survey testing

Establishing a direction convention

Agree before coding starts: for all rating scales, which end represents the highest value? The two common conventions are 'high number = positive outcome' (1=worst, 5=best) and 'high number = more of the attribute' (1=none, 5=a lot). Either works - the critical thing is consistency across all variables in the survey.

Document the convention in the data dictionary alongside each variable. Note any intentional exceptions - some established scales have fixed directions that cannot be changed (EQ-5D, PHQ-9, etc.) and these must be flagged for reverse-coding in analysis.

Check direction during cognitive interviewing. Ask respondents to explain which end of the scale represents 'better' or 'more'. Misunderstandings about direction are common and often not apparent from the response data alone.

For DCE attribute levels, direction applies to the ordering of levels. If 'wait time' levels are 1 week, 2 weeks, 4 weeks, the analyst will expect the coefficient to be negative (longer wait = lower utility). Confirm this expectation before finalising the design.

Direction error in a patient preference study

A patient preference study includes six symptom severity items, five of which use a 1=none to 5=severe scale. The sixth item - 'overall health' - is copied from a published scale where 1=excellent, 5=poor. The inconsistency is not caught in internal review.

In analysis, the composite symptom score includes the reversed overall health item. Patients with severe symptoms appear to have lower composite scores than patients with mild symptoms. The error is identified when a co-investigator queries a counter-intuitive regression result. Re-coding and re-running the analysis takes three hours. The inconsistency would have taken three minutes to catch before coding.