Testing attribute levels in transport SP surveys

Attribute levels that are too narrow produce insufficient variation; levels that feel unrealistic disengage respondents. Testing levels before the main study prevents wasted data.

This article explains how to test attribute levels in transport SP surveys, including the range testing approach, respondent realism checks, and how to use pilot data to calibrate levels.

Ben White, 07.07.2026

The attribute level calibration problem

In transport SP research, attribute levels must satisfy two competing requirements: they must span enough range to identify utility parameters with statistical precision, and they must feel realistic to respondents making genuine trade-off decisions. Levels that are too narrow (±5% around the current level) give insufficient variation for estimation. Levels that are too wide (10 minutes vs 3 hours for a commute) produce hypothetical choices that do not reflect real decision-making.

Why level calibration matters for VoTT

Value of travel time estimates are highly sensitive to the range of time and cost levels used in the SP design. Studies that use wider time ranges tend to produce lower VoTT estimates because respondents are less willing to pay when time savings are very large. Studies with narrow ranges produce more credible but less precise estimates.

The pivot design approach - anchoring levels to each respondent's actual trip - solves part of the problem by ensuring levels are always realistic for that respondent. But even pivot designs require careful calibration of the offset percentages to ensure sufficient variation for estimation.

TLDR Quick links

Transport resources

Pilot study design article

Open a free account

Testing attribute levels in SurveyEngine

Step 1: Conduct a quick realism check. Before the pilot study, conduct 5–10 short interviews with target respondents showing them the proposed level ranges and asking: 'Do these seem like realistic options for a journey like yours?' Note which levels are flagged as unrealistic.

Step 2: Use the pilot to test level discrimination. In the pilot study, check whether respondents consistently choose one alternative regardless of the attribute levels - this suggests levels are not discriminating. If 80%+ of respondents always choose Alternative A, the levels may not be creating genuine trade-offs.

Step 3: Calculate attribute overlap. In the pilot data, check whether the distribution of choices varies appropriately across the levels of each attribute. If removing a cost level from the analysis does not change the model fit, that level may not be discriminating.

Step 4: Adjust levels for the main study. Based on pilot results, widen levels that are not discriminating (too narrow) and narrow levels that respondents flag as unrealistic (too wide). Document all changes between pilot and main study.

Step 5: Re-check D-efficiency after adjustment. Any change to attribute levels requires regenerating the design and recalculating D-efficiency. Use SurveyEngine's design generator to check that the adjusted design maintains adequate efficiency.

Worked example - bus service quality SP survey

A bus service quality SP survey initially uses frequency levels of 5, 10, and 20 minutes and fare levels of £1.50, £2.00, £2.50. Pilot testing with 40 respondents shows that 76% always choose the 5-minute frequency option regardless of other attributes - the frequency range is too narrow relative to its importance, making the other attributes irrelevant to choice.

The frequency range is expanded to 5, 15, 30 minutes. In the revised pilot (n=40), frequency dominates less (58% choose 5 min always), and the fare and reliability attributes now show significant parameter estimates. The revised design produces a coherent set of preliminary VoTT and value of reliability estimates.