Soft launch and fieldwork monitoring for transport SP surveys

A soft launch - collecting 5–10% of your sample before opening fully - catches problems early when they can still be fixed.

This article explains how to conduct a soft launch for a transport SP survey, what to monitor during the soft launch, and the decision criteria for opening to the full sample.

Ben White, 07.07.2026

Why soft launches matter

Even after a thorough pilot study, real-world fieldwork can reveal problems that were not apparent in a controlled pilot environment. Respondents from different demographic groups, different geographic regions, or different device types may interact with the survey in unexpected ways. Technical issues - broken links, display errors on specific devices, quota misconfiguration - can affect data quality silently.

A soft launch with 5–10% of the target sample gives you a controlled opportunity to identify and fix these problems before they affect the majority of your data.

What to monitor during a soft launch

Completion rate: if completion rate is substantially lower than expected, there may be a technical issue or the survey may be longer or harder than anticipated. Target completion rates vary by mode and respondent type - online panel general population studies typically achieve 60–80% completion; patient studies with longer surveys may be lower.

Completion time: track average time per choice set. If respondents are completing too quickly (under 10 seconds per task), they may be clicking through without reading. If too slowly (over 3 minutes per task), the cognitive burden may be too high.

Data quality indicators: check consistency check pass rates, item non-response rates, and the distribution of choices across alternatives. Extreme imbalance - one alternative chosen 80%+ of the time - suggests a design problem.

Device and browser breakdown: SurveyEngine's fieldwork dashboard shows the breakdown of completions by device type. If mobile completion rates are lower than expected, there may be a display issue on mobile.

TLDR Quick links

Transport resources

Software documentation

Open a free account

Running a soft launch in SurveyEngine

Step 1: Configure quota limits. In SurveyEngine, set initial quota limits to 5–10% of your target sample for each quota cell. This prevents the survey from filling up before you have reviewed the soft launch data.

Step 2: Monitor the fieldwork dashboard in real time. SurveyEngine's dashboard shows completion rates, completion times, and basic data quality indicators as data comes in. Check the dashboard at least twice daily during the soft launch.

Step 3: Export and check data quality. After reaching your soft launch quota, export the data and run basic quality checks: completion rate, average time per task, consistency check pass rate, and distribution of choices across alternatives.

Step 4: Make any necessary adjustments. If problems are identified, pause the survey, make corrections, and document all changes. If changes to the survey instrument are required, you may need to restart the soft launch with a fresh sample.

Step 5: Open to the full sample. Once you are satisfied that the soft launch data meets quality standards, open the quota limits to allow the full sample to complete.

Worked example - mode choice soft launch

A mode choice study targets 400 commuters in a major city. The soft launch quota is set at 40 respondents (10%). After 40 completions, the fieldwork review finds: completion rate 71% (acceptable), average time per task 52 seconds (acceptable), consistency check pass rate 96% (good), but mobile completion rate 34% vs expected 55%.

Investigation reveals a display error on iOS devices - the choice task table extends beyond the screen width and requires horizontal scrolling. The error is fixed, and a second soft launch of 20 respondents confirms the fix. The full study opens and achieves 72% completion and 94% consistency check pass rate.