Choice experiment tutorial
– everything you need.

Detailed Guide

(this guide follows the video above)

In this tutorial you’ll learn about building choice experiments in SurveyEngine. We’ll go through each of the major classes of choice experiments from unlabelled, opt out and labelled experiments.
In a separate video we’ll show status-quo and pivot designs and Best-Worst

Its important to understand the quirks of setting up each of these major types of experiments and presenting them to respondents.

Setup a Skeleton Survey

We’ve covered in depth building questionnaires in a separate video – where we skimmed over the Choice Experiment. Today we’ll skim over the questionnaire building and dive into the practical detail of choice experimeent construction.

We first create a skeleton survey and leave a few empty pages for the experiment scenarios.
In this example we’ll focus on transport route choice and show how the different forms of choice experiments may be used.

The Experiment Editor

When you start with the experiment editor you’ll have our blank interface. create an experiment and give it a name. We’ll start with an unlabelled route choice experiment.

Some orientation. The experiment editor has four tabs up the top. Specification, Layout, Design and Advanced.

Specification – this is where all the alternatives, attributes and levels and choices are specified – this is sometimes known as the X-Matrix.

Then a tab devoted to Layout of the experiment
Then the Experiment Design for generating the experimental plan
and finally an Advanced tab for directly editing the experiment XML which we wont need to go into.

Specifying an Experiment

With an unlabelled experiment, all alternatives have the same attribute (and level) set.

Lets enter attributes for route choice attributes for travel by car with the following attributes

• travel time: 20min, 40 mins, 1 hour.
• congestion: no congestion, infrequent congestion, frequent
• toll costs: none, $2.00, $5.00 and $10

We’ll remove one of the alternaives so we have just two alternatives.

For each alternative there is a generic tick box. This tells the software that the attributes is generic and should use the same levels as in the first alternative. We’ll leave this as is but come back to it.

Under settings we can change the text for the choice question but leave the rest as is.

Layout of an Experiment

The default layout is the standard table. Before formatting the experiment. We can put random levels in the layout too see what it looks like. This is useful too see how levels of different lenghts work withthe rest of the layout.

You can also see what it looks like in small devices which is a consideration that sholdn’t be overlooked with more than 50% of web access via small devices.

On the right are options for controlling the layout ranging from simple to complex.

Structure – there’s either table layout or a more free form ‘boxed layout – which has its own options. The options for the table layout are

• alternative headers – unchecking hides the column label
• inner row borders – unchecking simplifies the look as does column borders
• row padding – unchecking removes the extra space padding which is useful when there’s a lot of content
• alternatiing row colors – unchecking removes the alternating colors
• choice labels – unchecking hides the alternative labels.
• Attribute position – either left, uinline or hidden – this is useful when you have many alternatives
• Alterntative widths – allows you to give more space to the Alternates or the attributes
• Choice question posititioning – above, below or hidden

Alternatively you can use the box layout.

This gives more fine grained control over positioning, sizing, colours, flow, concatenation, bullets etc. Also when we get to best worst and multiple alternatives gives a better user experience when there are multiple choice questions.

With come expertise you can specifiy CSS rules – for example the following CSS rule would change the attributes text to large and red.

.attribute {
font-size : 200%;
color :white;
background-color :red!important}

For ultra fine grained control – you can directly edit the html and move the atributes around. You just need to make sure you keep the attribute and alternative names intact.

We can of course reset everything to its default.

Experimental Designs

Experimental design is a critical aspect of choice experiments. An experimental design outlays the plan – what is shown in the experiment. This will be skimmed over here as it is covered in detail in other tutorials.

The easiest way to generate a design is to just click Generate Orthogonal.

This will produce an orthogonal design with some default properties. For unlabelled designs – only the first alternative is orthogonal – the subsequent alternatives are offset from the first – so some care needs to be taken with more than two alternatives becuase if the number of levels is less than alternatives its possible for alternatives to be identical. You should read up on sequential and simultaneous designs.

The second point about these designs is that it uses a ‘nearest fit’ algorithm such that the next largest design iis used and the additional levels may be padded, meaning its possible to get unbalanced designs.

An alternative is to use Ngene to generate designs – with this button – but we’ll cover that in another video.

Previewing the Experiment

You can now go back to the survey builder and replace the pages with the experiment scenarios and preview it to see how it fits into the survey.

Creating an Opt-Out Experiment

This is nearly identical to an unlabelled experiment. This is simply a matter or creating a third alternative – naming it Neither, then setting all the levels to empty.

We do this by unchecking the generic option and removing the unneeded levels.

We don’t need to regenerate the design but its good practise to always do so when you’ve changed your experiment spec. Checking the layout you can see the third option is available and as before you can place it in your pages and preview it.

Creating a Labelled Experiment

A labelled experiment, places labels on the alternatives. This introduces another piece of information to the respondent with each alternative having additional meaning and therefure utility. It also allows us to make the attributes specific to each alternatve allowing for a less unconstrained and more realistic experiment.

In this example we’ll use roadways as the alternatives.

Now we make 3 alternatives

• Suburban back streets
• Main road
• Autobahn

We’ll also use different ranges for time for each roadway. Longer rangers for back streets and shorter for autobahn.

It makes sense to also remove toll for backstreets and main roads.

You will need to regenerate the experimental design as the experiment structure has changed.

Reviewing the layout, you cansee the toll doesn’t change now for backstreets and main road and the travel ranges are more reaistic.As before you can place the experiment scenarios in your survey and see what they look like

Blocking and Combining Experiments

You can put multiple scenarios, even from other experiments, on the same page. While this is probably not a good idea for complex experiments, it may be a good idea perhaps if one experiment is an information condition followed by a context specific choice.

You can also specific the design row – either explicily or as a computed expression. This is useful for custom blocking schemes or when explicit rows need to be displayed – such as dominant or consistncy tests.

For more detail on using the SurveyEngine platform, visit the documentation pages.