A visual experiment on 200 possible layouts discovering a ‘best’ 10 times better than worst.

The experiment showed how people choose a magazine and has set the magazine cover style since 2010.  Actual results are reproduced below with kind permission of NewsCorp Australia.

The Problem
Gardening Weekly is just one of over thirty magazine titles produced by News Corporation Australia but was failing in the market. In an effort to boost magazine sales, a redesign of the cover was proposed. The designer presented a variety of concepts that included elements from existing layout and website as well as some new ideas. Naturally there were many opinions about “what works” SurveyEngine were hired to help put some data into the discussion.

Initially, NewsCorp wanted to simply market test the handful of concepts produced by the artist but SurveyEngine proposed to test all of the combination of ideas to explain what elements drive a consumer’s choice.

How we did it
The artist produced four general ideas on revitalising the existing design.

MastHead 

The designer suggested the masthead could be solid white as it had been or ‘bled-through. 

Font

 There were three possibilities  – uppercase, lowercase and a cursive script style

Layout

Four options were presented – a side feature layout, two horizontal feature styles as well as ‘no features’

Background

Eight backgrounds were suggested – a number of flowers, a garden, a bowl of berries and a gardener.

These raw ideas, were developed as an experiment within SurveyEngine which reported that there were exactly twenty four test layouts that would be needed for the experiment. This list was given to the artist who produced finished artwork which were attached to the experiment.
The experiment was sent out to 1,000 people appearing as a regular online survey. Once enough data had been collected, the results were automatically modelled.

Results

But what is the best layout?
We can bypass all the mathematics and find the best layout by picking the most positive effects.

The Best

The optimal setting for each of the attributes is a:
see through masthead
simple uppercase font
background of a field of lavender
simple article layout
with a predicted choice 93% compared to worst
The cover on the left was created from the above optimal setting. Before the model, it had never existed and it was never directly tested, it was predicted.

The cover on the left was created synthetically from the above optimal setting. Before the model, it had never existed and it was never directly tested, it was predicted.

 …and the worst

Predicted Choice 7% 
As an exercise, the worst layout was also created. This time all the most negative effects are picked, which are:

  • the solid masthead
  • the script font
  • the picture of a gardener
  • the side layout

with a predicted choice 7% compared to the best
This means the optimal would be chosen between ten and twenty times more often than the worst.