DBMM Report Card – Results Format

Last week I created an online survey to assess the factors behind the success of DBMM 1.0 and posted some preliminary results based on an analysis of the first 60 responses. Since then things have moved on apace and there are now 118 responses. This is excellent. Thanks everyone.

The survey still has a week to run.  Once closed, I’ll make the raw results public and set about summarising the trends and analysing the data for relationships for posting here.

In the meantime I thought I’d post an interim analysis of one of the questions explaining the approach and terminology I’ll be using to summarise the data.  Feel free to skip this if you want to.

The question I’ve chosen is Q19: Rate you overall experience with DBMM.  Respondents used a five point scale from Excellent (1) through Neutral (3) to Poor (5).  That DBMM has received an overall “Good” rating is obvious from the raw data:



There are two main ways of summarising the 107 responses above: as an average and by quoting percentage scores in the “top two boxes”.

An average is a useful shorthand but really needs to be quoted with a standard deviation to estimate the data spread.  Top box scores can have more impact for the general reader but have to be used with care as they can just as easily hide trends as highlight them.

For Q19 the results can be summarised as follows:

An Average score of 2.34 ± 1.27

• The average respondent gave DBMM a score of 2.34 - somewhere between Good and Neutral.
• The figure of 1.27 is the standard deviation and indicates that 65% of all responses lay between 1.07 (Excellent) and 3.61 (Fair).

A “Top box” score of 70% or 18%

• 70% of all respondents (75 of 107) rated DBMM as either Good or Excellent.
• 18% (19 of 107) rated DBMM as either Fair or Poor (if you look at the bottom two boxes).

In this case both approaches accurately describe the results but the top box score has significantly more impact on the page and if anything slightly overstates the case in favour of DBMM.  This is why I’ll  be quoting both types of summary data in my analysis.

The survey runs until 14th August. If you haven’t tried it yet, and you have five minutes to spare, visit kwiksurveys.com and let everyone know what you think of DBMM 1.0.