Contact us. We would be
happy to discuss your
business, marketing or
To reach us by phone, call (312) 348-6089
Benjamin Franklin said that “time is money” and as market researchers know, time is the key variable that drives the cost of research.
In quantitative research, the length of a survey—that is, the amount of time it takes a respondent to complete a survey—determines most other “time” variables as well: the professional time it takes to design research and analyze data, the fieldwork time it takes to collect data, and the respondent time that requires compensation through incentives.
If you need help figuring out the ideal survey length that will optimize the value and content of a survey within your budget, please read on, and call us at 312-348-6089 with any additional questions you may have.
The Versta Team
How to Estimate the Length of a Survey
On average it takes 7½ seconds to answer an online survey question. If you keep the questions simple, you can ask eight of them in one minute.
But of course most survey questions are not simple, and few of us can cram eighty questions into a ten minute survey. Most surveys have a mix of simple single choice questions, complex multiple-select questions, grid questions, open-end responses, and numeric answers that require thought and mental calculation. So the key to estimating survey length is knowing the equivalency of question-types vis-à-vis a prototypical short and simple question that takes just 7½ seconds to answer.
Why does all of this matter? Because the cost of a survey, whether you are doing it yourself or hiring an outside provider, is driven largely by time. The cost of designing, programming, testing, fielding, and analyzing data all depend on survey length. Finding respondents who are willing to participate, and the incentives you need to pay them, depend on survey length as well.
More importantly, how useful the data are to management may depend on survey length. Some surveys are too short and critical details go missing, while some surveys are too long with so much data you do not know what to do with it. In addition, how good the data are may depend on survey length. Research shows that data quality declines on surveys that are longer than 20 minutes, so a good rule of thumb is to aim for a survey that takes no more than 15 or 20 minutes to complete.
Here, then, is a method for estimating survey length that really works and that we have validated against hundreds of different types of surveys over the past several years. Start by creating a tally of “points” for the questionnaire, assigning points to each question or set of questions as follows:
Assign one point for each short and simple survey question that can be read quickly and that has a yes/no response option or easy-to-grasp answer scale.
Here are two examples, each of which counts one point:
Have you ever undergone surgery for epilepsy?
What is the highest level of education you have completed?
For questions that are asked in grid format, assign one point for each row of the grid.
For example, the following set of agree/disagree statements counts eight points:
Having a choice of treatments is important to me.
I trust my doctor completely.
Ulerative colitis has ruined my life.
People do not understand how disruptive UC is to daily living.
Having UC is embarrassing.
I sometimes forget to take my medication.
It is inconvenient to take my medication at the right time every day.
It is possible to live a normal life with UC.
For multiple response or multiple-select questions, assign one point for every two response options.
As an example, the following question counts four points:
Which of the following types of financial accounts do you own?
Select all that apply
Assign two points for every response that requires mental calculation.This might include age (many people do a mental calculation based on their year born), or the number of days engaged in specific activities, or allotting percentages or numeric values.
For example, we recommend assigning twelve points to the following question because each answer slot requires a numeric estimate relative to the others, and everything must add to 100%:
How much of your retirement funds are in each of the following types of investments?
For open-end questions, assign three points for every short phrase or sentence that you expect most respondents will type.
For lengthy instructions, or if any of the questions have a lot of words, assign one point for every three sentences the respondent must read.
Include all questions in the running tally, even if some respondents may not see the questions. An exception is if the skip logic specifies two mutually exclusive paths, in which case you should use only the longer path in the tally.
Once you have the tallied the total number of points, divide by the magic number 8 (this is the number of simple questions you can ask in a one-minute online survey). Voilà, you get the number of minutes it will take most survey respondents to complete your online survey.
What about phone surveys? Well, for that you could divide by 5⅓, which is the number of phone questions you can ask in one minute. But most calculators and spreadsheets don’t do fractions very well and 5⅓ is an awkward number to keep in mind. So instead, stay with the point system outlined above. Divide by 8, then multiple by 1.5 (if you can ask 5⅓ questions in one minute, you can ask 8 questions in one and a half minutes). Presto, you get the number of minutes it will take most survey respondents to complete your survey by phone.
Once you get the hang of it, it will take no more than a few minutes to accurately predict survey length for a majority of respondents, which in turn will allow you to estimate the cost, utility, and data quality of a survey.
There is one drawback to this technique, however. It works so well that unless you teach it to your colleagues, they will likely come to you again and again for help in estimating their survey lengths, leaving you less time to focus on turning your data into a story.
PR Professionals Turn to Versta for Survey Expertise Public Relations Tactics has published its October 2011 issue with a full page article from Versta Research entitled “Telling the whole story: When numbers aren’t enough.” The article describes the difficulties of using traditional numeric scales, common in survey research, for data that needs to tell a story.