Online Surveys: How long is too long?

Ask nearly anyone involved in online research and they’ll tell you that the ideal survey length is 15 to 20 minutes or less. Anything longer and you invite bad behavior (straightlining, satisficing), drop outs, or create ill-will and lack of respo...

Ask nearly anyone involved in online research and they’ll tell you that the ideal survey length is 15 to 20 minutes or less. Anything longer and you invite bad behavior (straightlining, satisficing), drop outs, or create ill-will and lack of responsiveness the next time around. Some brave souls are even proposing that we deconstruct surveys and create pods of questions and participants and then knit them together on the back end. For example, they suggest that it’s better to have 10 unique groups of 100 people answering 10 questions and then recombine the data on the backend than it is to have 1000 people answering 100 questions.

I get the point, but isn’t that a bit extreme? Instead of minimizing the time spent during a bad experience shouldn’t we instead be focusing on maximizing the time spent in a good experience?

At Invoke Solutions, we’ve been focused on creating a positive participant experience for over ten years. We believe that a positive experience ultimately results in a clear and tangible benefit for researchers: better, more reliable data and insights. The more a participant is enjoying their experience, the more likely they are to be thoughtful about their responses and tell you exactly what they think. They also are more likely to provide open ended responses that are deep, rich, filled with emotional content and expressed with relevant personal context. Happy participants are also more likely to generate higher completion rates and contribution levels and more likely to want to participate the next time. And to the primary point of this blog, they are more likely to stay online with you for much longer periods of time.

As evidence, we recently analyzed over 300 past Invoke studies. For several years now, we have been asking two questions at the end of every one of our studies: 1) How satisfied were you with your experience today and 2), how interested would you be in participating in a future Invoke study. The analysis we just completed revealed that participants in an Invoke study give their experience a 95% satisfaction rating (Top 2 Box on 5 point scale), and 99% of them are interested in participating in another study in the future. We also found that these 300+ Invoke studies achieved a 96% average completion rate and an average contribution level of 96%. AND THESE WERE ONLINE STUDIES THAT WERE 60 to 90 MINUTES IN DURATION!

If I still don’t have your attention, we recently asked 1000 random internet survey participants to tell us how an Invoke session compared to traditional online surveys. 85% of them stated that it was a better experience (Top 2 Box) and half of the sample said it was a MUCH better experience (Top Box). Some of the reasons they gave us for why it was better were related to the moderator driven experience, the fact that they could see and comment on responses of other participants, and the live, interactive nature of the experience.

  • “It’s more engaging. It keeps me more interested, even for longer sessions. I tend to get bored when taking a long survey where you have to move from one screen to the next.”
  • “The questions are less repetitive and the direction of a moderator helps me to feel as though my contributions are really being heard.”
  • “I like the more interactive feel to Invoke sessions. The survey moves along at a good pace and it’s interesting to see how other people answer some of the questions.”
  • “I like the interaction between myself and the moderator, the questions are more in depth and I get to see how other people answer as well.”
  • “This is better because it seems to be more personal and I feel more confident that these answers will be put to good use.”
  • “It allows me to express myself more, and it gives me the opportunity to see how others respond.”

So the next time someone tells you that its not good practice to keep someone online for more than 15 minutes, tell them that may be true with traditional, boring, tedious online surveys, but that you know of a better option. Then of course call us.

If you’re interested in discussing this further or would like to see the data, drop me a note at Nancy.Wellott@invoke.com

Get insights into
the exact things your audience is watching.

By entering your email address, you agree to receive marketing communication in accordance with our privacy policy.