Research Works Best When Respondents Can Respond!

I just participated in the most poorly-conducted online research study I've ever seen... and to top things off, the system made me wait nearly two minutes between me hitting the answer button and showing me the next question.  Finally it stopped working altogether.  That bad experience had prompted me to write that post, because online research only works when peopel can answer your questions.

Online research has become a staple in our business and, under the right conditions, can even be used to mimic focus groups.

When being used for qualitative research, respondents participate in interactive, moderated sessions.

Typically respondents are shown a video clip of a commercial or new product. They are then asked a series of typical quantitative survey type questions related to what they saw as well as open-ended questions.

The moderator can send a note to individual participants and ask for clarification from any of the people taking part.

Respondents also get to indicate whether or not they agree with the other participants’ comments by use of “agree” and “disagree” buttons.

Advantages of this Method

• You get people to self-select their participation – and can then qualify them using a series of questions upfront. This can significantly reduce recruiting costs.
• It’s quick and relatively easy to administer.
• Respondent fees are replaced with contest entries and points – again a substantial costs savings to clients.
• You avoid costs related to focus room use, food, taping and transcribing, etc.
• All comments are automatically entered making for easy tabulation, sorting and analysis.
• You can get input from people living anywhere without having to incur travel related expenses
• It’s much easier to get rural area respondents.

Disadvantages of this Method

• People self-select their participation, which can introduce a bias into the research, even if you qualify their participation.
• It’s much harder to probe the reasons “why” people believe something in an online setting, so it is almost impossible to get the same depth of understanding that is possible in face-to-face focus groups.
• Although people can see and comment on one another’s answers, online sessions miss out on points that are often raised when groups build off one another’s answers. Group dynamics can play a powerful role in helping get to heart of important matters.
• Not all respondents complete these interactive sessions. The degree of compliance is influenced by the topic (and the respondents interest in it), the length of the session... and how well the technology works. If respondents can’t answer the questions, it’s a waste of client’s time and money. An inability to complete answers not only causes respondents to shut the window, it can severely impact the likelihood of the respondent being willing to take part in future surveys.

A Couple of Things to Remember

• Ensure that your respondent can indeed see the video by first showing a couple of mini test videos and asking questions about the two images and sound tracks. This lets you know that the person will be able to see the research videos as intended.
• Provide downloads / plug-ins for the tools needed to view your videos and interact with your discussion forum and/or session moderator.
• Limit the participation time to a maximum of 15 minutes (or less).
• Offer participation rewards that appear generous in relation to the time investment required of the respondent – and yes, you always let the person know this upfront.
• Test regularly to make sure that all technology-related tools are working properly.
• Capture participants’’ e-mails addresses so that you can contact people who churn off part-way through to a) determine why and b) request they complete the survey (much easier than locating another person who fits the desired profile).

My next post will provide some of the results of research conducted to help determine what women want from men and their relationships with them. In the meantime, if you have any research-related questions, please feel free to drop me a line. JMC

Choosing the Right Research Company

Well, after having been asked the same three questions three times in almost as many days, I figured others might also be asking, “How do I choose the right research company?” and “How can I tell whose proposal makes the most sense?” and How much should I spend?”

1) Choosing the right research company

It goes without saying that you start by interviewing companies with experience... and ones that have worked for companies you recognize and respect. Call a few of their clients to find out what kind of work was done by your candidates (quality, type, etc.) and how well the data was analysed.

Once you have chosen the companies to add to your short list, you’ll need to brief the prospective agencies. The right firm will ask you questions before giving you a proposal. Lots of questions. The right company will also seek to understand the business challenge at hand before they make any recommendations about how to conduct the research project at hand.

At a minimum, the people preparing the proposal should want to know...

• What is the key issue to be addressed / problem to be solved?
• How/ where does it fit within the corporate vision and other issues under consideration?
• What needs to be learned...and why?
• Who do you believe is best to give you this info? (The researchers may make other suggestions, but they should gain an understanding of who you think is best-positioned to give you the needed details.)
• How you are going to use the information – specifically?
• What research do you already have on the topic? (Yes; they’ll want to see it)


2) “How can I tell whose proposal makes the most sense?”

When the company presents its proposal, the researchers should be able to explain clearly why they have chosen the routes proposed – and the answers should feel right.

You also want a company that respects a budget... and one that will recommend what is needed to get the desired level of reliability and confidence in the data, without going overboard.


3) How much will I need to spend?

That’s a little like asking “How long is a piece of string?” The short answer: That all depends on the size of the company, the size of the problem/ challenge and the impact the decision being made will have on the company.

Before you embark on a research project, you need to assess how the resulting decisions will affect the company’s bottom line. If the decision will only influence $50,000 - 100,000 of sales – and is not expected to affect business long term – then it rarely makes sense to spend more than half of that on research. (Four focus groups, a survey and analysis will usually top the $50M mark).

On the other hand, if your company is a multi-million dollar entity with hundreds of employees, and you are seeking answers that will take your company in a new direction, then it would be irresponsible not to do the appropriate due diligence. The ticket price on such research will usually start at $50,000 and go up. Sometimes way up.

For some, this may sound as though research is expensive. Not doing research, however, can be even more expensive. That being said, there are types of research that can be completed for less than $10,000. (Please feel free to contact me to learn more on this topic.)
I hope this helps.

Until next time... remember to ask questions and don’t be afraid to go out on that proverbial limb. JMC