|
There is no question that web surveys have added a new dimension to marketing research in recent years. After all, it is cheaper, environmentally friendly, and turnaround is faster. Moreover, it holds the promise of the future and is, frankly, much sexier than, say, a mail survey. All that is needed is a database of email addresses, a good web software package and, voilá, it's done.
But as quickly as we have embraced it, so, too, have we seemingly abandoned some of marketing research's basic tenets. In this article we examine the repercussions of disobeying two of these "commandments."
Thou Shalt Not Mix Methodologies
Not long ago, to consider mixing methodologies (combining, for example, mail and telephone survey results) was unthinkable, if not outright heresy. The foundation for this was established when test after test pointed to the reality that people answer differently in different research environments.
Simply put, survey responses are not always consistent when you contrast an individual's self-administered answers to survey questions with those obtained via phone or in a one-on-one setting. Everybody seemed to understand this, everybody seemed to embrace it, and few rebels dared to defy it.
It is now, however, almost common practice to supplement web surveys with telephone interviews or vice versa. It is rationalized that we are simply giving participants more options to take part in the survey. In theory, the trade-off here is mixing methodologies for a greater level of participation.
At Q & A Research, we're not dismissing this argument altogether. On the contrary, in many cases, while not optimal, mixing methodologies may be the only practical way to ensure the representativeness of the survey.
We suggest to avoid mixing methodologies if you can and, when you must, take the extra step of examining the data by the methods first before consolidating your data. See if there are any interesting nuances that suggest you may want to analyze your results separately.
Thou Shalt Have a Representative Sample
When doing quantitative research, who among us would argue that unless the results are representative, they are not projectable? Yet, as near as we can figure, the industry average for response rates on blind consumer web surveys, even with a reasonable incentive, is less than 5%, one seventh that of a comparable telephone survey. Not enough returns yet? Send out another 10,000 email invitations, that ought to do it!
And this shotgun approach is not the worst of it. If you are using purchased email sample, where did it come from? www.weirdpeople.com? How do you know if it is even reasonably representative of your universe?
The best solution for blind web studies, we believe, is to utilize a panel of individuals predisposed to participating in such surveys. "But aren't these professional respondents?" you may ask. If carefully and diversely recruited, rotated, and sufficiently large enough, the panel (like the one Q & A is presently building) will enable you to perform your work quickly and with a high rate of cooperation. It is worth remembering that sometimes the enemy you know is better than the enemy you don't.
Concluding Remarks
So, the word of the day regarding web surveys is "caution." They can be a powerful and affordable addition to your research arsenal, but let's not get so caught up in the technology that we forget the basics.
At Q & A Research, we not only offer web survey capability to our clients but have our own proprietary SPIDER software ensuring that our clients receive superior and reliable data collection.
|
