|
Falsified surveys, failing to probe and clarify open-ended responses, not following skip patterns, and double screening. If you don't fear these in the execution of your mall intercepts, you should. Or, perhaps better put, your marketing research supplier should.
Mall intercepts are a particularly valuable quantitative technique and are used widely for applications that include taste testing, in-home product trials, and evaluating packaging design, animatics, commercials or storyboards.
Mall facilities executing the research almost always serve as subcontractors to research consultants or suppliers in the chain of professionals utilized to meet a client's objectives. While the majority are conscientious and take pride in their work, the balance is suspect and serves as the impetus for the implementation of rigorous procedures to ensure successful execution of mall intercepts.
Where to Begin?
Quality control begins with the selection of your research supplier. The adage that "there is no substitute for experience" truly applies here. If a supplier does enough of this type of fieldwork, they will already know through experience which mall facilities do the best work and which don't, market by market. Unless you are just the gambling type, there's no sense taking an unnecessary risk by being the first to try a new field agency.
The Right Location
Where you conduct your research is probably just as important, if not more so, than with whom. As you might imagine, regional differences across the country can play a big role in consumer opinion. Even on a more micro level within key markets, some malls draw different consumers; some high end, some low end. Some, by virtue of their neighborhood locations, can skew older or younger or draw disproportionately by income, education or ethnicity.
"Unless you are just the gambling type, there's no sense taking an unnecessary risk by being the first to try a new field agency."
A few years ago, an inexperienced supplier was asked to help execute a study designed to get reaction to a new concept for a rice side dish. Knowing that one market was of particular interest to their client, the supplier sought bids from mall locations in the area. To no surprise, with no other basis for comparison, the low bidder was awarded the project (always a comforting thought).
The study was conducted and, based on the high marks received in its evaluation, the concept was hailed a success. Decisions were made and dollars were spent, all based upon these findings. There was one small problem, however, a piece of information that, if known, would have saved a lot of heartache on the front end of the project.
You see, 60% of this particular mall's foot traffic consisted of members of a minority whose diet, tastes and preferences, it could well be argued, do not mirror the more broadly defined market. Needless to say, the product never did as well as initial projected estimates due to this design flaw.
The message here is to ask for a demographic breakdown of mall patronage before you commit to doing research in an unfamiliar location.
"Ask for a demographic breakdown of mall patronage before you commit to doing research in an unfamiliar location."
Material Preparation
Once field agencies have been selected and schedules are in place, it is time to prepare all materials to be used in the interviewing process. These include, as a minimum, supervisor and interviewer instructions, show cards for many scaled or pre-listed answer categories, surveys replete with easily readable and understood interviewer instructions, tally sheets to record contact and terminate information, and any other materials critical to the execution of the survey.
Briefing
Next, it is time to brief the supervisor and interviewers. An in-person briefing is always preferred, but travel budget constraints often dictate that this be performed by telephone. In any case, a thorough review of the interviewers' responsibilities is in order. It is imperative that interviewers in all locations execute the survey in the same manner or you will once again have injected a good dose of bias.
Quality Control Procedures
Implementing quality control standards is also key to conducting successful intercepts.
Double screening (the practice of interviewing an individual for one project, then screening to see if they qualify for another) should never be allowed.
Several years ago, incredibly (especially in light of the fact that I was there to witness it, personally), an interviewer completed a survey with one person for someone else's project, then proceeded to ask if that person wanted to participate in mine. Suddenly all the qualifying questions for the other survey were unwittingly attached to my screener. Needless to say, the practice of double screening, while enhancing agency productivity, can lead to disastrous consequences.
Other important quality control measures include: - When conducting interviews in multiple cities, as time permits, kick off in one location a day or two prior to all others, allowing you time to fix any unforeseen problems in the execution of your research.
- Require the field agency to have a supervisory edit of all completed surveys.
- Instruct the agency to forward an initial batch of completed surveys for review by the supplier's project manager to ensure that surveys are properly filled out.
- A potentially expensive proposition, but available nonetheless, is to have a local "mystery shopper" screen him or herself in on your study to offer feedback specific to just how the research was conducted.
"Needless to say, the practice of double screening, while enhancing agency productivity, can lead to disastrous consequences."
Survey Verification
Verifying that the surveys actually were conducted is the only way to feel comforted that you haven't fallen prey to any of a myriad of motivators to falsify them. We suggest a minimum of 10% of all surveys be verified by the agency conducting the interviews and at least 10% by the supplier, ensuring that each interviewer's efforts have been checked in the process. If you find a suspicious survey, 100% of that interviewer's work should be verified. Why would an interviewer make up a survey? Sometimes out of laziness, sometimes to pocket an incentive targeted for a participant, sometimes to justify hours worked, sometimes because their own supervisors offer incentives for completed interviews.
Final Words
An executive was once heard to say "Half of my advertising works and half doesn't, I just don't know which half." If you ever find yourself questioning whether your mall intercepts are working or not, then it might well be time to "shop around" for a supplier that has experience and implements vigorous quality control procedures.
|
