|
Falsified surveys ... Not following skip patterns ... Failing to probe open-ended questions ... Double screening ... Skewed demographics ... If you don't already fear these in the execution of your mall intercepts, maybe you should.
Mall intercepts are a valuable quantitative technique used widely for applications requiring direct interviewer-respondent contact, such as taste testing, distributing products for in-home trial, or showing package designs, animatics, and commercials.
Mall facilities that execute the research almost always serve as subcontractors. A few perform excellent work. The majority do good work. But some are suspect and the reason vigorous quality control procedures need to be in place.
Quality control begins with the selection of your research supplier. Remember, there is no substitute for experience. If a supplier does enough fieldwork, they will know, market by market, which mall facilities do the best work and which are to be avoided. Unless you are the gambling type, and willing to bet the success of your study on the competence of an unproven facility, there's no sense taking an unnecessary risk by being the first to try a new field agency.
Another key component of quality control is the demographics of your mall locations, something agencies routinely track. Imagine inadvertently selecting a mall that has a disproportionate ethnic patronage, one that is predisposed to being extremely negative to your new entrée concept. At best you might have included an ethnicity question and caught this problem after the fact. At worst, you never asked and made a decision to pull the product based on severely biased data.
Study Preparation
Once you are sure you have a competent agency at a representative site, it is time to prepare the materials to be used in the interviewing. These include supervisor and interviewer instructions, show cards for scaled or pre-listed answer categories, surveys with easily readable instructions, and tally sheets to record contact and terminate information.
Next it is time for the most important quality control step of all-briefing the supervisors and interviewers. A face-to-face briefing is always preferred, but budget constraints often dictate a briefing by phone. Whatever the format, thoroughly review the interviewers' responsibilities. It is imperative that interviewers in all locations execute in the same manner or you will have opened the door for bias yet again.
During the briefing, make a point to emphasize that double screening will not be allowed. Double screening is the practice of interviewing an individual for one project, then screening to see if they qualify for another. Several years ago, I was monitoring an interviewer who completed a survey with one person for someone else's project, and then asked if that same person wanted to participate in mine. Suddenly, all the qualifying questions for the first survey were unwittingly on the front end of my screener. Needless to say, this practice can have disastrous consequences.
Study Execution
Once the project is underway, daily progress reports and representative survey batches sent by the field agency are essential. Progress reports keep track of where people are falling out of the screener and ensure that any incidence problems are detected early, before irreparable damage is done to the budget. Agencies should also edit the surveys, making sure that each survey was completed properly. They should then forward a representative batch to the supplier for review. This has saved many projects from improper execution.
Verifying that surveys actually took place is the only sure way to detect falsified surveys. Why would an interviewer do such a thing? Sometimes out of laziness, sometimes to over-report hours worked, sometimes to receive incentives for completed interviews. At minimum, a random sampling of 10% of all surveys should be verified by the agency, and at least another 10% by the supplier. If you find a suspicious survey, then 100% of that interviewer's work should be checked.
A lot goes on behind the scenes of successful mall intercept projects. It pays to shop for a supplier who not only understands the necessary quality control procedures, but consistently implements them as well.
|
