|
Often escaping the same scrutiny given to survey design and analysis is the sample from which the data are drawn. Sample sources are often purchased lists, the origins of which may not be certain. I call this "orphan sample." If you don't know where your sample comes from, how confident can you be that your results are projectable to the universe you have defined?
An example to illustrate the point: You are charged with determining the general public's reaction to a new freeway construction plan for Pleasant County, USA. Clearly, a list of recent voters is the best choice of sample. While you could get such a list from the registrar's office, you decide to save a couple bucks by purchasing a list of recent voters from a vendor.
The data are collected and the survey ran smoothly. Your survey findings: "The people have spoken. They are united in their opposition to the new freeway."
But who really spoke? It turns out that you interviewed voters from the No Growth for Pleasant County Party, whose conference roll call was sold to a list supplier just months before.
The moral here is that cheaper is not necessarily better. In fact, in my opinion orphan sample has no real value at all unless you are able to uncover its origin. Take a tip from the man in the red suit, "Make your list and check it twice!"
|
