Surveys Research - Sample Selection
In a perfect world, a survey research project could study all the members of a target universe. Generally, this is neither practical nor affordable. Instead samples of the larger population (universe) are generated - the sample is the base from which assumptions are made about the target universe. Further, the sample is constructed by using techniques and strategies that contribute to a valid and reliable study. Traditional market research is based on this idea that a sample - a representative group of respondents -- can be identified and accessed.
Representative Samples in Survey Research
In market research, the term representative sample refers to:
- The selection of a few consumers who match members of a target universe of consumers. An example of a target universe could be owners/users of SmartPhones, ages 20 to 30.
- The match between the sample and the universe must be strong for all attributes anticipated to be influential on survey outcomes. One example of a sample-to-universe match could be the selection of consumers for a perfume designed by a young, female celebrity. In this instance, attributes anticipated to be influential in survey outcomes would be: Female, 18-28 years old, entertainment-savvy. A secondary set of attributes might be: Urban-dwelling, enrolled in college, residing on the east coast or west coast, discretionary income (income levels).
- The proportions of members to whom relevant characteristics can be attributed in a sample must closely approximate the proportions of members in the targeted universe of consumers. For example, if the consumer universe contains business people, college students, and senior citizens, a representative sample could not be built from agreeable students in the university bookstore on Wednesdays afternoons. While this may appear to be an obvious consideration, various constraints related to access of survey participants can seriously hamper attempts to achieve a representative sample. This is one of the primary reasons why professional panels of consumers are often used in survey initiatives. Another effective strategy is to use a stratified random sampling procedure that assists a researcher to tease out data about sub-groups.
Sample Selection in Survey Research
Members of a sample are selected in a number of ways, all of which are intended to reduce bias, thereby increasing the probability that research conclusions are valid, and can be extrapolated to the target universe.
Survey samples are preferably selected through a randomizing process. For instance, if sample members are selected from a database, every third member in the database listing might be selected. Occasionally, members of a sample may need to be assigned rather than randomly selected. This is not a preferred approach as, even under the best conditions, surveys are subject to sample-based inaccuracies that have everything to do with chance and nothing to do with research design. Let's look at a list of sources of error, modified from voter telephone polling issues identified by Experimental Resources. This list includes possible sources of inaccuracies across survey design, survey implementation, and analysis of survey data:
- Incomplete information about members of a database result in important variables being left out of the sample
- Sample members who were selected are unwilling to participate in the survey.
- Sample members who decline to participate in the study are different with regard to an important variable in the study, than those sample members who agree to participate.
- Survey respondents provide false or incomplete responses to survey questions.
The items in this list, again modified from the telephone polling list by Experimental Resources, are related to survey design.
- A randomization process was used but - by chance -- it picks up too many outliers.*
- The questions on the survey are worded poorly and confuse the respondents.
- The order of the questions on the survey unduly influences the responses of subsequent questions.
- Survey responses are subjected to weighting or grouping that distorts the data.
Once a market researcher is reasonably comfortable that a sample is representative of the target population in his survey research, attention can shift to consideration of sample size and confidence intervals.
*Experiment Resources is an interesting website created by psychology researchers who were trying to figure out how to calculate and remove outliers.