Are Online Surveys Biased?

The past few years have seen explosive growth in the use of online surveys. The reasons for this development are obvious. Online surveys cost less to conduct than in-person or phone surveys, response times are faster, and the results are easy to compile and analyze because they are already in a digital format. But no survey method is perfect, and online surveys have been criticized by some as being biased because they collect information only from people who have access to the Internet.

Is Sampling Bias Inevitable?
In fact, most surveys must deal with this type of bias. For example, telephone surveys collect information only from households that have land lines, a shrinking percentage of the population. Paper surveys require that respondents have a certain level of literacy. Online surveys have the same requirement, as well as the obvious additional requirement that respondents have access to the Internet. As Internet access becomes more and more widespread, this is becoming less of an issue. According to recent estimates, more than 74 percent of people in North America have access to the Internet, and the number is growing steadily. Still, there is no question that Internet users represent a more affluent, well-educated segment of the population.

Online surveys must also deal with the common sampling problem of non-response bias. In most surveys, a certain percentage of those solicited will not respond. Survey administrators must somehow determine if non-respondents skew the survey population in some way.

Survey administrators must also have some means of excluding responses from people outside the target population. Because the Internet is such a wide-open, boundary-less medium, the response to a survey may be coming from a broader population than the administrator intended.

Finally, survey administrators may have to deal with sampling bias because of the sites they use to solicit responses. For example, the population of Facebook users includes more women and young people than the population of Internet users in general, so it seems likely that these groups would be over-represented in a survey conducted through Facebook.

Removing Sampling Bias
Mineful software offers a simple but effective way to deal with sampling bias. Post-stratification allows a survey administrator to correct for groups that are over-represented or under-represented in a survey population. Here’s how it works.

Survey respondents are divided into homogeneous subgroups (strata). For example, respondents might be divided into the strata male and female. Responses are recorded separately for men and women, and then a sampling fraction is applied to give each group its correct weight in proportion to the target population.

For example, suppose that a survey administrator wanted an equal number of responses from men and women. As it turned out, sixty percent of respondents were men and forty percent were women. The sampling fraction would allow the administrator to take all responses into account, but would give proportionally more weight to the response from each woman. This would allow the survey results to be an accurate reflection of the target population: half men and half women.

The same method can be applied to correct for imbalances in race, education, age, and other factors. The key to using post-stratification is to identify areas of potential sampling bias and then use survey questions to determine if respondents accurately represent the target population. In the example we used, the survey would ask about gender. Such questions would allow the survey administrator to use post-stratification to reduce the effect of sampling bias.


Post a Comment

Back to TOP