How Confident Do You Want to Feel?
The size of your sample depends on how confident you want to be about your results. They key question is: "Are these results truly representative of a larger population?" If you would like to have a confidence level of 95%, you would need to have a margin of error of 5% or less. This means that there would only be a 5% chance of your results differing from the results you would get if you surveyed the entire population. (The margin of error is also called the confidence interval.)
There is a simple equation you can use to determine margin of error:
The margin of error equals 1 divided by the square root of the sample size. Suppose you had a sample size of 400. Your equation would be 1 divided by 20 (the square root of 400), which equals 5%. So to have a confidence level of 95%, you would need to have a sample size of 400.
Obviously, the higher the confidence level you want, the larger your sample needs to be. A confidence level of 95% is widely considered to be acceptable.
Of course this assumes that you have a truly random sample. As experienced pollsters know, it is easy for sampling bias to sneak into a survey without anyone noticing.
How Many to Ask?
So once you have decided on a confidence level, you can easily determine how large a sample you need. But how many people do you have to survey to get the desired number of responses? That is a trickier question.
Response rates vary greatly from one population to another. One of Mineful’s clients, a medical association, received a 25% response rate to a member satisfaction survey. Another client had a similar response to a survey conducted in connection with a conference. But such high rates are the exception rather than the rule. For a survey conducted through cold email invitations, a response rate of 1% is considered good.
Response rates are affected by a number of variables:
- How strongly people feel about the survey topic.
- How much loyalty people feel toward the organization conducting the survey.
- How easy it is too complete the survey.
- How confident people feel that someone will pay attention to their responses.
Fortunately, there is an online sample size calculator that can make this whole process much easier (http://www.raosoft.com/samplesize.html). These are the variables you will need to plug in to use this calculator:
- Acceptable margin of error
- Acceptable confidence level
- Population size (for populations smaller than 20,000
- Response distribution