In order to have confidence that your survey results are representative, it is critically important that you have a large number of randomly-selected participants in each group you survey. So what exactly is "a large number?" For a 95% confidence level (which means that there is only a 5% chance of your sample results differing from the true population average), a good estimate of the margin of error (or confidence interval) is given by 1/√N, where N is the number of participants or sample size (Niles, 2006).
The following table shows this estimate of the margin of error for sample sizes ranging from 10 to 10,000. (For more advanced students with an interest in statistics, the Creative Research Systems website (Creative Research Systems, 2003) has a more exact formula, along with a sample size calculator that you can use. For most purposes, though, the 1/√N approach is sufficient.)
|Margin of Error|
|Margin of Error|
You can quickly see from the table that results from a survey with only 10 random participants are not reliable. The margin of error in this case is roughly 32%. This means that if you found, for example, that 6 out of your 10 participants (60%) had a fear of heights, then the actual proportion of the population with a fear of heights could vary by ±32%. In other words, the actual proportion could be as low as 28% (60 - 32) and as high as 92% (60 + 32). With a range that large, your small survey isn't saying much.
If you increase the sample size to 100 people, your margin of error falls to 10%. Now if 60% of the participants reported a fear of heights, there would be a 95% probability that between 50 and 70% of the total population have a fear of heights. Now you're getting somewhere. If you want to narrow the margin of error to ±5%, you have to survey 500 randomly-selected participants. The bottom line is, you need to survey a lot of people before you can start having any confidence in your results.
- This webpage calculates the sample size required for a desired confidence interval, or the confidence interval for a given sample size:
Creative Research Systems, 2003. "Sample Size Calculator," Retrieved June 28, 2006 from http://www.surveysystem.com/sscalc.htm.
- This website has information on statistics and statistical tests, written for the non-mathematician:
Niles, Robert, 2006. "Robert Niles' Journalism Help: Statistics Every Writer Should Know," RobertNiles.com. Retrieved June 28, 2006 from http://www.robertniles.com/stats/.
Support for Science Buddies provided by:
When conducting quantitative research, it is very important to determine the sample size for your study. Your sample needs to represent the target population you plan to examine. Sample size calculation should be done before you set off to collect any of your data. Almost all researchers generally like to work with large samples. However, this is not always feasible — especially for students (time, money, resources, etc.) — so it is a good idea to assess your need before any real research takes place to determine how many participants you really need (and can feasibly get) before planning your research.
This post presents a brief overview of sample size calculations and covers some of the basics. For complex cases and more detail, you will probably require more thorough text on the subject (see: Sample Size Determination and Power).
A Few Terms That Relate to the Size of Your Sample
Here are some expressions you will most likely come across when designing your study and deciding on a sample size.
- Population – This is the complete set of data points, for example, all Americans.
- Target population – This is the complete group for which you are studying; your data will have specific characteristics (demographics, clinical characteristics) that you are interested in — for example, Americans over the age of 65, who live at home and have had a stroke in the past 6 months.
- Sample – A subset of the target population that represents the target population.
- Margin of Error – The margin of error is about a degree of uncertainty in statistics. How much error will you allow? We would like the mean of our data to represent the mean of the target population; however, this is generally not going to happen. The margin of error tells us how much higher or lower than the true value will we let our sample mean fall. In articles, you usually see a +/-5% or +/-3% margin of error.
- Confidence Interval – Confidence interval (CI) is usually set at 90%, 95% or 99%. It tells us how confident we are that if the study was repeated again and again, we would get the same results. If confidence level is 95%, we would get the same results in 95% of the cases.
- Standard Deviation – Standard deviation tells us the variation in the data from your sample.
- Power – This refers to the chance of missing a real difference (‘false negatives’). Usually, studies have a power of around 80%, which means that you accept the possibility that in 20% of the cases, the real difference was missed (you concluded there was no effect when there was one). Larger samples generally yield higher statistical power.
How to Calculate a Sample Size
It is fairly easy to determine your desired sample size. Formulas found in textbooks often appear very intimidating. However, they can be broken down and simplified if you are familiar with the above terms.
Scott Smith, Ph.D., presents a rather simpler version. His formula for size calculation goes as follows:
(Z value)2 X standard deviation (1-standard deviation)/(margin of error)2 = n
This formula, however, can only be used for large populations or unknown population sizes.
The Z-value or Z-score corresponds with your chosen confidence level. There are usually Z tables available that tell you the Z-score. You can then insert that value into the formula. Below are values for the most commonly used confidence intervals.
You can use the formula to calculate a sample size for a confidence level of 99% and margin of error +/-1% (.01), using the standard deviation suggestion of .05.
(2.58)2*0.5(1-0.5)/(0.01)2 = 6.656*0.5(0.5)/0.0001= 16,641
The sample size for the chosen parameters should be 16,641, which is a very large sample. To make it a more realistic number, you might consider reducing your confidence level and margin of error. If you reduce it to 95% confidence level and 5% margin of error, you get a more manageable 384.16 participants, which you round up to 385.
Software for Calculating Your Sample Size
If this still appears like a pretty cumbersome task (or you have a smaller population size), you can also turn to various software programs and websites that will calculate the size for you. An example of such a site is The Survey System, which offers a free online sample size calculator. Another option is Survey Monkey’s sample size calculator, which can also be accessed online. These calculators usually ask you to enter your target population size, confidence level and margin of error.
You can also ask for guidance and assistance from other members of your department who might be more skilled at statistics. For additional ideas on how to get help with statistics, you can have a look at this post.