Below is a cache of http://nizkor.org/features/fallacies/biased-sample.html. It's a snapshot of the page taken as our search engine crawled the Web. The web site itself may have changed. You can check the current page or check for previous versions at the Internet Archive.
Yahoo! is not affiliated with the authors of this page or responsible for its content.
Fallacy: Biased Sample
Fallacy: Biased Sample
Also Known as: Biased Statistics, Loaded Sample, Prejudiced
Statistics, Prejudiced Sample, Loaded Statistics, Biased Induction,
Biased Generalization
This fallacy is committed when a person draws a conclusion about a
population based on a sample that is biased or prejudiced in some
manner. It has the following form:
Sample S, which is biased, is taken from population P.
Conclusion C is drawn about Population P based on S.
The person committing the fallacy is misusing the following type of
reasoning, which is known variously as Inductive Generalization,
Generalization, and Statistical Generalization:
X% of all observed A's are B''s.
Therefore X% of all A's are Bs.
The fallacy is committed when the sample of A's is likely to be
biased in some manner. A sample is biased or loaded when the method
used to take the sample is likely to result in a sample that does not
adequately represent the population from which it is drawn.
Biased samples are generally not very reliable. As a blatant case,
imagine that a person is taking a sample from a truckload of small
colored balls, some of which are metal and some of which are plastic. If
he used a magnet to select his sample, then his sample would include a
disproportionate number of metal balls (after all, the sample will
probably be made up entirely of the metal balls). In this case, any
conclusions he might draw about the whole population of balls would be
unreliable since he would have few or no plastic balls in the sample.
The general idea is that biased samples are less likely to contain
numbers proportional to the whole population. For example, if a person
wants to find out what most Americans thought about gun control, a poll
taken at an NRA meeting would be a biased sample.
Since the Biased Sample fallacy is committed when the sample (the
observed instances) is biased or loaded, it is important to have samples
that are not biased making a generalization. The best way to do this is
to take samples in ways that avoid bias. There are, in general, three
types of samples that are aimed at avoiding bias. The general idea is
that these methods (when used properly) will result in a sample that
matches the whole population fairly closely. The three types of samples
are as follows
Random Sample: This is a sample that is taken in such a way that
nothing but chance determines which members of the population are
selected for the sample. Ideally, any individual member of the
population has the same chance as being selected as any other. This type
of sample avoids being biased because a biased sample is one that is
taken in such a way that some members of the population have a
significantly greater chance of being selected for the sample than other
members. Unfortunately, creating an ideal random sample is often very
difficult.
Stratified Sample: This is a sample that is taken by using the
following steps: 1) The relevant strata (population subgroups) are
identified, 2) The number of members in each stratum is determined and
3) A random sample is taken from each stratum in exact proportion to its
size. This method is obviously most useful when dealing with stratified
populations. For example, a person's income often influences how she
votes, so when conducting a presidential poll it would be a good idea to
take a stratified sample using economic classes as the basis for
determining the strata. This method avoids loaded samples by (ideally)
ensuring that each stratum of the population is adequately represented.
Time Lapse Sample: This type of sample is taken by taking a
stratified or random sample and then taking at least one more sample
with a significant lapse of time between them. After the two samples are
taken, they can be compared for changes. This method of sample taking is
very important when making predictions. A prediction based on only one
sample is likely to be a
Hasty Generalization
(because the sample is likely to be too small to cover past, present and
future populations) or a Biased Sample (because the sample will only
include instances from one time period).
People often commit Biased Sample because of bias or prejudice. For
example, a person might intentionally or unintentionally seek out people
or events that support his bias. As an example, a person who is pushing
a particular scientific theory might tend to gather samples that are
biased in favor of that theory.
People also commonly commit this fallacy because of laziness or
sloppiness. It is very easy to simply take a sample from what happens to
be easily available rather than taking the time and effort to generate
an adequate sample and draw a justified conclusion.
It is important to keep in mind that bias is relative to the purpose
of the sample. For example, if Bill wanted to know what NRA members
thought about a gun control law, then taking a sampleat a NRA meeting
would not be biased. However, if Bill wanted to determine what Americans
in general thought about the law, then a sample taken at an NRA meeting
would be biased.
Bill is assigned by his editor to determine what most Americans
think about a new law that will place a federal tax on all modems and
computers purchased. The revenues from the tax will be used to enforce
new online decency laws. Bill, being technically inclined, decides to
use an email poll. In his poll, 95% of those surveyed opposed the tax.
Bill was quite surprised when 65% of all Americans voted for the taxes.
The United Pacifists of America decide to run a poll to determine
what Americans think about guns and gun control. Jane is assigned the
task of setting up the study. To save mailing costs, she includes the
survey form in the group's newsletter mailing. She is very pleased to
find out that 95% of those surveyed favor gun control laws and she tells
her friends that the vast majority of Americans favor gun control laws.
Large scale polls were taken in Florida, California, and Maine
and it was found that an average of 55% of those polled spent at least
fourteen days a year near the ocean. So, it can be safely concluded that
55% of all Americans spend at least fourteen days near the ocean each
year.
