Since the meteoric rise of social media got started over ten years ago, people have had a rapidly accelerating interest in sharing information. It has become increasingly common that people strive to concoct solid arguments for their positions by incorporating the tenets of logic, citing evidence, studies, and news articles.
This has been a marvelous step forward in our collective thinking and debating skills- but there are still a number of pitfalls that we should all be aware of. One of these pitfalls is the array of ways that statistics can be flawed.
Paul Harrell, former Marine and gun safety expert once said, “Statistics are like a bikini, what they reveal can be interesting, but what they conceal can be even more interesting.”
Mr. Harrell is not the first to say this, but he makes a great point. Whenever a study is done in order to deliver a statistic for use in bolstering an argument, there are a number of ways that the study can be flawed and reasons to be skeptical of it.
By examining the source of the statistic, and scrutinizing the way the study was conducted, we can save ourselves the embarrassment of citing a bad statistic. We can also defeat arguments that rely on bad statistics where we would otherwise be shut down by them.
Consider the Source
The first thing you should do when confronted by the statistic is to look at the agency that conducted the study. If, for example, Planned Parenthood were to conduct a study of successful birth control outcomes based on different methods of birth control, and they published a statistic that suggests that the birth control pill is the best form of birth control- we would have good reason to doubt that statistic.
For one thing, Planned Parenthood does not provide prenatal care. This was revealed by independent investigators who called Planned Parenthood offices across the U.S. and by whistle-blowers.
What’s more, we also know from whistle-blower sources that Planned Parenthood pushes the pill specifically because it is the least reliable form of birth control- especially for young women- and they do so because it increases their likelihood of becoming abortion patients.
On this basis alone, we can say that Planned Parenthood has a vested interest in promoting an unreliable form of birth control because abortion procedures are their big ticket item. This would give us a good reason to look closer at the study.
Lurking Variables
Sometimes stats are quoted which are based on a sampling that is unrelated to the point at hand. Suppose, for example, you had two hot dog stands. Someone says to you that stand A satisfied 20 customers last week, and stand B satisfied 15 customers last week. This would seem to indicate that stand A serves better hot dogs.
But what the stat may not be telling you is that stand A is set up outside of a baseball park, while stand B is standing next to a health food store. Clearly, stand A has a better location. This situation actually makes it look like stand B serves better hot dogs since they are converting health food customers.
Representation
Another factor that can skew a statistic, is sample size. If I tell you that 9 out of 10 women prefer mint ice cream to vanilla, but I only asked ten women in my survey, then my sample is very poor. You should expect me to do a much larger study before being convinced.
Another weakness in a study may be the relevance of the population surveyed. Suppose I agreed that I need to ask a larger number of women about their ice cream flavor preference before reporting my statistic. Then, I go to a mint ice cream fan club convention, interview the women there and then report back with better numbers for my claim. Clearly, I would not be a very trustworthy researcher, were this the case.
These are just a small number of the problems that can contribute to a faulty statistic. Knowing these is a good start to being a better judge of statistics. Other potential flaws in stats include variation and standard deviation. These require some mathematical knowledge of probability to understand, and are easy to conceal as problems in a study.
Armed with this knowledge, you no longer have to be shut down with a seemingly convincing statistic. Examine the study to discover if the statistic is solid or not. Chances are, you’ll learn a lot about how stats are frequently used to push an agenda.
~ Conservative Zone
