Why are some studies wrong?

The only guarantee is there are no guarantees

One unfulfilled expectation of science is flawless accuracy. Unlike math, where formulas are always right and answers stay the same across millennia, health science is the attempt to explain and apply findings from an ever-evolving and changing planet to the current state of human wellbeing. What was effective 100 years ago is no longer useful because microbes and humans and the environment are continuously changing. Today’s world is not the same as yesterday’s and those shifts, however subtle, require the development and discovery of new knowledge and advancement.

Science will never be a crystal ball

Because of this ever-changing potential combined with the uniqueness of each individual, health science does not use standard mathematical formulas to arrive at conclusions. Instead, health research relies on statistics, which is a method of predicting outcomes based on a set of data. One of the key underlying concepts of statistics is that this practice relies on probability rather than certainty. While math teaches that 2+2=4 every single time, quantifying the risk of catching the flu this winter is an attempt to pinpoint an ever-changing number. Therefore, the best we can do is predict a given range of the risk, accompanied by an estimate of our level of certainty regarding that range. In the absence of a crystal ball, no existing scientific practice can predict the future with 100% accuracy.

This scientific principle is not unique to health risks and treatments. It is seen across the spectrum of scientific inquiry. For example, when tomorrow’s weather forecast predicts a 90% chance of rain, then tomorrow comes, but it does not rain, the natural response is to call the prediction wrong. But in reality, the prediction was accurate. It predicted a 90% chance of rain. Which means there was a 10% chance that it would not rain. Probability is not certainty, even when the forecast predicts rain at 99.999%.

This holds true in medical research. Because of the limitless number of factors that make each of us unique, scientific studies on risk factors and on treatment efficacy will never produce results with 100% certainty. That’s why the word prove is a scientific swear word. Rather than prove, science discovers evidence and when that evidence reaches an acceptable level (something that varies by field and topic), it is considered to be sufficient evidence to accept the finding as the best available answer to that specific question.

The best available is always the best available

If science can’t be certain, then why bother? If all we are doing is predicting probabilities, why isn’t my guess as good as yours? Not being able to predict outcomes at 100% does not mean research studies are predicting at the same level of success as folklore or internet speculation. Everything in life can be reduced to probabilities. You turned on your computer or device to read this article relying on the probability that you would not electrocute yourself in the process. You may have left your home today and gotten into a car on the strong probability that the car would deliver you safely to your destination. None of these things are guaranteed, but you have become accustomed to these behaviors and are comfortable with the odds being in your favor.

Similarly, researchers are able to interpret the strength of the probability to predict outcomes as accurately as is possible with the current available data. It’s not guesswork any more than your decision to step into the shower was based on an unconscious calculation that the odds predict that you will not slip and fall and seriously harm yourself. The best available evidence is often quite strong, in many cases much higher than 99.999%, but because scientific ethics require honesty, the fact that the 0.000000001% possibility exists must still be disclosed.

How do you decide what to do?

This is where research interpretation comes into the picture. Just because everything cannot be known does not mean that many things are not known. Correctly and accurately interpreting and translating findings from complex scientific studies is an acquired skill that cannot be self taught. Clinical researchers spent many years of full time training in the complexities of analyzing scientific data to produce knowledge that helps to close those gaps. Going straight to the source for interpretation and understanding of those findings is always the best and most accurate way to apply them to your life.


Get Updates

Sign up for Franklin Health Research Foundation news & updates to get the latest cutting edge research.