What if scientific studies disagree?
Based on Science
There are many reasons why scientific studies may have different results, even when their methods are the same. If two studies disagree, it does not necessarily mean that one of them must be wrong.
Last update April 16, 2021
Repetition is part of science.
Science helps us learn about the world. For example, a study comparing two kinds of wheat can help us learn which one grows best in a particular place. Scientists use studies to answer all sorts of questions, such as whether a new medicine works, how our brains learn a language, or how the moon formed.
But studies can be wrong. How do scientists tell if they got the right answer? One way is to do the study again and see if the results are the same.
There are two main ways to redo a study:
Examine the same question using the same methods and the same data. This is called reproducing the study.
Examine the same question using the same methods and new data. This is called replicating the study.
If a study is done multiple times by different scientists with similar results, scientists become more confident that the studies are providing the correct answer.
There are many reasons why studies disagree.
Scientific studies often come to different conclusions about the same topic. Sometimes, studies have different results even when scientists try to use the same methods and data.
Scientists try to share their methods and data so that others can redo their studies. But even a small change, like using a different brand of beaker, can cause the results to be different. It’s easy to make a small change without even knowing it. Often, disagreements result from:
Not writing down the methods fully or correctly.
Not following the methods fully or correctly.
It’s not always possible to redo a study exactly the same way. For example, some computer algorithms work in a way that gives slightly different results each time. Also, some types of data change or go away over time, making it hard to collect the same data twice. For example, if a comet whizzes by a telescope, you might only be able to get one picture before it flies away. It is common for studies to disagree if they:
Use methods that do not work the same way each time.
Study something that is rare or that naturally changes.
Occasionally, disagreements result from fraud, bias, or mistakes. In these cases, it is important to hold those involved accountable and correct the record.
There is always some uncertainty in science.
Scientific methods often have variability that can’t be avoided. For example, devices like scales and thermometers can give slightly different results even when measuring the same thing.
Data can change as well. If you measure the temperature outside today and measure it again tomorrow, you’ll probably get different results. That’s not because you measured wrong, but because temperature changes over time. Some differences are to be expected. It is important to understand the level of uncertainty or error that is expected in a study. Just because all studies have uncertainty doesn’t mean that they are wrong. Sometimes, results that are slightly different can still be considered close enough if the difference is within the expected range of uncertainty. When studies disagree, it is useful to try to find out why. This can help improve scientific methods and even lead to new discoveries.
