Two Medical Journals stand above the rest. The Lancet and the New England Journal of Medicine (NEJM) are generally regarded as the most respected medical journals worldwide.
This is what the editors (current and former) of the Lancet and NEJM had to say about the state of publishing in their field.
Richard Horton, current editor of The Lancet, wrote
The case against science is straightforward: much of the scientific literature, perhaps half, may simply be untrue. Afflicted by studies with small sample sizes, tiny effects, invalid exploratory analyses, and flagrant conflicts of interest, together with an obsession for pursuing fashionable trends of dubious importance, science has taken a turn towards darkness
Richard Horton: The Lancet
Marcia Angell, former editor of The New England Journal of Medicine penned
It is simply no longer possible to believe much of the clinical research that is published, or to rely on the judgment of trusted physicians or authoritative medical guidelines. I take no pleasure in this conclusion, which I reached slowly and reluctantly over my two decades as editor of The New England Journal of Medicine
Marcia Angell: Former Editor of The New Emgland Journal of Medicine
Clearly what these two well respected Editors write carries great weight. And this problem is well known within the scientific community. What are some of the reasons published science is in such poor condition? In no particular order here are the top ten reasons.
- Publication bias: Studies that report positive results are more likely to be published, while studies that report negative results are less likely to be published, leading to an over-representation of positive results. (Dwan et al., 2008; Fanelli, 2010)
- P-hacking and selective reporting: Researchers may selectively report only the statistically significant findings from their studies or manipulate data to produce statistically significant results. (Ioannidis, 2005; Simmons et al., 2011)
- Reproducibility crisis: Many studies cannot be replicated by other researchers, raising questions about the validity and reliability of their findings. (Begley & Ellis, 2012; Prinz et al., 2011)
- Conflicts of interest: Researchers may have financial or other conflicts of interest that influence the design, conduct, and reporting of their studies. (Chimonas et al., 2007; Lexchin et al., 2003)
- Flawed peer review process: Peer review may not always be effective at identifying errors or improving the quality of research, and may be subject to biases or conflicts of interest. (Lee et al., 2013; Smith & Glass, 1977)
- Funding biases: Funding sources influence the design and outcomes of medical research studies, leading to biased or unreliable results. (Gøtzsche, 2013)
- Lack of diversity and generalizability: Many studies are conducted on specific populations or with small sample sizes, which can limit the generalizability and applicability of their findings. (Baquet et al., 2003; Inclusion Across the Lifespan II Workshop Committee et al., 2018)
- Poor study design and methodology: Poorly designed studies with inadequate methodology can produce unreliable results. (Deaton & Cartwright, 2018; Ioannidis, 2005)
- Prejudice and discrimination: Prejudice and discrimination can influence research questions, study designs, and interpretations of results, leading to biased or unreliable findings. (Crenshaw, 1991; Jones, 2000)
- Publish or perish: Pressure to publish research can lead to a focus on quantity over quality, incentivizing researchers to produce studies that are not adequately designed or executed. (Kidd & Barreau, 2016; Nylenna et al., 1994)
Published scientific articles have profound effects on society. They can move markets, underpin public policy and greatly influence medical decisions for example.
Is all science impeachable? No. Is there great science that contributes to humanity in profound ways every day? Yes. But to be informed is to be forewarned.
notrightnotleft.net advocates for and encourages the scientific method. We at the same time strongly discourage the oddly fashionable word phrase ‘belief in science’. Belief in science is perhaps the most unscientific and ironic concept we have ever encountered. Science is the opposite of belief. Science is a search for accuracy and truth about the world. Science is perpetually questioned and debated; this is the nature of science.
