Astronomer Carl Sagan observed in his popular 1980 television show Cosmos, “There are many hypotheses in science that are wrong. That’s perfectly all right; it’s the aperture to finding out what’s right. Science is a self-correcting process. To be accepted, new ideas must survive the most rigorous standards of evidence and scrutiny.”
The scientific community’s historical willingness to change course when evidence suggests that a course correction is warranted — no matter who happens to benefit — may help explain why science enjoyed high levels of public trust for many decades. The Pew Research Center reported in 2020 that “public confidence in the scientific community stands out as among the most stable of about a dozen institutions rated in the [General Social Survey] since the mid-1970s.”
That’s changing.
In data collected at the end of 2021, Pew found that among the public, “the share with a great deal of confidence in scientists to act in the public’s best interests is down by 10 percentage points.” The trend can partly be explained by a general decline of trust in institutions, partly by a growing partisan (and educational) divide in trust in scientists, and partly by perceptions of the management of the Covid pandemic.
In my areas of expertise — the intersection of science with contested political issues such as climate change — self-correction in science is facing some serious challenges. Those challenges may further erode trust to the extent that the public cannot reliably evaluate scientific claims (and nonsensical claims) independent of who might benefit or suffer the consequences of setting the record straight.
One important mechanism of self-correction in science is the formal retraction of peer-reviewed scientific publications found to have used flawed or fictional data or employed other questionable practices. The Committee on Publication Ethics explains why this matters: “Retraction is a mechanism for correcting the literature and alerting readers to articles that contain such seriously flawed or erroneous content or data that their findings and conclusions cannot be relied upon.”
Scientific papers with obviously erroneous data would seem to make easy cases for retraction. That hasn’t been my experience.
In 2019, a group of Danish scientists published a paper on historical hurricane damage using a fatally flawed “dataset” in the prestigious journal Proceedings of the National Academy of Sciences, or PNAS. Though the authors published a spreadsheet along with their paper, the dataset simply does not exist outside of their paper. It did not reflect observations from the real world. I know this because my colleagues and I developed the dataset that was the original basis for theirs, which metastasized into something unrecognizable and decidedly unscientific.
The details are amazing. The paper — known as Grinsted et al. 2019, or G19 — built on our work starting in the 1990s, which sought to assess the economic impacts that past U.S. hurricanes would have had if they had made landfall with today’s levels of inflation, population, and development. We developed a methodology called normalization. For instance, the Great Miami Hurricane of 1926 resulted in less than $100 million in damage when it occurred. We estimated that the same storm would cause more than $300 billion in losses in 2024, owing to Miami’s massive increase in building and wealth over many decades and the changing value of the dollar due to inflation.
In their paper, G19 sought to apply a new normalization methodology, contributing to a growing literature on the subject. Their fatal mistake was to (inexplicably) use a dataset of historical hurricane losses that they found online on the website of a (now defunct) insurance company. After I carefully examined G19 and discovered its flaws, which I detailed on my Substack in February 2024, I contacted PNAS and the editor of the paper, Kerry Emanuel of the Massachusetts Institute of Technology, and requested that the paper be retracted.
You won’t ever find a more obvious case for retraction, so I thought the response would be quick and straightforward. Five months later, in July 2024, I heard back from PNAS: “The final decision is that no public action is warranted, be it retraction or correction.”
PNAS apparently did not even look at the dataset or consider the evidence I provided them. Instead, they relied on a review of a letter to PNAS about G19 I had submitted (but they did not publish) back in 2019, which discussed different issues with their paper, well before I was aware of the depth of the problems with the dataset. They sent me a review of that five-year-old letter to justify their inaction.
One additional detail must be mentioned: Of the 70 or so normalization studies around the world for various weather phenomena that have been published over the past 25 years, G19 is the only one that claims to have detected an increasing trend in losses after normalization and that attributed this trend to human-caused climate change. Perhaps as a result, it has been singled out and highlighted by the Intergovernmental Panel on Climate Change and the U.S. National Climate Assessment. That makes it politically important. Its retraction would have resulted in a long series of falling dominoes and embarrassment for those official bodies that jumped to promote its findings while ignoring many dozens of others that make for a compelling scientific consensus. We can only speculate whether that inconvenient fact played any role in the PNAS decision to endorse the fake dataset.
I had a similar experience in 2019 when several colleagues and I discovered that data used in a paper by the International Association of Athletics Federations (IAAF, since renamed World Athletics, which oversees international track-and-field competition) was riddled with errors. The paper, published in the British Journal for Sports Medicine (BJSM), was crucially important because it was the sole peer-reviewed basis for the IAAF’s regulations governing the eligibility of certain women with high testosterone.
We provided the editor of BJSM with incontrovertible evidence of the data errors, which were confirmed by the authors of the paper, and we requested that the paper be retracted. The BJSM editor refused — again, inexplicably. We then wrote up our findings and submitted them for publication in BJSM, and our paper was rejected — we were told that our paper was rejected because we were being critical of the journal. We published elsewhere, and our findings were featured in the New York Times.
A few years later, when the editorship of BJSM turned over, we approached the new editor and asked him to take another look at the flawed paper. He agreed, and the result was a correction published in 2020. Despite that, the original BJSM paper continued to be used by the IAAF to justify its gender regulations until they were superseded. World Athletics no longer mentions its fatally flawed 2017 paper.
I’ve seen these dynamics occur many times whenever science meets policy and politics. Some studies that are fatally flawed but politically important are surrounded by a sort of anti-correction Teflon force field that prevents retraction or even acknowledgment of flaws. They are apparently too big to fail.
There is no more prominent example of these dynamics than with the so-called Proximal Origins paper published by Nature Medicine in 2020, which said of Covid-19’s origins: “We do not believe that any type of laboratory-based scenario is plausible.”
Dogged work by investigative journalists and congressional committees revealed that the paper was motivated by U.S. government officials, included an unacknowledged ghost author, and reflected views at odds with those its authors expressed to one another in private. After the paper was published, those same officials and the article’s editor at Nature Medicine pointed to the paper, apparently to quash discussion of the possibility that the pandemic may have resulted from a research-related incident.
When the congressional oversight committee investigating Covid-19 origins held a hearing earlier this year to explore the publication of the Proximal Origins paper and the broader relationship between scientific journals and the government, two of the three editors of leading journals declined invitations to testify, and the third did not discuss the paper. Such a refusal to publicly defend editorial decisions is difficult to understand.
A wide range of experts (including me) have publicly called for Proximal Origins to be retracted as more evidence emerged that the paper did not accurately reflect the views of its authors but instead was part of an orchestrated effort to shape discussion of the possibility of a lab leak. That effort succeeded until the case for the possibility of a lab leak became much more widely accepted. The paper was hugely influential in creating a misleading narrative. In an era when science is used tactically to buttress political stances, there can be incentives to plant politically convenient research in the scientific literature even if that research is flawed.
The opposite sometimes occurs as well. A recent paper by a group of Italian researchers on extreme weather argued that there is no climate emergency based on their review of the most recent conclusions of the Intergovernmental Panel on Climate Change. That paper was criticized by the Guardian and AFP, which quoted several climate scientists demanding that the paper be retracted. Remarkably, the journal obliged and retracted the paper.
A whistleblower shared with me the reviews and emails associated with the entire process. One reviewer explained that the authors’ claims were “correct” but that the “editors should seriously consider the implications” of publication. The paper was retracted not because of any error in the paper but out of apparent concern for its possible political implications.
Science — the systematic pursuit of knowledge — works because the community adheres to a shared set of norms. One widely discussed formulation, articulated by sociologist Robert K. Merton in the 1930s, holds that, for science to work, four key things have to happen: 1) Scientists must collaborate; 2) recognize that scientific findings are not based on who is making a claim; 3) insist that scientific institutions (such as journals) should be disinterested rather than advocates for a cause; and 4) express skepticism of claims and always subject them to scrutiny.
In the examples I recounted above, each of these norms was violated as journal editors seemingly sought to protect or even weaponize scientific publishing to protect or advance a perspective deemed to be important beyond science. But when self-correction in science is short-circuited, science fails to work. That, in turn, threatens public trust and effective decision-making.
The larger context here reveals a sort of scientific omertà among experts and journalists. Although many scientists have spoken out on Covid-19 origins, many have also faced personal attacks and threats to their careers from both their peers and journalists at major outlets. Climate research may be even worse. My own experiences are well known: I’ve been attacked by the White House, investigated by Congress, and hounded out of a writing gig at Nate Silver’s 538 by efforts with support from a shadowy billionaire — all for the sin of publishing a summary of accurate but unwelcome peer-reviewed research. The social and professional pressures in climate research are immense.
The only way that science in these areas gets back on track is with stronger leadership recommitted to scientific norms. That means calling things straight, even — and maybe especially — when that might mean retraction of a paper with political significance. Journal editors who do this and find themselves in the proverbial crosshairs will need to be supported by editorial boards and publishers who also have the backbone to respect scientific norms.
Ensuring scientific integrity in published research is a choice. This choice is consequential and goes well beyond the short-term benefits and detriment that result from a particular retraction decision. It’s on such choices that enduring public trust in science ultimately rests.