Should you trust scientific research?

Do you trust scientific research? Should you?

Scientific research has been going on for millennia. At its earliest, research was just observational. Those early scientists just watched what was happening and tried to describe it. “During storms, every time time that bright light flashes down from the sky, there’s a loud noise.” Then they’d attempt to connect the two events with some explanation. But they had no way to test their ideas empirically.

As time went on and knowledge increased, these researchers added some tools to their armamentarium. They found ways to measure things like length, volume, time, distance, speed and they could use these to quantify their observations to a point.

As the interest in explaining the world around them grew, more tools were developed and larger strides were made. We had discoveries in mathematics and physics. Magick and alchemy got transmuted into chemistry. And some basic concepts about health began to be investigated.

Ampules via Shutterstock

Ampules via Shutterstock

With more progress, those researchers began to develop the basis of the “scientific method.”  Thing such as using exact measurements, written records, and sharing information with others to spread knowledge. Once the precise sequence of steps was formulated for a particular experiment, anyone could reproduce it by following those steps. And, if it couldn’t be reproduced, the early researchers would often verbally attack whomever claimed to have made the experiment work.

Fast forward a few centuries.

Science has become extremely rigorous. Hundreds of analytical techniques have been developed, new analytical instruments have been built. There are prolonged, formal training programs for scientists. Now foundations have sponsored research. Universities have nourished it. Scholarly journals have disseminated the results. National and international awards have been given to the foremost researchers. Many, many people have become researchers.

However, foundations wanted concrete results for their monies.

Universities wanted researchers who published prolifically and brought prestige to their halls. Journals wanted the best and newest information that would amaze the scientific community. Pressure was put on the researchers. And sometimes the research wasn’t quite as rigorous as it had been previously. Scientists tended to accept the results of other researchers without question, without repeating their experiments. Why?

Well, science should be above reproach. Each researcher should have a sterling character and be ethically pure. Other researchers want to do ORIGINAL research, not merely validate another’s work. And, even if they wanted to repeat previously done research, who would pay for their time? Who would pay for the analytical tools necessary? The reagents, technical assistance all cost money. Foundations wanted to spend funds on a breakthrough, not rehashing a previously run experiment. Universities didn’t have money in their budgets to reproduce research. Some researchers weren’t as scrupulous as they might be. Some wanted a list of published journal articles, grant funding, more analytical devices, more graduate students. So, some of them fudged. They altered some data. They manipulated some statistics. They stole some information from rival labs. Some of them outright lied.

Manipulation of data to get the “expected” results certainly isn’t new. Gregor Mendel, Austrian monk and early geneticist, probably fudged his data. The results he got, in so many experiments were so right on the money that he probably wasn’t completely honest. But, when other people repeated his experiments, they got similar results. His data may have been too perfect, but his overall results were correct.

Unfortunately, over the past several dozen years, more and more research has been shown to be tainted. In the social sciences, the Mark Regnerus study on same-sex parenting comes to mind. In the physical sciences, regrettably, there have been hundreds and hundreds of instances that have been discovered where the researcher(s) deliberately falsified data, plagiarized, tampered with the experiment in question or lied.

Most reputable journals, before printing a research paper, will submit it to anywhere from one to three scientists/physicians who are experts in the field that the paper comes from. They read it critically and evaluate it. If they think that the research is valid and compatible with current knowledge, they approve it and it gets published.

But, with all of the medical and scientific information coming out constantly, it is difficult just to keep up. It’s estimated that there are about 130 journals with information for internal medicine and family medicine physicians that are published EACH MONTH. Imagine how difficult it is to try to keep track of that information, plus information that comes out in the basic sciences (physiology, biochemistry, pharmacology, etc.) and in the other fields (genetics, biology, chemistry and others). And then, to have to review new information and vet it for accuracy. It can easily be an impossible task.

Recently a researcher in Europe who was doing a study in rheumatology was caught red-handed deliberately adding something “extra” to her experimental mixture to cause the desired results. She was dismissed from the university and her papers were retracted. Another researcher, from Japan, had numerous papers retracted for data “manipulation.” I believe that he had over 20 papers retracted before his university fired him. Another, a physician researcher, had at least three papers retracted for having deliberately incorrect data. His university fired him. But another university scooped him up and he is now in the process of suing several different organizations.

There have been many others. Some individual researchers have had GREATER THAN 30 articles of theirs retracted from reputable journals for some of the reasons stated previously. There are now a number of sites that police published papers and get tips from journal readers and scientists about fraud. Consumer Health Digest, and Retraction Watch are some.

When fraud like this happens today, there are two major problems that didn’t occur in Mendel’s era. One is that information spreads so quickly that the data gets read AND INCORPORATED into other journal articles very quickly. For some of the articles mentioned previously, many have been cited in other papers 20+ times. That means that this faulty research has been accepted by members of the scientific community and has been passed on as valid in their own papers. And sometimes the fraud in the original paper isn’t detected for years. So the original incorrect data may become dogma in that period.

Another problem has to do with the journals that publish these papers. As reputable as they are, they don’t want their provenances besmirched. When many of these journals publish a retraction, it’s just a small squib, buried somewhere in the journal. Very easy to miss. So the “retraction” may go unnoticed for quite some time.

The Retraction Watch team have talked with several journals about their lax retraction scheme. Most of the journals feel that the tiny, hidden squib is enough. Some do a little more. They change the available online papers to have the word “Retracted” watermarked on each page. But that obviously doesn’t appear on the already downloaded copies of the papers. Only if the reader goes back to the journal website to re-download the paper, would he see the word “Retraction.” How often is that going to happen?

A few journals are more transparent. They’ll publish an “Erratum(-a)” on the contents page that will list the paper and often give the reason for its retraction. They’ll also do the retraction watermark on the online copy of the paper. That’s about the best that anyone can do, but it still leaves papers out there in journals on library shelves, on researchers’ computers and in other places that have been retracted but can still be read and thought of as being legitimate research.

Right now, there doesn’t seem to be a good fix for these problems. Certainly, the universities that these researchers belong to should investigate the alleged fraud and punish the researcher if the allegation proves true.

The funding organizations can also investigate and, if the researcher is guilty, demand that the funds be returned. They could also ban the researcher from additional grants. But investigations take months and, meanwhile, the information is regarded as correct and continues to be out there available for use.

The journals need to mutually agree on some set policy as to how to handle retractions, so that the largest number of people become aware of the circumstances, and of the fact that the research is tainted.

Scientists should check the (relatively) few websites that post problems of this sort. But even those websites probably miss a number of retractions.

But those options, even if all were enacted, still don’t do enough. There are still papers being published that are fraudulent. Undoubtedly, there are other papers, still out there, that are have falsified information and are are yet to be discovered. Perhaps we need some kind of online central clearinghouse where all retracted papers are listed. That might be a start towards weeding out the tiny percentage of these illegitimate “scientists” from the ranks of those who do their best to present good solid data.

Mark Thoma, MD, is a physician who did his residency in internal medicine. Mark has a long history of social activism, and was an early technogeek, and science junkie, after evolving through his nerd phase. Favorite quote: “The most exciting phrase to hear in science... is not 'Eureka!' (I found it!) but 'That's funny.'” - Isaac Asimov

Share This Post

© 2018 AMERICAblog Media, LLC. All rights reserved. · Entries RSS