Time to Assume Any Health Research is Fraudulent?

(Central Florida Bob ) #1

That’s the controversial title of an opinion piece in the British Medical Journal.

I’m skeptical of too much science being very poorly done as it is, and I was stunned by some of the numbers the author presents.

(Bob M) #2

I never heard of a “zombie” trial until now. Shocking.

(Joey) #3

Thanks for sharing this link.
Sadly, it doesn’t seem like a terribly controversial title to me :roll_eyes:

http://www.doctoringdata.co.uk/
https://www.badscience.net/



(Central Florida Bob ) #4

And I’ve long been promoting John Ioannidis’s landmark paper on junk science. From 2005 (!)

When I learned about this issue in medical research, the realization hit me that if I had done that in my profession, I’d have been in jail. Not discredited among some subset of readers, I’d have been in federal prison.

(Bacon is a many-splendoured thing) #5

I’d leave off the question mark at the end of the title.

(Jane) #6

I had a roommate working on her PhD at our college with a large agricultural research facility.

She wanted to do research on artifical insemination on cattle but was forced to research chickens because that was the only grant money available. She said they were expected to give the company who provided the funding the answer they wanted and this was back in the 80’s so been going on a long time. They designed their experiments around the expected outcome so the data lined up nicely. Totally corrupt.

(Joey) #7

I’m guessing that artificially inseminating chickens can be equally intriguing as a field of study. Then again, soft music and a little privacy is likely most cost-effective. :hatching_chick:

(Bacon is a many-splendoured thing) #8

There’s a famous cartoon of a chicken and an egg together in bed, smoking cigarettes. The legend is, “Well, I guess that answers that question, doesn’t it?”

(Joey) #9

Love that one. The one I recall seeing has the chicken sitting on the edge of the bed - the only one smoking. The egg, still in bed, is in tears with the caption you’ve noted above. :laughing:

(Bacon is a many-splendoured thing) #10

That’s better than the one I remember!

#11

“It may be time to move from assuming that research has been honestly conducted and reported to assuming it to be untrustworthy until there is some evidence to the contrary.”

I guess I’m naive, but I assumed the peer-review process was supposed to assume the research was wrong until proven correct.

(Central Florida Bob ) #12

That’s the way it’s supposed to be, but the process is clearly broken.

Look at it this way: they mail manuscripts to reviewers. The reviewers don’t get paid for this, and duplicating the work costs money, not to mention the researcher’s time. The review also has a publisher’s deadline. All of that adds up to the review being reading the paper and looking for obvious dumb ■■■■ mistakes.

The original author has the “publish or perish” incentive. The publisher, though, also has that; if they don’t publish papers, who needs the journal? The reviewer has the negative incentive of not working on their own “publish or perish” needs, because spending lots of time reviewing someone else’s paper means not working on their own.

#13

Then I guess the question is, how do we change the incentives where the reviewer will take their time to do the work and be rewarded for it?

I like the authors conclusion, but this doesn’t seem to be the solution if the “publish or perish” incentive isn’t changed.

I know, I’m a simple man, with simple pleasures. :slight_smile:

(Joey) #14

One perspective worth considering…

(Bacon is a many-splendoured thing) #15

This quotation sums up the problem admirably:

We often hear from scientists like Elodie Chabrol, a researcher in neuroscience at UCL London, who says: “I’m not an expert on peer review or an editor. I’m just a frustrated scientist. Getting published is essential to building a career and it’s not easy. It is frustrating to know that my research won’t be published for months. I know that some of the papers I read are old news by the time they reach me. No-one would read the news with months of delay, but that’s what we scientists do.”

So what this researcher wants is instantaneous results, even if wrong, rather than for her and her colleagues to make a lasting contribution to the sum total of human knowledge. What a shame!

In this researcher’s defense, however, the problems are systemic and are the result of shifts in how science is done, shifts that have not been thoroughly examined. Back in the days when researchers were independently wealthy and could afford to pay for their own experiments (which were also less costly back then), their reputation and career advancement didn’t depend on churning out published paper after published paper. If you are not rich enough to fund your own research—and who is, these days, in any field?—you have to depend on your university or other funding sources to support your research and pay your salary. And donors want splashy, sexy results, so they don’t want to fund the boring old confirmatory experiments that are the backbone of science. If all the tenure committee sees is the number of papers on your résumé, then you have no incentive to care about getting correct results, only in persuading the journal to publish your paper now, before and not after the meeting of the tenure committee.