March 21, 2009

What Happens To Fake Studies?

What, you think they go back and fix them?

Here's the setup, from Scientific American:

Over the past 12 years, anesthesiologist Scott Reuben revolutionized the way physicians provide pain relief to patients undergoing orthopedic surgery for everything from torn ligaments to worn-out hips. Now, the profession is in shambles after an investigation revealed that at least 21 of Reuben's papers were pure fiction, and that the pain drugs he touted in them may have slowed postoperative healing.

 A doc on various Pharma payrolls authors studies promoting Neurontin, Celebrex et al, and they turn out to be fakes.  There are plenty of people who will/want to make the obvious points, so I'll restrict my comments to the less obvious.


Reuben managed to trick some othopaedic surgeons to be coauthors on his paper.  One such orthopod, Evan Ekman, was reviewing one of Reuben's studies.

Ekman agreed to review a Reuben manuscript on surgery on the anterior cruciate ligament (ACL) in the knee.  But when [Ekman] asked the anesthesiologist [Reuben] for the name of the orthopedic surgeon on the study, Reuben ceased communication with him.
Undoubtedly, everyone is focusing on the identity of the unknown orthopaedic surgeon as a red flag.  But here's a question that will become obvious once I ask it: how did Ekman know the identity of the author of the paper he was reviewing?  This is "anonymous peer review," right? 

Next: even if he was alerted by 9 pixies that it was Reuben who wrote the paper, why would he think he was further entitled to know the rest of the authors' identities?

I'm not saying he should/should not know; I'm saying that this had become so commonplace to him that he doesn't even think twice about asking; or telling Scientific American that he asked.  He actually forgot to pretend peer review is anonymous.

You may also ask why Ekman was chosen as a peer reviewer in the first place, since Ekman had previously coauthored other papers with Reuben.  Oh, yeah: when you submit a paper to a journal, you choose people you'd like to review it.  This way there is no chance whatsoever of institutional bias.


Then, last year, Ekman was invited by Pfizer to give a talk. While there, he was handed a version of the very manuscript Reuben had asked him to review, which had subsequently been published in Anesthesia & Analgesia. To his surprise, and horror, he was listed as a co-author: Reuben had forged his signature on the submission form, Ekman says. 
If a tree falls in the forest, and no one hears it, does that mean everyone is deaf?

Ekman is an academic doctor who also speaks for Pfizer-- in other words, he's a "thought leader."    Does he not read the journals?  Do none of his friends read the journals, to send him an email, "hey, saw your study..."  If he's not reading them, is anyone else?

What is the impact of a fake study that no one reads?  Think about it for a moment, here's my answer: volume.  "There is some evidence that..."  It doesn't matter if the study is wrong, or silly, or fake; just as long as it exists, to be able to get away with that sentence.


You think I'm kidding.

The faked article that Ekman reviewed and then "coauthored," purported to discover that Celebrex helped reduce pain, and consequently promoted more vigorous participation in rehab.

It also found that patients used a lot less oxycodone:

celebrex reuben.JPG

Casual readers of this blog will perhaps be thinking, "hmm, 20mg less oxycodone, does that seem possible?"  But astute readers of this blog will ask the larger question:  where the hell did you find the graph?  It's not still on PubMed, is it?

Oh, yes.  It's also free direct from the journal.    Here's a follow-up study, also fake, also free.

"But surely the article is labeled as fake, or something?"  Something being like a big red watermark that says, "RETRACTED"?   No.  The only hint is at the bottom, under "This article has been cited by other articles:" and one of the 12 citing articles is the retraction notice by the editor.

It's hard to explain why the article isn't simply deleted; or, better, loudly labeled as a fake so that we can learn even from fakes.

But it is very easy, very very easy, to explain what will happen to a large number of doctors who never read journals, let alone Scientific American, who may perhaps be motivated just to see how many articles on Celebrex there are in the world-- "wow, that's a lot"-- or read a review article that cites these articles-- reviews that will never find a forum for retraction or explanation.

It's even easier-- sit down for this one-- to explain what will happen to new research on  Celebrex-- "does this thing work, or not?"-- now that Pfizer has absolutely no incentive to pay for it.

Good luck.