July 9, 2008

Clinical Experience vs. Clinical Trials



In CNS Spectrums, Dr. Rosenheck takes Dr. Marder to task for his suggestions that CATIE results are limited and flawed, and clinical trials may not be better than clinical experience.

The article must be very important, because it is labeled as a Communique, yes, just like the one that called for the normalization of relations with China and decreasing arms sales to Taiwan, which brought us the Beijing Olympics 2008.   Thanks, Mr. Nixon!


He offers a detailed response to Marder's criticisms of CATIE, and quite effectively defends CATIE's results of "no health advantage for [atypicals] over older" drugs.

Many examples have emerged in recent years of costly and painful treatments that were deemed on the basis of clinical experience to be self-evidently effective--until double-blind studies showed them just as clearly not to be.16,17

What's awesome about that sentence is that the references are books about alternative medicine and why we make irrational decisions.  Is that really the best he could come up with?  Why not mention, say, Depakote?

The bias-- for him and most du jour psychiatrists-- is that research shows that new things are not as good as we thought, and we should change our behavior accordingly.  What about when research shows that they are better than we think?  Does that change our behavior?  CATIE showed that Geodon worked as well, but caused less weight gain than the others.  But I'd wager good money Rosenheck prescribes very little Geodon in comparison to other atypicals.  So much for science.

While clinical experience must guide care of individual patients and the development of public policy it can not replace experimental research as the fundamental source of legitimacy of clinical medicine, and within it, of the profession of psychiatry.

That's not fair, let alone rigorous.  You're taking the research and coloring it with your own "experience." 

Here's an example: CATIE found that the newer drugs were no better/worse than the older typical agents.  Ok: so why can't I just use only the newer meds?  Why does CATIE mean I should go back to typicals?

Oh: cost.  Was cost a relevant factor in the CATIE analysis?  I missed that page.  And are you asking doctors to actually consider cost as a factor in prescribing?  Why?  What part of their awesome medical training allows them to make economic and policy risk/benefit analyses?  They are not all that good at medical risk/benefit analyses, I should remind you.

And if we all agree that doctors should consider cost, then shouldn't they consider the whole cost?  Not just Geodon's cost, but it's impact on hospitalization, on polypharmacy, food costs, cost of (not) needing a cardiologist, etc.  Fortunately all these things were explored fully in CATIE.  (/sarcasm)

In other words: if doctors are going to be thinking about money, they should be handling the budget.  Every patient gets $20/d pharmacy budget.  Go.

One final observation: Rosenheck derides clinical experience, but what he cannot see is that, as an academic psychiatrist who primarily does research, studies are his clinical experience.  As suggested above, he does not apply them formally or objectively, but instead picks and chooses what studies or results he thinks are important based on an internal barometer that has nothing really to do with science (e.g. Geodon, above.)   

In other words, the application of clinical experience and the application of clinical trials are flawed in the exact same way: strongly subject to selection and confirmation bias.




3 Comments