April 29, 2014

The Maintenance Of Certification Exam As Fetish

Jeorg_Breu_Elder_A_Question_to_a_Mintmaker_c1500.jpg

no need to wait for the receipt

(I had reworked an old post for a psychiatry trade journal, which I would happily have linked you to, except that page 2 is behind a login wall. So here is the version I submitted before the editors edited it, slightly longer with more typos. I am posting this because of the new lawsuit against the American Board of Medical Specialties.)

The mission of the American Board of Psychiatry and Neurology's Maintenance of Certification (MOC) Program is

to advance the clinical practice of psychiatry and neurology by promoting the highest evidence-based guidelines and standards to ensure excellence in all areas of care and practice improvement.

That's what the website says, I have no reason to believe they are not earnest. But far from succeeding, the program does the exact opposite. We have come to a moment of truth in psychiatry, and we are all going to fail. By which I mean pass.

We can start with the 200 question certification exam. The most obvious clue that there was something suspicious going on with the test was that there were no questions about Xanax. How do you measure "excellence in all areas of care and practice" without asking about the most commonly prescribed medication in America, let alone psychiatry? Meanwhile there were several questions about pimozide, a medication which appears to be prescribed exclusively by psychiatrists who want to brag about prescribing it. I was repeatedly assessed on my competence in Dialectical Behavioral Therapy, but was not asked to display my knowledge of SSI. You might retort that SSI isn't really psychiatry, but then why is so much of my time spent on it? The only thing I spend more time on is Xanax.

But though the missing Xanax was a clue, the insidious problem with the exam was not the content. To see the bad faith obscured by the questions, put aside the usual college freshman complaints of, "why do we need to know about pimozide?" and ask instead, "what happens if I get the question wrong? What happens if I get them all wrong?" The answer is nothing. There are no consequences for failing this test, at all. First, 99% of the applicants pass, I assume the other 1% forgot to bring two forms of ID. Second, even if you fail, you can take it again and again, as many times as you feel it's worth the $1500. Third: there were a thousand easy ways to cheat, here are three: I could have walked out of the building on an unsupervised "break"; I could have Godfathered an ipad to the back of a toilet; or I could just picked up the phone and called everyone. Who was going to stop me? There is more security at a pregnancy test, which made me wonder if how easy it was to cheat wasn't... on purpose. The retort is that doctors are expected to behave honorably, but the honorable ones were going to pass anyway. Those in danger of failing-- the very people the test should detect-- would be most tempted to cheat. Doesn't the ease of cheating render the test unreliable? If the test is unreliable and 99% pass, why have a test at all? Which reveals the gimmick: the point of the test isn't to measure competence, but to convey the impression that competence was measured. The point of the test is to say that a test was given-- and nothing else.

The question is, to whom are we saying this? It is as if psychiatry was in denial about its ordinary reality and was trying to create a different identity through the test itself. A psychiatry where there are right and wrong answers. Where pimozide and Dialectical Behavioral Therapy happens, a lot. Let me anticipate your retorts: that the questions are carefully constructed for their validity; that the test itself "incentivizes" learning; that not everyone prescribes Xanax; that if I'm such a smartypants, what system would I use? If these are your replies, you have missed my point: a flawed system isn't better than no system at all, it is worse than no system at all, because at least with no system we are forced to be accountable to ourselves for our education. "Not everyone will be so dedicated." Correct, but now those same undedicated people get an official blessing of their ignorance. Who doesn't walk out of even this ridiculously meaningless exam not feeling smart, accomplished, up to date? And who would dare, after passing, to criticize the exam that warmed his ego?

In addition to the test, the Board also requires a nauseating number of CME credits, but these CMEs are an even worse affront to learning. The only thing that CMEs guarantee is that money was spent on buying them, $80 and no questions asked is all it takes, which is even sillier than it sounds since I could go to a number of websites which offer instant and unlimited free CMEs, so long as I skip the long text and just take the post-test, which I can take as many times as I want. I can get 1 CME every 25-50 seconds, depending on my ability to click "b".

The retort is that the system is predicated on a certain level of honor, that physicians shouldn't cheat. Fair enough, but if you're trusting them to be honest in revealing what they learn, why not simply trust that they're going to learn it? Because the point isn't the education. The CME exists to say that there is CME; the CME exists to say there is oversight.

To clarify: the important criticism here is not that the multimillion dollar CME industry is a gigantic money making scam, something on the level of the 15th century sale of indulgences, because to say that would be actually to defend that very system: the money is a diversion, a patsy, what is corrupt about CME isn't the money but, as the default mechanism for continuing education, it subverts its own purpose. It reduces the interest in actual education so that it can pretend that it explicitly monitors it. If you have a minute to spend on your "education," the system pushes you towards CME. "Why not do both?" Why do both, who can do both? There are only 24 hours in a day. In other words, the system doesn't just fail, it forces failure.

Last year there was a large cheating scandal at Harvard, over a hundred students were accused of plagiarism in a government class, and amidst the usual self-aggrandizing criticisms of the college kids as entitled, lazy, or stupid, what no one wondered is why, in an introductory survey course predicated on institutionalized grade inflation and no wrong answers, did the students feel compelled to cheat when they were all going to get As anyway? The terrifying answer is that they weren't cheating to get the right answer, there was no right answer, they were forced to cheat to concoct the answer the professor wanted-- because that's the system. Meanwhile, while they were spending their time "cheating", what real learning could be done? None. So--- why bother with an exam at all? Why not just offer the course and give everyone an A anyway? Because the purpose of the test is to say a test was given, to prove to some hypothetically gullible entity that learning occurred-- and to prove it to ourselves. Which is why our reflex was to criticize the kids, not the system: we are products of that system, to criticize the reliability, let alone validity, of that system would be to open ourselves to scrutiny, to deprive us of a core part of our own identity. "Things were a lot more rigorous when I went to college." First of all, they weren't. Second, even if they were, why, when you got to be in charge, did you change the system to this?

Seen this way, these tests, whether Harvard government exams or MOC exams, are nothing more than fetishes: a substitute for something missing which saves us from confronting the full impact of its absence. In less abstract terms, these tests allow us to believe NOT that we learned something, NOT that we know something-- but that there is something to know. Since there is nothing new to learn, therefore there must be a test. The logic of a 10 year MOC exam is to keep us up to date, so it's fair to ask: what in psychiatry has changed in ten years, what are the major advances? Depakote was discovered to be the default maintenance mood stabilizer despite no evidence supporting this, but that fell into disuse at a time oddly coinciding with its patent expiration, which is suspicious but I'm no epidemiologist. Anyway, it wasn't on the test. Anything else? A few new medicines have come out, though none of them appeared on the test either. There's money to be made on the west coast using giant magnets, (fortunately) also not on the test. So? Was the ABPN worried I'd forget how to use MAOIs? I'm never going to use them, I have enough problems monitoring Xanax. The astonishing truth is that despite millions of dollars and hundreds of academic careers psychiatry has made no progress in almost 20 years, let alone ten, a claim no other medical specialty can make, and the truth which cannot be spoken out loud. Hence an exam.

Are you prepared to look inside yourself? When a nurse practitioner asks you what about your board exam is difficult, what will you say? Take a minute, it's important. "Well, it has neurology in it." Note carefully that the psychiatry questions aren't "harder," the appeal here isn't to a higher level of expertise in psychiatry, but an expertise in something else, something "more" than psychiatry, and it is this link that symbolizes our status as "experts." Older psychiatrists will be quick to assert that "clinical judgment" counts for a lot, and I don't disagree, but it's probably not testable, and it most certainly wasn't tested. So what does $1500 buy you? "Existential support." I hope it was worth it.

What makes the MOC not just a bad exam but evidence of a pathology is that though college kids have no idea what they're up against, that the system works against their education, psychiatry is the very discipline that articulated these defense mechanisms. It should know better, it is supposed to know better; which means that we are either unable to see what we are doing or believe that we are somehow exempt from this. But here we are, spending time and money on cosmetics and pageantry to pretend that we are learning, to pretend that we are being measured, all the while slinging random neurochemicals + Xanax based on an a suspect but billable logic in the hope that something sticks and no one notices. Frantic activity as a defense against impotence. There is a term for that, but you can bet your career it won't be on the test. Pass.






237 Comments