What's Wrong With Research In Psychiatry?
Apart from the high fives, bravado, and binge alcoholism.
Dynastic:
There are no independent psychiatric researchers. Young academics are mentored by older academics; this isn't optional, for either person. In virtually no circumstance do they study something entirely of their own choosing, it is either an outgrowth of the mentors' research, or is the mentors' research.
Distant from reality:
Young academics almost never work outside of the university. Theirs is all selection bias. The only patients they see are the ones the university gives them: either Medicaid/indigent on the inpatient unit, or patients of the disposition to want to be involved in clinical trials. Academics are like government economists: "we haven't had two consecutive quarters of declining GDP, so we're not in a recession." Regular psychiatrists are the management at Wal Mart: "I'm not sure what this is called, but no one is buying anything."
Groupthink:
Academic psychiatrists are nearly all on the same page, and refer to one another as if they have a relationship, even when they've never met. ("Chuck Nemeroff is doing some good work on...") It's pointless to list the other characteristics of groupthink here, except to highlight one: the purpose of groupthink is not to promote an ideology, but self-preservation, and this is unconscious. They don't realize that their lives are devoted to preserving the group, yet young researchers are brought on who connect with the group; peer reviewers-- and journal editors-- come from the group; grant reviewers, and NIMH people themselves came from, and support the group.
An example of groupthink preservation is the referencing of studies. Academics support their propositions with previous studies; however, no one checks the accuracy of these studies. No one has the time, and the group necessarily must trust the work of others in the group. Even if an error were to be found, it would be described as an isolated error. A cursory stroll through this site alone suggests just how "isolated" such errors really are.
Financially isolated:
Medicine is a market. Buyers and sellers.
Academics make a salary, but their survival at university depends on the grants they can bring in. That means their market, their customers, are funding agencies, not patients. It doesn't mean they don't care about patients, it means that the service they provide is nuanced and directed towards Pharma or the NIH.
If the funding agencies are stacked with people who like antiepileptics for bipolar; if the grant goes to Pfizer who is looking to create a bipolar indication, etc, etc, that's the research that can be expected. I'm not even worried that the results will be... predestined. I'm worried that such pressures direct what kind of research, what kind of questions get asked, at all.
Too much data
We're busy talking about bias and hidden results and skewed statistics and nonsense. So we call for more studies, as if they will somehow be better studies, despite no other structural changes being made. The reality is that we have information inflation: new studies have less value because they get lost; and old studies completely disappear, as if somehow their validity is temporal.
There are a quadrillion studies already conducted in psychiatry. There is plenty of data that can be analyzed, meta-analyzed, pooled, parsed. If all current research ground to an immediate halt, and researchers just looked back at what we already have, we would save billions of dollars in future research and future bad treatments, and we would learn so much.
Outcomes Research Is Purposefully Avoided, or Ignored:
You might think in a field with nothing but outcome studies (e.g. Prozac vs. placebo) I might not be able to make this claim, but I do.
Most studies are short term. The few long term studies that exist (e.g. Depakote for maintenance) are either equivocal (e.g. Depakote for maintenance) or show no efficacy (e.g. Depakote for maintenance.) And they are ignored.
But these outcomes are distractions. The question isn't is Depakote good for maintenance bipolar. The question is, is there any value to the diagnosis of bipolar? In other words, if you called it anxiety, or personality disorder, or anything else, and then treated them ad lib, would the outcome be different? Is there value to the DSM? You might argue the diagnosis leads us to the treatment, but in most cases, meds are used across all diagnoses, and more often than not a diagnosis is created to justify the medication.
Are hospitals valuable? You would think that by now we'd have a clear answer to this, the most expensive of maneuvers. I can say, however, that reducing the length of stay from several months to 5-7 days has not affected the suicide rate. I'm not saying they are or are not valuable, I am saying that I don't know-- and that's the problem. It is 2008 and there are more studies on restless leg syndrome then there are on hospital vs. placebo. You know why? See above.
Are one hour sessions associated with better outcomes than 2 minute med checks? I know 2 minute med checks sound bad, what I want to know is if they are actually bad. Higher suicide rates? More days absent from work? More divorce? More sadness?
The system is completely ad hoc, with each party yelling loudly to protect their fiefdom. It allows everyone to declare themselves an expert without having to prove it. Tell a Depakote academic you're suspicious about the utility of the drug, and he won't tell you you're wrong, he will tell you you don't understand. Try it. He will evade the existing data ("not enough people," "studies are difficult to conduct," "we know from clinical experience," "more work is needed") and rely on appeal to authority. Appeal to authority is the signal you're being bullshitted.
Outcomes research will never be conducted in psychiatry because its existence depends on not knowing the answers. It will eventually be conducted on psychiatry. You can't tell you're an idiot, someone has to tell you.
Digg | 
del.icio.us | 
StumbleUpon | 
Reddit | 
Yahoo! | 
My favourites
May 23, 2008 1:06 PM | Posted by : | Reply
The points you raise are why I like to use case studies to point out off-label uses or wacky side effects, and European and Canadian studies when giving the readers of my site an idea of how well a med might work and/or comparing the efficacy of two or more medications.
With case studies it's often a doctor whose has found something that might be of interest or concern to the larger community of doctors. Assuming these doctors actually read the journals piled up in their offices. In the case of European studies, their use of active placebos within the framework of state-sponsored health care (for both European and Canadian studies) that cuts down on the number of people who will sign up for anything to get some form of medical attention or cash strikes me as having more meaningful data from a more meaningful population.
Misdiagnoses are always possible, but with somewhat less of a profit motive involved in both instances, one can still find some worthwhile studies, it just takes more digging.
Score: 0 (0 votes cast)
May 23, 2008 3:24 PM | Posted by : | Reply
These are all valid comments. But what I'd like to know from you is what is RIGHT about academic medicine. I say this, because, frankly, you sound like you have a personal ax to grind -- it's easy to tear down other people (and they might well deserve it) but it's much harder to offer a practical solution that is better. For instance, just how would we avoid the mentor problem you mention? Would it really be wise for junior researchers to just do whatever they wanted without any guidance? Would junior folks want to do that and would they produce better results? I'm not so sure. Besides, this is hardly unique to medicine after all - just about all areas of academia suffer from the ills you describe.
Score: -4 (4 votes cast)
May 25, 2008 12:30 PM | Posted by : | Reply
How to improve the quality of psychiatric research?
It's of course important to test mental health hypotheses using rigorous quantitative measures. But as you've suggested, much of the bias in current research actually comes during the hypothesis-generation stage: A relatively small number of well-connected researchers get the funding and authority to explore their favorite hypotheses, the ones they personally find the most interesting or financially rewarding. A quantitative study may measure exactly what it was designed to measure, while also failing to detect that there are a few large elephants in the room.
A vast reservoir of alternative intriguing ideas and hypotheses can be unearthed by actually talking in greater depth with the people who experience these mental health difficulties personally, and asking them open-ended questions about what they've learned. Find out what different people have found helpful, and what they have found unhelpful or even harmful. More qualitative descriptive studies are needed to explore people's life experiences, symptoms, obstacles, personal coping strategies, and more. Patterns will emerge that will lead to new and exciting hypotheses.
To learn how to better help people recover from schizophrenia, first find out why the recovery rates for schizophrenia are better in developing countries than developed countries (according to two WHO studies). Is it better social support? If so, do we need to radically change our priorities and strategies for social support? Is it that medication in the long term for some hinders rather than helps recovery? There are studies supporting that possibility.
To learn how to better help people recover from depression, or childhood bipolar, autism spectrum, etc., first find out why the diagnoses of these have all been skyrocketing in the past couple of decades. How much of it is due to increasingly expansive diagnostic criteria? How much of it is due to better detection? How much is due to overt or inadvertent research bias due to drug company funding? How much is because more people are actually having more or worse symptoms than those 30 years ago? Is there a real increase? If so, why? This would be an urgent public health matter. What would people 30 years ago with these same symptoms have done? Where were they? Why didn't we hear more about them then? Is there something in our diets that is increasing the incidence of these? Something in our environment? Our increasingly in-your-face sensory overload multitasking culture? Less solid family or social structures?
To learn how psychiatry can better help people in general, first find out why there are so many intelligent thoughtful people who feel strongly that psychiatry has significantly harmed them, and have actively turned away from psychiatry to look for other alternatives. Learn more from the perspectives of people such as those who contributed to the report "From Privileges to Rights: People Labeled with Psychiatric Disabilities Speak for Themselves" in 2000 by the National Council on Disability, or have formed numerous organizations such as the Icarus Project, World Network of Users and Survivors of Psychiatry, or MindFreedom International (which is recognized by the United Nations as a human rights NGO with Consultative Roster Status).
When looking for statistical correlations between genetic or neurological findings and mental health problems, skip the DSM diagnoses altogether and look directly for correlations with individual symptoms. DSM-defined "disorders" are an imposed artificial construct ("Committee-Created Common Criteria Clusters", based on inadequate past science) that serve only to blur and overgeneralize the picture. (For example, there are 256 possible symptom permutations that all get reduced to the one oversimplistic label "Borderline Personality Disorder".) A particular genetic or neurological mechanism might be expressed as aspects of different DSM "disorders" in different people, and conversely a variety of different genetic or neurological mechanisms might result in similar-looking manifestations leading to the same DSM "disorder". Correlating directly with symptoms instead of DSM "disorders" at least removes a level of unnecessary and often-misleading abstraction.
Consider hypotheses that relate to emotionally healthy environments and development, as well as particular genetic or neurological findings. Consider the common role that stress, stressful environments, and stress responses may play in numerous mental health problems.
When exploring "theory of mind" hypotheses for autism spectrum disorders, consider that we all approach "theory of mind" toward others based on our own internal experiences and assumptions. It goes both ways: Some writings by adults with autism have expressed amazement at the lack of "theory of mind" on the part of researchers and clinicians toward them! Consider the "Sally-Anne" theory of mind test: The subject usually recognizes this is all staged, so who's to say what "Sally" is thinking when it's clear that Sally and Anne are putting this on together? Consider emotion-facial-recognition tests: Were the people photographed actually feeling those emotions? Were they asked what emotion they were feeling at the time? Or were they merely "acting" the emotion? Whose answer is then more correct?
Addressing these questions, and others, is needed for psychiatric research to make some new headway.
Score: 3 (3 votes cast)
May 26, 2008 8:58 AM | Posted by : | Reply
There is a lot of truth in the points that you raise. Fortunately it is only a small part of "psychiatric research". The real nuts and bolts of such research is done in labs under the title neurophysiology (or others). You would be hard pressed to state that people like Fred Gage are not doing highly relevant and unbiased research.
Score: -1 (1 votes cast)
May 27, 2008 4:30 PM | Posted by : | Reply
Do you have any links to the WHO study. I use this as an example in Recovery Training but I have tried finding the original study and no-one can help.
I have a long standing history of MH problems namely Schizophrenia which I feel has had a negative effect on my life for nearly 20 years. It was only by beginning to make sense of my 'psychosis' in the context of my life was I able to begin to find myself and my life again. I had to fight the MH system to do this and I'm glad I did but I wish that others didn't face the same disempowering and lack of adequate intensive support to move through their experiences and on with their lives.
Thanks.
ps I love this site for stimulating my thinking and debating my understandings.
Score: 0 (0 votes cast)
May 28, 2008 3:02 PM | Posted by : | Reply
Very interesting post. I'm the last thing from a mental-health professional (I'm a business journalist & Ph.D. student in history), but a lot of the biases you diagnose here are rampant in the fields I do study. I'm thinking of parallel after parallel from my own experience.
Score: 0 (0 votes cast)
May 30, 2008 10:19 PM | Posted by : | Reply
Les: I found references for the two WHO studies on schizophrenia recovery on the website for the book "Mad in America" by Robert Whitaker, about halfway down this page:
http://www.madinamerica.com/Mad%20In%20America/Antipsychotic%20Drugs%20and%20Chronic%20Illness.html
(I bought the book and haven't gotten around to reading it!)
Score: 0 (0 votes cast)
May 31, 2008 4:27 PM | Posted by : | Reply
Thanks Phil for the link it was very helpful. I followed links from there and eventually found scanned copies of the studies-fab!
http://www.abcmedsfree.com/schizophrenia.htm
Les
Score: 0 (0 votes cast)
June 12, 2008 4:08 PM | Posted by : | Reply
I have a Ph.D. in clinical psychology, but there are numerous overlaps in how our fields deal with these conflicts (in some areas of research I think there are better protections against groupthink than others, but still). The process of building a young professional's career on their mentor's old data, and the importance of collecting new data before the old is analyzed, is a necessary evil of our current system of almost all research positions in mental health being "soft money." Without continuously applying for new grants and collecting new research, a research psychologist cannot maintain their employment, which leaves little time to fully explore their own data. This winds up forcing graduate students into the awkward position of negotiating the boundaries of what the data state empirically, and the perspectives of an advisor/co-author who may be out of touch with the data b/c their time is devoted to writing for new grants, not observing their own findings.
I think there are additional biases in the research being published based on where funding is available, as well. As the collective pool dries up under this anti-science administration, drug company grants and positions are very enticing.... they're concerned about a new depakote vs. placebo study, but NOT research on the value of a particular diagnosis over the life span, social support as intervention, etc.
Score: 0 (0 votes cast)
Comments