March 4, 2009

Impulsivity In Kindergarten (Does Not) Predict Future Gambling

Here's how (not) to read a study.




Most people read this (Reuters):

Distracted kindergarteners [sic] become gamblers: study

or perhaps the actual study:

Results  A 1-unit increase in kindergarten impulsivity corresponded to a 25% increase in later self-reported child involvement in gambling (SE = .02). This was above and beyond potential child- and family-related confounds, including parental gambling.

and ask critically, "well, I wonder how they define impulsivity?"  or  "are there other variables that could explain why distractiveness or impulsivity predict gambling?"

That's not where the money is: we can confidently assume psychiatrists have plenty of measures for impulsivity.  The real question is whether psychiatrists know anything about gambling. 

Typically, people question the causes of pathology; I want to know, what are they labeling as pathological nowadays? 

So, I check:

cards; bingo; bought lottery, instant lottery (scratch and win), or sports lottery tickets; played video games for money or video poker (eg, at arcades); and placed bets at sports venues with friends or on games that require skill (eg, billiards, pool, bowling)

Let's all agree that this is precisely not at all what anybody thinks about when they think about gambling.  You might say, "well, it still counts!" but it's hard to say you can predict X when everyone thinks X means something different from what you do.

Or another way of saying it: even if the results are accurate, why should we care what predicts playing?

The answer is, we shouldn't-- and neither do the authors.

 II.

Psychiatrists, by which I mean residents, who actually spend any time reading journal  articles often despair.  They hit something like this:

The equation also controls for child (sex and early emotionally distressed behavior) and family (maternal education and family dysfunction) factors. Our results bear on this fully controlled model: CGBi6GR = a1 + β1 IMPiKE + β2 PGi6GR + {gamma}1 CHILDiKE + {gamma}2 FAMILYiKE + eit, where a and e represent the intercept and stochastic error, respectively, CGB indicates sixth-grade gambling behavior (6GR), IMP indicates kindergarten impulsivity, KE indicates kindergarten entry, and PG indicates parental gambling involvement, for each individual i.

and succumb to  self-doubt.  "I don't know enough to interpret studies, I guess.  How am I  going to learn this stuff?"  But while wondering if this study was a good one or not,  something sneaky happens to them: the premise is absorbed, implicitly.  They don't realize they've just been infected by a mind virus.  "It's so hard to tell if this study really showed what we all know to be true...." 

P values and ANOVA are red herrings.    The single most important question to ask when reading any scientific paper, ever, anywhere, is this question only, pay attention: what do the authors want to be true?

Nothing else matters.  If you think there's a scientist somewhere publishing a paper that says, "son of a gun, I've devoted my life to this question and it turns out I've been wrong all along" you are beyond a crack pipe.  Even if the research comes up negative, the scientist's thoughts inevitably turns to, "that study was flawed."  The premise is never questioned.

Don't be fooled: "well, in our lab we looked at the serotonin receptor, and it didn't pan out. So we accepted that we were wrong, and changed focus to the dopamine receptor."  Do I need to tell you that that is the exact same thing?

III.

The mistake is thinking the purpose of the study is the outcome of the study.  The issue is not problem gambling, otherwise we would be limiting it to people with a gambling problem.  The issue is not trying to predict which child will become a gambler.  The issue is ADHD:

"Attention problems are a public health issue," she said.
That's what the study is about.  The gambling is just another excuse to say ADHD needs treatment.

IV.

Once you ask the key question, you'll know exactly where to look for inanities.  The Introduction is usually full of them.

Data suggest that in most cases, youthful recreational gambling predates pathological gambling in adulthood. (1,2,8,9)

The authors cite four studies for this premise. Usually, a reference means a primary source that has direct data about the issue.   I know without looking that none of those four references are that.  I know this not because I know anything about gambling or pediatric studies, but because I know that if they're going to count bingo and bowling as gambling, they're not going to be too careful with their references in the Introduction.  "But it's stuff everyone already knows."  My point exactly.

NB: I'm not saying they knowingly pick non-supportive references.  They absolutely think those references are supportive.  They're caught in a self-reinforcing groupthink.

To illustrate my point, I looked up the four references.  Only reference 9 had direct data on  how many adult gamblers started as kids.   The other three references (1,2,8)  were about  related issues (e.g. psychiatric comorbidities in gambling).  They simply asserted the child-adult link in their respective Introductions.  They, too, referenced their assertions: references 2 and 8 both used the same other reference to support that claim-- reference 9!  Reference 1 and 2 both cited the same author, who wrote in two different venues.

Reference 9 found that 20-30% of adult pathological gamblers began "wagering" before age 15, as defined by 10 types of gambling, including "stockmarket, sports, and games of skill."  Not only is this not really the definition the average person uses for gambling, it also isn't the definition used in this kindergarten study. 

V.

I want to emphasize that studies are never about discovery, they are about confirmation. Researchers already know what they want to find.  This is why I have long argued that financial biases in research are trivial, if for no other reason than they are obvious.  Ideological bias, institutional bias, personal bias-- these are far more difficult to detect.  There's no disclosure for them.  They matter much more.  If a researcher's son is on Depakote, how will he interpret a study he does that finds Depakote doesn't work?  Would it really matter at all whether  Abbott paid him or not?

None of this means that research is corrupt, or that it isn't valuable.   But it does mean that you can't accept conclusions as written, and the Discussion section is basically an op-ed.

Nothing wrong with that either, except that everyone thinks it's journalism.

















Comments

Reminds me of a post from s... (Below threshold)

March 4, 2009 4:40 PM | Posted by Aaron Davies: | Reply

Reminds me of a post from several years back by another blogger I'm rather fond of about psychological rent seeking (relevance is to most studies' ultimate goal's being to make researchers more important, not to confirmation bias per se).

Vote up Vote down Report this comment Score: 0 (0 votes cast)
Since I agree completely wi... (Below threshold)

March 5, 2009 6:06 AM | Posted by varangianguard: | Reply

Since I agree completely with your observations and conclusions, great post!

Vote up Vote down Report this comment Score: 1 (1 votes cast)
I enjoy this sort of post f... (Below threshold)

March 5, 2009 11:05 PM | Posted by Carlton: | Reply

I enjoy this sort of post far more than those that theorize about the sources of our cultural habits and narcissisms.

Vote up Vote down Report this comment Score: 0 (0 votes cast)
Do you think there's some f... (Below threshold)

March 6, 2009 10:14 AM | Posted by Anonymous: | Reply

Do you think there's some fault here in the **anonymous** peer review system?

Vote up Vote down Report this comment Score: 1 (1 votes cast)
I'm going to take this and ... (Below threshold)

March 6, 2009 3:55 PM | Posted by La BellaDonna: | Reply

I'm going to take this and run it through my personal filter. Based on my own personal experience, Impulsivity In Kindergarten Does Not Predict Future Gambling.

In kindergarten, I was an impulsive child. Without a doubt, I was impulsive. Now, more than forty years later, not only am I not a gambler, I have not been a gambler for all that time. There might have been a maximum of two (2) occasions in all that time in which I agreed to wager a nickel on the outcome of a competition and/or dispute of some sort - but only because I already knew the outcome/answer. In other words, no gambling actually took place.

I'm starting to think that an awful lot of people run studies, and then look for a hypothesis onto which they can tack the studies when they're done.

Vote up Vote down Report this comment Score: 0 (0 votes cast)
This study is refuted by th... (Below threshold)

March 9, 2009 12:33 AM | Posted by Common Reader: | Reply

This study is refuted by the existence of the Chinese people, both studious and gambleous.

Vote up Vote down Report this comment Score: 1 (1 votes cast)
TLP = Anno. How could we ha... (Below threshold)

November 16, 2013 7:40 PM | Posted by Anonymous: | Reply

TLP = Anno. How could we have all missed it.

Vote up Vote down Report this comment Score: 0 (0 votes cast)
What are you talking about,... (Below threshold)

January 9, 2015 11:18 AM | Posted by Atarii: | Reply

What are you talking about, Alone? Don't you know that these studies are peer-reviewed? Don't you know that peer-reviewed journals are ALWAYS cited honestly, intelligently, and without any bias or manipulation?
Know you not that scientists are impartial, unbiased, egalitarian beings beyond reproach and above temptation?

This article is nonsense. Imagine! Questioning scientists!

Vote up Vote down Report this comment Score: 1 (1 votes cast)