August 1, 2010

How Do You Lose Weight? Which Diet Is The Best?

christian bale.jpg

There are a lot of studies about diets and even more studies about weight loss drugs.  So why can't we decide which is the best diet?

There's a stack of studies a meter high.  "Look," they say, "the evidence is pretty strong that they are all the same."  Really?  That seems intuitively wrong-- shouldn't some diets be better for certain people?  And yet, there's the stack.

This post won't tell you which is the best diet.  You can go back to now.

This post is about why the studies can't tell you this.


Most clinical trials have high dropout rates.  What do you do with them?

1.  You could use only the data of people who finish the trial, so you can say, "it had a 100% cure rate for the 13 people that didn't explode."

2.  You could leave the last recorded score intact, and just carry it to the end. 

Clinical trials have usually employed #2: LOCF: the last observation carried forward.

Researchers regularly remind us that this type of analysis can potentially underestimate the effect of a treatment.  If a patient drops out because of side effects after one day his score will remain "sick", bringing the overall average down.  Perhaps had he stayed on the medicine, he may have been cured.

Example: consider N=2.  One person gets cured, the other person dropped out on day 1 (no improvement.)  So the study found that Drug A gets you 50% improvement. Unhelpful.


Here is an unrealistic example for the purposes of illustration.  Take a bunch of 300 lb individuals.  I give half a 100 calorie diet, and the other half a 5000 calorie diet.  Read it again.  Which will result in more weight loss? 

After one day, most of the 100 calorie diet group drop out-- "this sucks"-- and so their last weight carried forward is 300 lbs.  Average weight loss at the end= 0 lbs.  If the 5000 calorie people can stick to the donut-ham-hamburger diet and lose even a single pound, the study would conclude that the 5000 calorie diet resulted in more weight loss.

The study isn't useless, because it tells you something very important: across a population, the 100 calorie diet is going to fail-- most people like dessert.

But that doesn't tell you what to do.  This study does not help with that decision at all.  In fact, it may confuse you because now you are confronted with the evidence that the 5000 calorie diet is better, or at least not worse.  Bacon up.

Studies can be further misapplied  when they (read: media) overgeneralize the results.   Are all obese people obese by the same mechanism? 

Point: in an LOCF study, dropouts don't just minimize the importance of the study, they ruin  the study if all you are doing is looking at the primary outcome.  Point 2: you can't take the results of an LOCF study and simply apply it to an individual in front of you.  You have to consider the contex.  "Most of the people who dropped out did so because of X.  Is this likely for my current patient?"


Now consider an FDA trial of an appetite suppressant.

Lorcaserin is a new 5HT2C agonist (think opposite of Remeron) that theoretically promotes satiety and/or suppresses appetite. What happens when you give it vs. placebo to a bunch of 100kg people who are told to exercise and eat 600 less calories a day, for a year?

lorcaserin 1 year weight loss.jpg
You see that the placebo group lost an average 2.2 kg, which was 2% of their baseline body weight, while the lorcaserin people lost about 5.8 kg, which is 5.8% of their body weight.

Is that 3.5 kg difference meaningful?  No: according to the FDA, you have to beat placebo by 5% or you don't get FDA approval.  To the FDA, this failed a primary efficacy measure.

However, in theory, the drug should work in certain kinds of people-- maybe those whose obesity is a function of hunger?  How many people were able to lose significant weight, like 10kg?  Answer: three times more than with dieting alone.

lorcaserin significant weight loss.jpg
Is this a drug you'd be willing to try?  There is a group of people for whom the drug might be awesome-- if you could predict who those people were.

The study becomes difficult to interpret because 50% of the people dropped out.  When did they drop out?  Doesn't say.    But if 25% of them had dropped out by the fourth month (4kg or less weight loss)-- let alone earlier-- the rest of the people would have had to have lost about 7kg in order to generate an overall average of 5.8kg of loss-- and those guys would have met the required 5% superiority required by the FDA. 

I'm not saying that happened (or didn't happen.)  I'm saying that for the purposes of practicing medicine, you cannot say "studies show this drug works/fails" without an understanding of why it worked/failed.


Now take the Atkins diet: is it better than conventional "low fat" diets?  Let's ask the gated community socialists at the New England Journal of Mendacity:

(from N Engl J Med 2003; 348:2082-2090)

atkins diet locf.png

You can see that at 6 months, the Atkins diet people lost more weight.

But, it's LOCF: by month 3, 30% of the conventional diet people bailed, vs. only 15% of the Atkins.  If you assume that very little weight loss went on in the first three months, then the weights for the conventional diet will appear heavier than they could have been, dragged "up" by the dropouts who lost no weight because they didn't stick to it.

We won't know what could have happened if all of those conventional dieters stuck to the plan.  This isn't to say Atkins didn't work; it is to say that it may not have been better.

Analyze the data a different way.  Instead of using the morally weak quitter's last score to carry forward, the authors reverted to their baseline score (e.g. no weight loss), no matter how much weight they had lost in their brief time in the study. 

If you take the curve using this analysis (B above) and compare to the LOCF (A below), then there are three possibilities:

If they initially lost a lot of weight, then this analysis would "artificially" worsen the curve (i.e. make it appear like there was no weight loss.)  A curve would be higher than B curve. 

If they had magically gained weight before dropping out, then this analysis would hide that fact and the B curve would appear lower. 

If my assumption is correct-- that they didn't lose much weight in those early months, then the curves should be the same.

atkins diet baseline forward.pngNote that for the conventional diet, the curves are almost the same: they didn't lose much weight, and they dropped out.  The effect at month 6, therefore, is to make the overall weight loss of the conventional group appear less. 

In other words, conventional diets may not be as good; or they may better.  The same can be said about Atkins, which is to say, nothing can be said at all.

The point here is about the studies showing Atkins is superior: they really mean only that more people stick to it. 


Now to the meat of the issue.  What about all the studies that show that the diets are the same?  Surely those aren't flawed?

Let's find out if the percentage of fats, carbs, and protein matter for weight loss.  Let's pull a major study from the stack, something from the NEJM:

800 people put on various diets: high/low protein, high/low fat, and a range of carbs, e.g., 

20% fat, 15% protein, 65% carb
20% fat, 25% protein, 55% carb
40% fat, 15% protein, 45% carb
40% fat, 25% protein, 35% carb

Check back at the midterm elections.  Which was the best?

macronutrient composition all.jpg
macruntrient diet by time.png

From the Discussion:


In this population-based trial, participants were assigned to and taught about diets that emphasized different contents of carbohydrates, fat, and protein and were given reinforcement for 2 years through group and individual sessions. The principal finding is that the diets were equally successful in promoting clinically meaningful weight loss and the maintenance of weight loss over the course of 2 years. Satiety, hunger, satisfaction with the diet, and attendance at group sessions were similar for all diets.

Or, from the abstract:

macruntrient diet conclusions.png
All diets resulted in the same weight loss!  This proves it!  Oh, wait, this was published in NEJM, where peer review= spell check.  Better look more closely.

Though patients were told to eat a high fat (40%) vs. low fat (20%) diet, using a fixed protein amount, here's what they actually ate:

high-low fat diet.png 

That difference of 20% has been reduced to a difference of 7%, i.e. what should have been a difference of 33g of fat is now a difference of 11g.

What about high (25%) vs. low (15%) protein diets?

high-low protein diet.png
That 15% vs. 25% difference in average vs. high protein diets has been reduced to no difference whatsoever.  In fact, these people all managed to eat 20% protein no matter what diet they were supposed to be on.

So this study did not test various diets against one another; it tested essentially the same diet four times.  And it found that pretending to be on a high/low protein/fat diet has very little effect on the outcome, which if written that way would never made it into Children's Highlights, let alone NEJM.

Strangely, that's not the finding reported in the media-- or even by the lead author himself:

sacks interview.PNGHow could it matter if it wasn't actually different?


So what do these studies stacked a meter tall tell us?  That lorcaserin doesn't work (except when it does work awesomely); conventional diets suck (except in those who stick to them); Atkins diets may work or suck, and most people give up after Labor Day anyway, just in time for the season premiere of The Bachelorette.  (FYI: She's on Pinot and apples diet.)

If you were hoping the effect size or p values were going to guide you would have been lead astray.  Those p values aren't telling you anything useful, they are at best confusing and at worst misleading.  A glance at the methodology has more practical value than the little asterisk above a score at month 6.  

Look at that stack of studies a meter high.  I've just fed them to a goat.  Tasty.  Can you use them to say whether you should take lorcaserin?  Whether Atkins was better than conventional diet?  Can you use them to guide your decision about whether you should eat more bacon or more ice cream?  Nope. And they will never be able to, because the purpose of these studies is not to determine the answer, the purpose of these studies is to be published, truth be damned.

So I'm sticking to bacon.  And getting my sugar from you know what.



"But, it's LOCF: by mont... (Below threshold)

August 1, 2010 5:53 AM | Posted by Dave: | Reply

"But, it's LOCF: by month 3, 30% of the conventional diet people bailed, vs. only 15% of the Atkins."

That datum supports Atkins right there, doesn't it? It shows individuals are more likely to stay on Atkins than a conventional diet -- and if you can stay on a diet longer, all things equal, you'll lose more weight.

"If you assume that very little weight loss went on in the first three months,"

Why would you assume that? The opposite is usually the case (at least with Atkins, which I'm more familiar with). Talk to individuals who have done it and you'll typically hear that they dropped weight pretty rapidly in the beginning.

"[...] then the weights for the conventional diet will appear heavier than they could have been, dragged "up" by the dropouts who lost no weight because they didn't stick to it."

Maybe I'm missing something, but I'm still not sure how you find this study so inconclusive given that it shows 1) participants were more likely to stick with Atkins than conventional; 2) among the participants who stuck with their respective diets for the duration, the ones who did Atkins lost more weight.

Vote up Vote down Report this comment Score: 2 (8 votes cast)
The problem with diets is t... (Below threshold)

August 1, 2010 6:30 AM | Posted by Cimek: | Reply

The problem with diets is they're 'irrational'. People start them, lose some weight and then go back to unhealthy eating habits, thinking that they somehow "moved down" on the body fat ladder and now they will stay there. It should be obvious, that it's about getting rid of all the sugar from soft drinks/sweets, all the fat from potato chips, fast foods etc. -> eating healthy stuff and getting plenty of exercise + time + stick to it for the rest of your life = Great Success! . Apparently it isn't so obvious or people are just not able/willing to do the hard work required. How about we teach proper diet in schools? Seems to be a terribly important skill.

Vote up Vote down Report this comment Score: 11 (15 votes cast)
"And getting my sugar from ... (Below threshold)

August 1, 2010 9:38 AM | Posted by Anonymous: | Reply

"And getting my sugar from you know what."

Rum? Is it rum?

I think it's rum.

Vote up Vote down Report this comment Score: 37 (37 votes cast)
This is why I have some hop... (Below threshold)

August 1, 2010 11:35 AM | Posted by Matt: | Reply

This is why I have some hope for personalized medicine; at some point we might start getting studies that don't ask "Is Atkins better than conventional dieting?" but "What kind of people stick to/benefit from the Atkins diet?"

Vote up Vote down Report this comment Score: 4 (4 votes cast)
The best weigt-loss approac... (Below threshold)

August 1, 2010 11:37 AM | Posted by SteveBMD: | Reply

The best weigt-loss approach, in my opinion, is the low-Zyprexa, low-Seroquel diet. Works every time.

Vote up Vote down Report this comment Score: 17 (17 votes cast)
I officially have given up ... (Below threshold)

August 1, 2010 12:10 PM | Posted by Dolores: | Reply

I officially have given up my belief in peer review. How could they have missed that? Thanks for pointing out once again never to believe what they are trying to sell to you.

Alone, what's your opinion on hierachy of evidence?

Vote up Vote down Report this comment Score: 4 (4 votes cast)
Thanks for the breakdown. T... (Below threshold)

August 1, 2010 12:32 PM | Posted by David: | Reply

Thanks for the breakdown. This kind of critique, in statistical meta-analysis, is beyond me. I've been able to take a lot away from these kinds of posts.

Vote up Vote down Report this comment Score: 1 (1 votes cast)
I like this approach, which... (Below threshold)

August 1, 2010 1:16 PM | Posted by Ted: | Reply

I like this approach, which encourages looking at whether you're eating to satiate hunger or to avoid emotions:

I'm also interested in books like "When Food is Love" and "Women, God, and Food."

Vote up Vote down Report this comment Score: 0 (2 votes cast)
It seems logical to me that... (Below threshold)

August 1, 2010 4:44 PM | Posted by Anonymous: | Reply

It seems logical to me that the only diet that will really work is one you stay on for your entire life. Therefore if people don't like a diet and drop off that does say something about its efficacy as a guideline for the general population - probably something more important actually than how effective the diet is if stuck to. An hour of intense exercise per day is obviously a great way to stay healthy, but if people get sick of it and don't actually do it it's useless or worse.

That doesn't take away from the point TLP is making though. These papers are pretty weak.

This topic kind of reminds me of a funny video that sort of sums up the 'secret' of weight loss:

Vote up Vote down Report this comment Score: 1 (3 votes cast)
That's true, point well tak... (Below threshold)

August 1, 2010 6:48 PM | Posted by Cimek: | Reply

That's true, point well taken, what I was saying was only that(should have written it before): it seems useless to me to make studies on something which doesn't work in the long run, because it's not the right approach to eating.

Vote up Vote down Report this comment Score: 0 (0 votes cast)
It's not a diet, it's a lif... (Below threshold)

August 1, 2010 10:25 PM | Posted by Anonymous: | Reply

It's not a diet, it's a lifestyle change.

Vote up Vote down Report this comment Score: 4 (6 votes cast)
I bought 2box zi xiu tang. ... (Below threshold)

August 1, 2010 10:27 PM | Posted by maikloe: | Reply

I bought 2box zi xiu tang. I've lost 14 lbs. in just under 4 weeks while using this product. This product significantly reduces the appetite and eliminates cravings. I did not have to exercise or change my routine in any way while using this product. I have had no adverse side effects while using 2daydiet Japan Lingzhi. I highly recommend buying this product if you want to lose weight. And experience dramatic changes in your diet and body size. I took 1 capsule for the first three days as recommended on the labeling and increased to 2 capsules daily on the fourth day. This product does increase your thirst so drinking the recommended amount of water each day is important, and actually, quite easy to do while using this product. I can hardly wait to reach my weight loss goal with the help of this product (no shipping fees)

Vote up Vote down Report this comment Score: -31 (33 votes cast)
My take on this topic, incl... (Below threshold)

August 1, 2010 11:43 PM | Posted by Dave: | Reply

My take on this topic, including some personal experiences, and my theory about how America's obesity epidemic is related to stagnating incomes over the last few decades: Diet and Exercise.

Vote up Vote down Report this comment Score: 0 (4 votes cast)
I was surprised that the di... (Below threshold)

August 2, 2010 1:42 AM | Posted by Gil: | Reply

I was surprised that the diets of different compositions yielded pretty much the same weight loss. So much for cutting out carbs or fats or whatever. However as stupid as it sounds and has been pointed out by others it's about a permanent lifestyle change not a temporary fat loss method. There's something about yo-yo dieters who claim they lose the weight then put it back on and then claim the diet didn't work. But it did work! They lose the weight but don't realise when they go back to their old habits the weight will pile back on.

The solution is indeed a permanent restructing of caloric intake. In other words and quite frankly, yes, eat less. Eat less food but make sure it's balanced and healthful food. If you 500 calories over your body's daily limit then you're going put on fat regardless of the quality of the food. Hence people change their food from junk food to healthful food but forget about the caloric intake and wonder why their weight doesn't budge or goes up.

Simply put when you eat less over time your stomach will shrink and you will find you need food to feel full. Unfortunately the transition to less food intake is unpleasant because your body hates to lose its fat store. Yet you have to stick with it if you want to lose weight.

Vote up Vote down Report this comment Score: 1 (9 votes cast)
If you're looking for a rev... (Below threshold)

August 2, 2010 10:34 AM | Posted by Anonymous: | Reply

If you're looking for a review of the research on weight loss, read "Good Calories, Bad Calories" by Gary Taubes. That is all. Obesity is a disease of fat storage. Carbs trigger insulin, insulin causes fat storage, fat storage causes hunger at the cellular level by stealing energy from the rest of your body to put it in fat cells, and hunger causes more eating of probably carbs. People stick to Atkins better because they are less hungry on it.

Fat people eat because they are hungry. Shocking. Hungry people eat, just like cold people seek warmth and breathless people seek air. Dieters quit because they can't take being hungry anymore. Except for the ones who don't. Those ones fixed the underlying problem and stop being pathologically hungry, or are crazy masochists.

Vote up Vote down Report this comment Score: 4 (6 votes cast)
It's a mistake to make gene... (Below threshold)

August 2, 2010 11:17 AM | Posted, in reply to Anonymous's comment, by Anonymous: | Reply

It's a mistake to make generalizations about why people overeat and that it's all due to physical hunger for food. It seems to me - being purely anecdotal and based upon experience helping people with chronic pain modify their diet and understand the body/mind connection - that quite a lot of overeating has to do with not actually being able to discern what one's body needs and habitually using food as the solution for any discomfort (the irony being that obesity increases physical discomfort). That and bad habits and easy access to junk food.

High fat and sugar foods change the way we feel temporarily - much in the same way drugs do. If someone has been taught that food is a way to avoid unpleasant feelings (including emotions, which are physical), that will be their habitual coping/avoidance mechanism. A lot of these bad habits run in families and are a part of family culture and can even be the primary way a family bonds, expresses love and "feels good" together. Not that that's a problem in and off itself - it all depends on other lifestyle factors.

Also, we all have a limited capacity for self control. If someone is using up all their self control in one area of their life they won't have much left to resist the temptations of full pantries and drive-through binges. Throw in how long many of us spend sitting at desks or in traffic and you've got a recipe for overeating and lack of exercise (which will eventually create a metabolic disaster). I'd actually suggest that doing this to your kids is a form of (unintentional) child abuse even though parents are often trying to express their love through food (though sometimes they're just trying to shut up their kid too). I'm not judging since obviously people can't teach what they don't know themselves. Like with anti-smoking campaigns, probably the best way to deal with this on a larger social level is actually educating kids so they can make informed choices and bring that knowledge home to their parents.

Vote up Vote down Report this comment Score: 5 (7 votes cast)
Here is the big diet secret... (Below threshold)

August 3, 2010 1:19 PM | Posted by Anonymous: | Reply

Here is the big diet secret folks: you must eat less calories than you consume.

Vote up Vote down Report this comment Score: -9 (13 votes cast)
I wonder if that is true? ... (Below threshold)

August 3, 2010 6:09 PM | Posted, in reply to Anonymous's comment, by Heavier anyonymous: | Reply

I wonder if that is true? It's not as simple as calories in, calories out, is it? Some people eat more calories and stay thinner, with no _obvious_ difference in exercise. What about "the higher metabolism?" (and I'm truly asking.)

Vote up Vote down Report this comment Score: 0 (2 votes cast)
From my experience it is th... (Below threshold)

August 3, 2010 9:10 PM | Posted, in reply to Heavier anyonymous's comment, by random: | Reply

From my experience it is that simple, though you also need to be physically active not just for weight loss but also to be healthy. I've seen a few people go the route where they make a whole mental preparation and switch into 'diet' mode, which means they usually go on a shopping spree for processed foods that label themselves as diet, end result being a shopping cart with items that have a low fat / reduced fat / only 2 calories! marketing flash plastered across it. The method ive seen work well for multiple people is to not actively go on a diet. A lot of those diet items drastically increase some other bad-for-you ingredient to improve the processed food taste, and sorting out which are bull and which arent seems like a lot of work. You should just start eating well (and drinking well - restricted soda), example instead of going to KFC buy some chicken and make it yourself, its not as hard as you think and it doesn't require 100% strict adherence - its just healthy has to become the rule, not the exception.

Some people can eat junk food all day and not gain weight, but are they healthy? And is it relevant to you? If at the end of the day it seems like your body cannot manage processed junk food without significant weight gain whether someone else can get away with it wont do anything to change the human body model you got stuck with. Trust me, I would be first in line to trade-up or customize if the option was ever possible, but it doesn't look you get to pick and choose the specifications of the body your life will be lived in.

Vote up Vote down Report this comment Score: 0 (2 votes cast)
No, the secret to losing we... (Below threshold)

August 3, 2010 9:11 PM | Posted by Anonymous: | Reply

No, the secret to losing weight isn't to eat less, it's to not be hungry.

Vote up Vote down Report this comment Score: 10 (10 votes cast)
"Also, we all have a limite... (Below threshold)

August 6, 2010 4:25 PM | Posted, in reply to Anonymous's comment, by Lurker: | Reply

"Also, we all have a limited capacity for self control. If someone is using up all their self control in one area of their life they won't have much left to resist the temptations of full pantries and drive-through binges."

I was not aware of this curious limitation. Do we have a self-control gland that is limited in it's output? Does it work like your adrenals: if self-control is imposed from the outside one loses the capability to produce self-control and is damned to a life of excess? (I think I could make the rounds on the talking-head circuit with this would be a hit with the lapsed Catholic crowd).

Anecdotally, I think those with true mastery are able to exert self control in all areas of their life in which they require it.

Vote up Vote down Report this comment Score: -4 (6 votes cast)
"I was not aware of this cu... (Below threshold)

August 9, 2010 5:16 PM | Posted, in reply to