SP Premium
1-99 SparkPoints 16

Observational versus Experimental Studies

Saturday, July 21, 2012

There's a study making the rounds in dietary literature that claims low carbohydrate diets increases risk of Swedish women's cardiovascular disease. The study was based on a questionnaire that asked women what they ate in the past 16 years.


According to the summary:

"A one tenth decrease in carbohydrate intake or increase in protein intake or a 2 unit increase in the low carbohydrate-high protein score were all statistically significantly associated with increasing incidence of cardiovascular disease overall"


Well. That should be enough evidence for all of us to turn course and go back into the safety of skim milk and low-fat cottage cheese.

But wait. We can't conclude anything from this study at all.

It's an observational study.

Tom Naughton (of "FatHead" documentary fame) has a YouTube video speech he gave on a low-carb cruise explaining how to spot good science from bad science.

Good science = Clinical/Experimental Study
Bad science = Observational Study


An example of flawed observational study conclusion:

"People who like to run are thin. I am fat. Therefore, if I want to be thin, I should run."

But there are numerous people who run and never get thin. They get shin splints and metatarsal stress fracture. If they can't run, how can they get thin?

The real conclusion is: "People who are thin tend to chose running for exercise."

Here's another observational study. 16 years ago (the period in which this study was conducted), I was a college student. I ate no breakfast, ramen noodles, pizza, beer, and Snackwells low-fat cookies. I was slim and weighed 125lbs. What can we conclude? Was it no breakfast, ramen noodles, pizza, beer or Snackwell's cookie that kept me slim?

We can't conclude anything. Even if we add in that I was walking an average of 2 hours per day to classes, uphill both ways, we can't conclude anything. Even if we added in my entire girl's dorm who were doing the same thing, we can't conclude anything.

Snackwells cookies were very popular in my dorm. Did it keep us college age girls slim? Did most of us gain weight after college because we stopped eating Snackwells? Should we eat more like we did in college to lose more weight?

Here is a breakdown analysis on the flaws with the Swedish study. She is not a low-carber; she is a raw food vegan who hates bad science.


Her summary:

- This study is observational—meaning it tells us nothing about cause and effect.
- This study relies on one food frequency questionnaire that the women filled out upon enrollment in the Women’s Lifestyle and Health Cohort. We have no idea what they were eating during the next 16 years.
- Women reported their food intake during the early ’90s, when low-fat diets were the gold standard for healthy eating. The most health-conscious women were likely to be eating the most carbohydrate at that time.
- Food frequency questionnaires are all sorts of terrible and don’t let anyone ever tell you otherwise.
- The study’s “diet score” design assumes that all low-carbohydrate diets are necessarily high in protein—resulting in a point-assigning system that doesn’t always identify true low carbers.
- The women with the lowest carbohydrate intake in this study were still eating up to 123 grams of it a day (an amount that’s probably wildly underestimated, to boot)—meaning they weren’t actually eating a low-carbohydrate diet.
- The true increase in cardiovascular disease risk from scoring high on the “low carb, high protein” scale is pretty minor—and pales in comparison to other lifestyle variables.

Emphasis: 123g daily carbohydrates was the lowest carb intake in this study. The conclusions are not drawn on a low-carb diet at all.

Here is a Standford clinical study comparing Atkins, Zone, Ornish, and LEARN (standard american recommended diet). Atkins outperformed all of the other diets in terms of weight loss, blood lipids, blood pressure, increased HDL, decreased LDL, normal cholesterol, low triglycerides, and overall compliance following the diet. The head researcher is a life-long vegetarian, and although surprised by the results, reported the facts as they stand. Conclusion: Atkins did not increase the risk of metabolic syndrome and heart disease, but decreased all known long term clinical markers.

Abstract: nutrition.stanford.edu/d

Youtube: www.youtube.com/watch?v=

The other interesting conclusion on the Stanford clincal trial is people who had some degree of insulin resistance performed better on Atkins. People who had no degree of insulin resistance performed better on the LEARN diet.

Here is an article about North Carolinian dieticians attempting to use legislation to censor a blogger who writes about how he cured his diabetes through a low-carb diet.


The blogger in question is named Steve Cooksey. Here is his blog, and all the ailments he cured through diet alone:


There's so much conflicting information, it's difficult to tell the good from the bad. Recognizing the difference between an observational study versus an experimental/clinical study is an important way to weed out the good from the bad.

Overall, self-experimentation was the only way I knew for sure.

Does my diet:
- Make me feel better
- Improves quality of life
- Leaves me with energy throughout the day
- No energy crashes
- No hunger
- No cravings
- Loses weight
- Easy weight maintenance

Does my diet reduce risk of Metabolic Syndrome/Heart Disease:
- Normal blood lipids (HDL,LDL,Trigylcerides)
- Normal blood pressure
- Normal body weight/body fat

My answer to all of the above is a resounding checkmark +.

Thanks to JustBirdy for raising the awareness on the ADA censorship and Denise Minger's analysis of the Swedish study.

"What did YOU eat 16 years ago?"

"Forbes article exposes the "academy's" creepy practices"
Share This Post With Others
Member Comments About This Blog Post
    Oh, wait@! The Swedish women who were more educated had less heart disease events!
    3105 days ago
    hey, we should conclude that people shouldn't go to college!!!!
    3105 days ago
    I took a bazillion classes in statistics and research methodology when I was in grad school so I learned a lot about how to read research - unfortunately, most people don't realize there is a huge difference between 'correlation' and 'cause/effect' - correlation just means that A happened when B occurred but unless an extensive multivariate analysis is done, there's no way to even begin to understand the problem OR to form the next hypothesis for the next research study.

    And even more unfortunately, I know just how easy it is for people to be fooled by marketing - I have many years experience in the industry and teach it at the college level. It's not to say that 'marketing' is bad or even manipulative, although the goal is to get people to 'buy' something - maybe a product, maybe a political stance, maybe a religious opinion - marketing is present in every single aspect of life. But a VERY small percentage of adults have ever studied it enough to realize what's going on and how subtle associations can be used to convince them to 'buy' whatever is being 'sold,' in this case the idea is that low carb is bad. I'm very happy to see you pointing out the fallacies many people might not realize are there and helping people understand that they can't necessarily draw a conclusion from a particular research study unless they really KNOW how to read the info!

    The worst part of this, to continue my rant, is when DOCTORS and other professionals we think we should trust also don't know how to read the research much of the time! I know we 'want' to believe that a medical professional has extensive training in reading research but they often have just the basics and focus more on 'practice' than 'research' in med school. They also get zero or almost zero training in nutrition, weight loss or gain, fitness and so forth unless they seek it out for themselves as an extra topic to study. So we often hear ridiculous theories from doctors who honestly are subject to the same flawed thinking as the rest of the population about many topics.

    OK, stepping off the soap box for the moment!

    emoticon emoticon
    3106 days ago
    Very nice blog. It is so easy to allow studies, no matter what form, govern our reactions. As you pointed out, it ultimately comes down to how what we eat affects our health and well-being. People may look at the amount of protein I eat and think I am overdoing, but if I do not have that my nails break and I crave (and thus eat) carbs causing weight gain. But yet the next person may only need a little bit of protein to satisfy. There is not one diet that will fit all people, and the studies as you point out, are generalizations about one aspect.
    3106 days ago
    Any study where intake of food is self reported is immediately grounds for discard. I don't know a single person who doesn't suffer from what I like to call "food amnesia". This is why tracking is so incredibly important for weight loss, IMO. But that's irrelevant to this. Denise Minger, while wordy, definitely sums up my opinion on the matter. Ridiculously flawed to the point of stupid.
    3106 days ago
  • DIANE7786
    Great blog. The Cooksey info is scary.
    3106 days ago
  • Add Your Comment to the Blog Post

    Log in to post a comment

    Disclaimer: Weight loss results will vary from person to person. No individual result should be seen as a typical result of following the SparkPeople program.