How fake was the ‘fake’ chocolate study?

News outlets across the globe were embarrassed last week when Dr John Bohannon announced his widely publicised study on chocolate and weight loss was fake.

Stories about the Bohannon study had appeared in dozens of outlets from Shape magazine to theTimes of India, under headlines such as “Scientists Say Eating Chocolate Can Help You Lose Weight” and “Dieting? Don’t Forget the Chocolate.”

The study itself was real enough, in the sense that it involved real test subjects, randomised trials, and honest data. When Bohannon says his study was junk, he means that “it was terrible science. The results are meaningless.” His team deliberately went fishing for statistically significant differences in a way that virtually guaranteed they would find one, and they packaged their results in a way designed to keep lazy journalists from asking too many questions.

But the story gets even more complicated: Bohannon’s study may have been a gonzo prank, but real studies involving hundreds and even thousands of subjects have found that moderate chocolate consumption is associated with lower BMI and better health. People who found Bohannon’s study plausible were not necessarily gullible dupes.

So does chocolate help you lose weight or doesn’t it? Unfortunately, there is no conclusive proof either way just yet. That’s one reason why government should stay out of people’s diet decisions—nutritional science is constantly evolving. The diet bugaboo of yesterday often becomes the diet hero of tomorrow. There is very little we know for certain, even about something as simple as whether chocolate is good for you.