Phil Nuttridge continues his series of articles looking at the modern take on diet and nutrition. He explodes many of the dietary myths that defined the latter decades of the twentieth century and left their legacy of chronic illnesses in the first decades of this century. In this month’s article he looks at the how we have been misled by nutritional studies over the last fifty years. Headlines that start “Diet study shows…..” are all too often leading us into a trap and Phil shows you how to avoid such traps. More information can be found on Phil’s website cuttingcarbs.co.uk or by following him on Instagram: CuttingCarbsUK
I have a theory about swearing.
I have five good friends who come to me for manual therapy treatment and if I press hard enough on their tight muscles during those sessions they swear. I have six other good friends who tell me that they do not have tight muscles and no matter how hard I press, they do not swear.
My conclusion is therefore that having tight muscles causes you to swear. And of course, you can see I have the data to prove it. Hence forward, let this be known as ‘The Nuttridge Law of Swearing’.
Now of course (I hope) you see that this is utter nonsense. Just to be clear, the nonsense is as follows:
- I have ignored the fact the there are people who have tight muscles who do not swear and there are people who swear who do not have tight muscles. For the Nuttridge Law to work I have to ignore these.
- My data is only for eleven of my friends. The other seven billion people on the planet may not behave in the same way.
- I have not defined what I mean by swearing. Does a ‘strewth’ count as swearing or does it have to be the sort of language who might hear on Game of Thrones? My friends are definitely more at the ‘strewth’ end of the spectrum and yet I am implying that the Nuttridge Law applies to the whole spectrum.
- I have completely ignored that it is my pressing on the tight muscles that elicits the swearing, yet the Nuttridge Law implies that it is enough for there to be tight muscles for the swearing to ensue.
- I make no mention of how often I prodded the people and or how hard I prodded the people or how long I spent trying to find spots that would elicit swearing. Surely that must make a difference?
- The five friends who I identify having tight muscles are so determined from my prodding; the six friends who do not are ‘self-reporting’ their lack of tight muscles. Maybe the friends with knots warned these other friends that if they confess to having tight muscles, it’s going to hurt!
- No matter how strong the apparent link between swearing and tight muscles in my data, the Nuttridge Swearing data assumes causation(tight muscles cause swearing). The data however no more strongly suggests that tight muscles cause swearing than it suggests swearing causes tight muscles.
- I have completely ignored the possibility that things other than tight muscles might better correlate with swearing but yet these other things might also be correlated to tight muscles. For example, having a stupid boss at work might both give you tight muscles and increase the likelihood that you swear.
You can tell I am a statistician – I have managed to create eight bullet points of analysis from just two sentences of data. As one of my colleagues from the days when I had a proper job once said, Phil sure knows how to put the ‘anal’ into ‘analysis’!
But there is a very serious point here. All of these elements of nonsense have been used to fool you – yes, I mean you – and me and everyone else into believing all sorts of stuff about food and nutrition. What I am referring to here are those newspaper headlines “Study shows that eating <food x> causes <illness y>”. Or “Study shows that <diet x> is good for you/bad for you (delete as applicable)”.
In the context of my tight muscle/swearing data, it is pretty obvious that the wool is being pulled over your eyes. But amidst the complex data of the reported nutrition studies, it is far less obvious. And let’s face it, unless you are a geek like me you are not going to delve below the headline allowing you to be drawn in the direction the headline wants to take you regardless of how flawed the study is. No wonder we are all confused about what to eat and what not to eat.
What I would like to do in this article is give you two real examples of how we we have been led down the garden path with nutritional studies. We all want to believe that science is robust and to be trusted; what I want to show you is that you have to be rather more discerning.
FAT CAUSES HEART DISEASE, RIGHT?
In my earlier article about Fatphobia, I described the ‘Seven Countries’ dietary study. To recap, this was an analysis of the eating habits of seven countries and how that correlated to heart disease rates in those countries. The data seems to demonstrate perfectly that countries with low consumption of fat had the lowest rates of heart disease; those with the highest consumption of fat had the highest rates. So convincing was this data, that this study has become one of the most cited scientific studies of all time. Without question it is the bedrock on which the ‘fat is bad’ paradigm has been built.
But just how robust a study is it? Specifically, does it fall into any of the nonsense traps of the Nuttridge Law of Swearing?
Yes it does. Spectacularly.
In fact, this pivotal study that has shaped most of our nutritional thinking for the last fifty years fails on pretty much all of the items listed in my demolition of the Nuttridge Law:
- The author of the study chose to look at data from just seven countries but he actually had data from twenty-two countries. He chose to ignore countries with high fat consumption and low rates of heart disease and also countries with low fat consumption and high levels of heart disease – these countries would be ‘inconvenient’ to his conclusions. It is the same as me ignoring people who have tight muscles but do not swear and people who do swear but do not have tight muscles.
- The study only looked at the detailed food diaries of a relatively small number of people in each country and then, only on a very small number of occasions. It is like saying that my eleven friends’ swearing habits are a true measure of all seven billion people on the planet. Even if he had used the data from all twenty-two counties he had at his disposal it would only have been a fraction of the world’s population and only sampled on a small number of occasions. Astonishingly he only collected data from men; not one woman’s dietary habits was measured in the study. Not one. Intentionally only measuring the correlation between fat and heart disease in men but then applying the results to women too is an ‘interesting’ statistical approach to say the very least.
- There were some breathtaking holes in his data collection. He chose, for example, to ask a Greek orthodox community about their fat consumption during Lent. Is it any surprise then that their reported fat consumption was low? Was it how I pressed the muscles and when I pressed them that caused the swearing in my data? Similarly, was it the way the Seven Countries’ data was collected that showed low fat consumption?
- Interpretation of this study has fallen hook, line and sinker into the causation trap. The data, even if it had not fallen foul of the points above, would only show correlation not causation. In fact if you recast the data from the original study there is a stronger correlation between sugar consumption and heart disease (expected) and even between TV ownership and heart disease (not so expected). Yet the study is used to show high fat diets causeheart disease. It is just like my swearing data being used to show tight muscles cause swearing. And the sugar correlation is like my analogy of the stupid boss who might be correlated better with both tight muscles and swearing. Because the author of the study was so convinced that fat was the villain he ignored the better correlation with sugar.
I cannot emphasise just how pivotal this Seven Countries study has been in implanting the paradigm that eating fat causes heart disease. And yet it fails on so many of the flaws of the Nuttridge Swearing data. The next time you refrain from having that piece of cheese, or get tempted by that ‘low-fat’ yoghurt, or say ‘no’ to that drop of cream on your bowl of strawberries or baulk at that three egg omelette, just remember that the ‘fat is bad’ reflex engrained into us all is based on a study scarcely more robust than my swearing and tight muscle data.
BACON CAUSES CANCER, RIGHT?
In 2015 there was a news headline ‘Eating bacon increases your risk of cancer by 20 per cent’. Horror, horror, horror. In the interests of keeping the bacon butty, let us examine how this study measures up against the Nuttridge Swearing data. Can I save your bacon?
Let me sort-out the numbers first, after all, that is what statisticians do. A twenty percent increase in cancer risk sounds truly horrific, doesn’t it? But there is some naughtiness here – there is one very vital word missing from that headline.
Going beyond the headline, the numbers in this study are: For every hundred people in the ‘no/low bacon’ group, five got cancer; for every hundred in the ‘high bacon’ eating group, six got cancer. So, where did the 20 percent in the headline come from? Well an increase from 5 per 100 to 6 per 100 is a relative increase of 20 per cent. But the absoluterisk of getting cancer only went up by one per cent. And when it comes to cancer, it is absolute risk that matters.
Of course a newspaper headline ‘Eating bacon increases your risk of cancer by 1 per cent’ isn’t so catchy is it? If you were told that eating four or more rashers of bacon every day for the rest of your life would only increase your risk of cancer by one percent, would you be worried? For comparison, regularly drinking moderate amounts of alcohol increases your risk by far more than that. So, by omitting the word ‘relative’ from the headline and working on the assumption that very few people would delve into the data, the headline writers let you believe eating bacon is far worse than the data actually suggest. But why would they do that?
Money.
It was the shady informant character called ‘Deep Throat’ who told the journalists investigating the Watergate scandal that they should “follow the money”. And so it is here too: This bacon study was sponsored by a breakfast cereal manufacturer. Yes, the people who do not want you eating bacon but instead want you to consume their high profit margin, high carb and high sugar breakfast mueslis and cornflakes were behind the shenanigans here. So (and thankfully I can say that the Nuttridge Swearing data does not fall foul to this one) always, always beware of who sponsors a food study.
Let me ponder the numbers a little bit more in this study. Remember my stupid boss analogy in the swearing data? How can we be sure that there is not some other unreported factor at play here that has a stronger link with cancer?
Well, if you again delve into the data of this bacon study, you will see something rather odd. The people who ate the most bacon also drank the most alcohol, smoked the most and exercised the least. By a significant margin on all counts. So, if the bacon eating group have a higher rate of cancer, is it because of the bacon or is it more likely the booze, fags and sloth? The cereal manufacturer does not want us to think in those terms so these ‘facts’ were excluded from the headline. In fact, alcohol and tobacco have such a strong correlation with cancer risk that for those in the bacon-eating group to have ‘only’ increased their risk by 1 per 100 (tobacco alone would increase the risk by around 4 per 100), maybe eating bacon was in some way protective for the smokers and mitigated most of the cancer risk from smoking. Now there’s a good reason to tuck into some bacon!
Of course, even if we ignore the skulduggery over the numbers, as with pretty much all such food studies, the bacon study wants you to confuse correlation with causation. Just as I wanted you to believe tight muscles cause swearing even though my data only showed correlation, so it is with bacon consumption and cancer. And with only a one percent difference in cancer rates even though they smoked and drink the most, bacon-eating only had a very weak correlation to cancer at best, and maybe even a beneficial link.
There is one further point which also applies to pretty much every food study that I want to conclude with.
Recall how in my swearing data, it was bit unclear whether it was the presence of tight muscles or me pressing them that related to swearing? And remember how those without muscle knots were self-reporting? You could say that my data collection methods were a ‘bit vague’.
In the bacon study, participants were given a questionnaire asking them to list what they had eaten and drunk in the previous week and what activity they had undertaken. If I gave you a questionnaire and asked you the detail of what you ate last week, how accurate do you think your memory would be? How much could I trust you not to over-report the foods that you think are ‘good’ for you and under-report the foods that are ‘bad’ for you? How could I be sure that you don’t forget those biscuits you had last Thursday or that lump of cheese you had just before bed on Saturday? Or that sneaky top-up of wine on Sunday?
And yet, because it would be unethical to keep people isolated in a laboratory and control every aspect of what they eat and do, pretty much every nutrition study has to rely on self-reporting questionnaire data like this. Shockingly, even though this bacon study followed the cancer outcomes of the participants for ten years, they were only once asked to record what they ate. How robustly can I assume that what you ate last week (or rather, what you saidyou ate last week) will be representative of what you will actually be eating in ten years’ time?
Is it any wonder then that the results of nutrition studies can seem contradictory – today’s superfood is tomorrow’s cancer-causing horror?
So, perhaps eating saturated fat is not linked to heart disease and perhaps eating bacon is not linked to cancer. Of course those companies that want you to eat their processed carbs instead of natural fats and those that want you to eat their ‘healthy’ breakfast cereals instead of bacon will wave these studies at you. And if you are too trusting of science, you will believe those studies.
Let me give you a simple tool to help de-clutter some of this nonsense. Let’s have a close shave with something called Occam’s Razor.
Occam’s Razor is a philosophy that is really an appeal to that rare commodity – common sense. The Razor essentially suggests that when you have competing hypotheses, the one that requires the fewest assumptions is usually the best one. In other words, the simpler the better.
Let’s apply Occam’s Razor to the hypothesis: “Eating saturated fat causes heart disease, eating unsaturated fats and carbs reduces the risk of heart disease”.
One hundred years ago, there was very little heart disease. Then, we ate more saturated fat than we do now and far fewer refined and processed unsaturated fats and carbs. Today we have higher levels of heart disease. We now eat less saturated fat (because we have been told to), we eat more chemically created unsaturated fats (because we have been told to) and far more processed and refined carbs (if we have less fat on our plate, we need to fill it with something else).
Can it really be the case that we now eat less of something that according to our hypothesis causes heart disease (ie saturated fat) and more of the foods that are meant to be protective (unsaturated fats and carbs) and yet we now have much higher levels of heart disease? Compared to a hundred years ago, we now eat less of the (alleged) bad foods and more of the (alleged) good foods and yet we get more heart attacks. It just does not make sense.
Wouldn’t it be far simpler (Occam’s Razor) to assume that the thing we eat less of (saturated fat) in an age of more heart disease is actually the thing that protects against heart attacks? Eat less of it, less protection. And the things we eat more of (unsaturated fats and carbs) in an era of more heart attacks are the more likely cause of those heart attacks? Eat more of them, more heart attacks.
Simple.
So beware. If you see a headline “Study shows……” be very suspicious. Do not let a journalist’s spin on a study, probably sponsored by a food company and with probably some very dubious data collection techniques that almost certainly can only at best show correlation rather than causation, sway you into thinking that a food or diet is good or bad for you. Healthy cynicism with a good deal of common sense is the best diet.
In my next article I shall tackle the tricky subject of food supplements. “To supplement or not to supplement, that is the question” looks at the case for pills over food.