I am posting this article again to put it on the front page as I will be talking to Phil on 1150 am about it today.
When I first started looking into nutritional science, I was shocked at how badly it was done. To be fair, nutrition science if very difficult. There are so many lifestyle and nutrition choices that can have a positive or negative influence on health, that it is hard to isolate the effect of any one food or group of foods. There are two main types of studies; Observational (Epidemiological) studies and Controlled Experiments. Tom Naughton has a really good video called “Science for smart People” that explains the difference in an informative and humorous way.
When I first started looking into nutritional science, I was shocked at how badly it was done. To be fair, nutrition science if very difficult. There are so many lifestyle and nutrition choices that can have a positive or negative influence on health, that it is hard to isolate the effect of any one food or group of foods. There are two main types of studies; Observational (Epidemiological) studies and Controlled Experiments. Tom Naughton has a really good video called “Science for smart People” that explains the difference in an informative and humorous way.
The gold standard of science is the controlled experiment. In a controlled experiment, you hold all variables constant except for the one variable that you are
studying. For example, if you wanted to
study the effect of a fertilizer on growing plants, you would plant two groups
of the same seeds. You would use the
same soil, same size pot; put the pots in the same area where the temperature
and light exposure are the same, and give them the same amount of water. You would put fertilizer in one but not the
other. If the plants in the fertilized
pot grow faster and larger, then this would support your hypothesis that fertilizer
helps plants grow.
It is very hard to do this in a nutritional study. The only way to be sure of the quality and
quantity of food consumed, would be to lock people in a metabolic ward where food could be controlled. You would have
to weigh and measure all the food served, as well as all the food that wasn’t
eaten. Since most people are interested
in being healthy for the rest of their life, and not just for the next 6 months,
these studies would have to last 10 to 20 years to be really meaningful. Not too many people would volunteer to be in
a study that would lock them in a metabolic ward for 20 years! (well, maybe Al Bundy would volunteer). Even if you could do a study like this, it
might not be relevant to the real world.
In a metabolic ward you have to stick to the diet; there is no other
food choices. In the real world, there
are food choices on almost every street corner.
People have to be able to stay on a nutritional lifestyle long term for
it to be useful.
Very few controlled studies are done in nutritional
science. Almost all the studies we hear
about are observational studies. In
these studies information is gathered on what people eat, usually with food
frequency questionnaires. These people
are followed for a number of years and their health outcomes are observed. Correlations are made between the food people
ate and their health outcomes. These
types of studies have many limitations, the biggest being that they cannot
provide any information on cause and effect.
Observational studies are useful for coming up with a new hypothesis,
but then the hypothesis has to be tested in a Controlled Experiment. A couple hypothetical studies from Tom Naughton's video might explain
why this is.
If we were to do a study of the BMI of marathon runners
compared to the BMI of the average person, you might find that running marathons is
correlated with a lower BMI. The
conclusion of the study could say that running in marathons was linked, or
correlated, or associated with a lower BMI.
They could not say that running in marathons causes you to become lean
and have a lower BMI, even though most people hearing about this study would
think that running marathons does cause people to have a lower BMI. It conforms with our preconceived notions
about exercise and weight, so we assume that causation is proved by the study. If we
look at another hypothetical study, it will illustrate why this is not the
case. If we did a similar study but used
professional basketball players and height, we would find that playing
basketball is correlated with being taller than the average person. Does that mean that playing basketball causes
you to grow taller? If you are 5’6” and
want to be 6’ tall, can you play basketball for a few years and expect to grow?
Of course not! Playing basketball doesn’t cause you to grow taller; it’s
just that if you are tall you are more likely to play professional
basketball. The same logic can be used
in the other study: Running marathons might not make you lean, it’s just that
lean people are more likely to run marathons.
When something in a study (A) is correlated with something
else (B) it is easy to jump to the false conclusion that A is causing B. A may or may not be causing B, as there is no
way to know just from an observational study.
A may be causing B. B may be
causing A. A third variable, C may be
causing A and B. This third variable C
is what is called a confounding variable.
An example of this is that ice cream sales in Florida are correlated
with shark attacks. Does this mean that
eating ice cream causes shark attacks?
Maybe the sharks like the ice cream dripping down your chin so they are
more likely to attack. Of course this is
silly. When it is hot, people eat more
ice cream and they also go swimming more, which leads to more shark
attacks. Observational studies are full of confounding
variables. There are two main groups of
people that influence the outcomes of nutritional studies. There are people who are very health
conscious and do whatever they can to be healthy, and those that do not care
about their health and make food and lifestyle choices that are purely based on
giving them pleasure. People who are
health conscious tend to have better
health than people who are not health conscious. They
follow
the health advice that has been generally given the last 40 years; they
smoke less, drink less alcohol, exercise more, take vitamins, eat less
calories, eat less sugar, eat less refined processed foods, and eat more
vegetables. Any one of these variables could be
contributing to their good health, but from an observational study you
can’t
tell which one. You would need a
controlled experiment to do that.
Another problem with observational nutritional studies is
that the data collected is not very reliable.
It is usually collected using food frequency questionnaires. In these studies people are asked to recall
what they ate in the last day, month, year, or even four years. You can find an example of a questionnaire here.
Most people can’t remember what they ate last Tuesday, let alone what they ate
three years ago. People who see
themselves as healthy tend to overestimate foods they consider healthy and
underestimate foods they consider unhealthy. Who wants to admit that their
breakfast consisted of twinkies and oreos? If the data that the study is based
on is not accurate, then how useful is the study?
Some scientists can have such a strong belief in what the
outcome of their study will be that they become biased. Since there are so many confounding variables,
you can make the outcome of a study say pretty much whatever you want it to
say. Most studies will try and account
for these variables, but it is almost impossible to know exactly how much of a
part each one played in someone’s health. Some researchers will use a third variable to link two items when there
is no direct link. A good
example of this
is saturated fat, cholesterol and heart disease. Many studies (such as
Dr. Jolliffe's Anti-Coronary Club experiment mentioned here) will claim to show that saturated fat causes heart disease even if the data in their data shows that
people who ate more saturated fat had a lower incidence of heart disease. They do this by saying that saturated fat intake was associated with higher cholesterol, which is claimed to
be a marker for heart disease. (Even though it has never been proven that high
cholesterol causes heart disease)
This is why we get so many mixed messages from the so called
“nutrition experts”. The way
Observational studies are reported, it gives the idea that they determine cause and effect when they really
only show correlation. In one group of people a certain food may be correlated
with high cancer rates, in another group they may be correlated with low cancer
rates. The truth may be that the food
has NO impact on cancer rates. As long
as we remember the limitations of the study we won’t get sucked into these
false assumptions. So the next
time you
hear about the latest study telling you to stay away from a certain
food, or that another food is a miracle cure, remember that 99%
of these studies don’t actually prove anything.
It is time to stop funding these types of observational studies. How many studies do we need that give sensational headlines but do not add to our knowledge. We have enough hypotheses about health and nutrition. We need to start doing controlled experiments to find out which hypotheses about diet and nutrition result in healthy outcomes. With obesity related diseases and health care costs skyrocketing, we need to find these answers now.
It is time to stop funding these types of observational studies. How many studies do we need that give sensational headlines but do not add to our knowledge. We have enough hypotheses about health and nutrition. We need to start doing controlled experiments to find out which hypotheses about diet and nutrition result in healthy outcomes. With obesity related diseases and health care costs skyrocketing, we need to find these answers now.
No comments:
Post a Comment