There are a number of different types of scientific study. Each has their own strengths and weaknesses.
Original study or review paper
When you start looking at scientific papers you can either look at original studies or review papers. An original study generates new results. A review paper or meta-analysis, groups together a number of related studies and summarises their collective findings. Often these findings are contradictory, and then the review paper weighs up the evidence in favour of each.
Review papers are often easier to read than original studies, containing more text and fewer statistical calculations. However, both original research and review papers can be biased and it may be worth digging down into the study references cited by the review paper if you want to be sure about their conclusions.
Whether an original study or review paper, the majority of studies in the field of nutrition are epidemiological.
Epidemiological studies look for possible causes and effects such as risk factors and diseases. An example is shown in the newspaper article above, in which a risk factor (wine) is proposed to cause of a range of chronic diseases.
Epidemiological studies come in two main forms. Observational and experimental. Often the observational studies come first. They may suggest a cause for a certain disease or health outcome. The experimental trials then follow, typically testing animals first and then humans in a series of clinical trials. Both observational and experimental studies have their problems, and headlines based on them invariably mislead.
These studies appear very often and are reported almost as if they are fact. In reality they are quite limited by the assumptions and skill of those who carry out the studies. In a nutritional context they try and produce associations between diet and various diseases. Examples could be saturated fat intake and heart disease, salt intake and high blood pressure, sweeteners and cancer or even vaccines and autism.
Observational studies try to look at existing groups of people and see if their diets or lifestyle can be linked to the prevalence of disease in that group. Often it is just one dietary factor and one disease that are investigated. These groups are not controlled like laboratory rats however. They are free living human beings, and so what they eat and how they live is complex. As a result trying to associate one food with one disease is riven with difficulty, as many other confounding factors come into play.
Example observational study associating salt with high blood pressure.
A link between salt and high blood pressure could have several confounding factors:
The measurement of salt intake is difficult to do accurately. Different foods contain different amounts, but may vary depending on their source as well as what type they are. For instance bakery bread can contain mountains of salt, whereas a home made loaf may contain none.
People who eat salty food could also be more likely to eat other nutrients that could raise blood pressure. Equally they could also have lifestyle differences. For instance convenience food is mostly salty and people who eat it are less health conscious, as such they are less likely to exercise and eat plenty of fruit and vegetables. These factors would also raise blood pressure.
Different populations may have different susceptabilities to changes in blood pressure resulting from salt. So if a study linking salt with high blood pressure was carried out on people in one country it may not apply in another. For example black people are on average more prone to high blood pressure. This may be generally true, but again these bland statements often hide important details. For instance there is more genetic difference between a black member of San tribe originating from Namibia and a typical Nigerian, than between a typical Nigerian and a typical Scotsman. Surprising maybe, but true. It should never be forgotten that people are individuals in a very real sense, and extreme care should be taken when treating people as typical members of a group.
A flow chart that could help you decide what type of study you are looking at can be found here.
These epidemiological studies use large groups of subjects to test a hypothetical link between some dietary factor and an outcome such as biological marker (e.g. cholesterol level), disease (heart disease) or death. They are normally prospective studies that enroll subjects, record their status and then monitor them for a period of time for predicted outcomes. Sometimes a retrospective study is done that selects subjects at a past date based on past data for exposure to certain risk factors. For instance nurses who worked at a hospital 10 years earlier, some who smoked and some who didn't. The current prevalence of a specified outcome is then found. For instance 1 in 10 of the smokers may have developed heart disease 10 years down the line. Perhaps only 1 in 20 of the non-smokers have. You can use criteria called the Bradford Hill criteria to try to tease out stronger evidence of cause and effect in these epidemiological studies.
Case Control studies
Unlike cohort studies, these epidemiological studies select effectively 2 cohorts. One has already got the outcome, such as a disease (case), and the other is disease free (control). Normally the case control study then looks for differences between the exposure of the 2 groups to certain risk factors.
Experimental studies normally control the environment of the study subjects. Typically one variable is changed between a group being tested and a control group. Experimental studies can be done in test tubes (in vitro) or with live subjects (in vivo).
In vitro experiments are often misleading as the human body is a complex interacting environment which is impossible to duplicate in the test tube. In vitro results can suggest possible explanations, but can never be considered to prove what causes a particular effect. They are most effective at looking at specific steps that could happen inside the body. For instance, can a substance induce a cell to change in a way that could lead to cancer?
In vivo experiments are generally animal experiments, clinical trials or intervention studies:
These experiments are carried out mostly on invertebrates such as flies. Invertebrate research is generally unregulated and often involves fruit flies. When it come to mammals mice are used the most with 20 million rats and mice being used each year in the US(1). By comparison the number of cats and dogs used in research is of the order of tens of thousands in the US.
Animal experiments, apart from being ethically questionable are limited in a number of respects due to the animals being used being fundamentally different to humans.
- The animals involved are not human and therefore will often respond to nutrient and drugs in different ways from humans. For instance chocolate may seem dangerous if you feed it to dogs, not so for humans. If you deprive a human of any vitamin C for long enough they will die, most other mammals make their own, and so would be fine.
- Experimental animals are normally different sizes, move differently and have different metabolic rates when compared with humans. As such energy and nutrition studies, which are often carried out on rats, cannot easily predict what would happen to humans. Rats burn up fat to keep warm, which we do hardly at all, they also move more efficiently. These are just two of a number of differences that can invalidate results.
- Animal testing is done in extremely artificial conditions. If a drug/nutrient reacts with say with a protein in cheese and there is no cheese in the animals experimental living conditions, then this will not show in the testing.
These trials normally involve following a group of people with a specific intervention intended to change the outcome, such as disease progression. It is important to have a control group as without that you will not be sure if your intervention is having any effect beyond what one would normally expect. Equally, you need a large enough group of people if you are randomly selecting participants in the intervention and control groups. If you have a small group to test it is important to match people in each group so that their disease status and progression are likely to be similar. Not an easy task!
The placebo effect is an important consideration in these trials and it is vital to ensure that participants do not know if they are in the intervention group or control group. The testers may also give clues to the participants, and this is why they also should not know, who is in which group. When testers and participants are both unaware of the groupings then the trials is said to be "double blind, placebo controlled".
Of course many medicines come in pill form, and are easy to create a placebo for. In the case of foods this is more difficult if not impossible to achieve.
Intervention studies normally look at large samples over a long duration. Due to the duration it is hard to conduct blind trials, and there are often issues with controls either wanting in the treatment if they learn it is effective.
1) Nutrition - A Health Promotion Approach