Tag Archives: policy

Where are the facts?

This was the demand from a very pleasant woman we met on a train a couple of weeks ago. She had been enquiring as to how our son, now 6 months old, was getting on with his sleeping and weaning. Not great on either count, since you ask.

She was, however, slightly taken aback that current medical advice is to start weaning at 6 months, rather than at 3-4 months as she was advised when she was raising her babies (30-40 years ago, at a guess). She was also a little dismayed that babies are now overwhelmingly placed on their backs to sleep, instead of on their front.

So, “where are the facts?”

Well, science doesn’t really work like this and scientists don’t deal with immutable facts. Research is conducted, evidence is gathered, and conclusions are drawn. And the cycle is repeated over and over and over again. Sometimes, the new evidence backs up the previous conclusions and those ideas become strengthened. Sometimes, however, the new evidence doesn’t match the previous conclusions, and so thinking may change. As such, scientists should always be prepared to change or modify their positions as new stronger, evidence emerges. In the case of babies sleeping on their backs, an accumulation of evidence over many years that showed a reduced risk of cot death that led to the change in received wisdom.

There are also different strengths of evidence and there some types of studies in which we can more confidence. A study that takes a part of a population, randomly splits them into test and control groups before monitoring the effects of a treatment versus control has far more power than a study that looks back in time at a population and tries to work out why some groups are different. Because of this, a new study, using a more reliable method may be carried out that modifies experts’ views. This is especially relevant for pregnancy and child development, as it’s hard to perform randomised controlled trials (the first type of study I mentioned above) with pregnant women and children, because parents are generally less likely to enter into anything they perceive as potentially risky.

In the case of weaning at 6 months, one only has to look at the expert review that forms the basis of WHO’s current advice to see that we aren’t dealing with cast-iron ‘facts’.

The experts concluded that 6-month exclusive (or predominant) breastfeeding is required to provide the right nutritional balance and to protect against gastrointestinal infection. After 6 months, solids should be introduced in order to meet a baby’s additional energy and nutritional requirements. There are even apparent benefits to the mother from exclusively breastfeeding for 6 months rather than 4 – mothers showed greater weight loss (if this is indeed desirable) and had a longer post-natal infertility period. It wasn’t cut-and-dried, though, as care must be taken to avoid iron deficiency in babies exclusively breastfed for 6 months, as well as special care in developing countries to ensure that babies are not malnourished as a result of poor maternal nutrition.

But, there are a few cautionary tales to add to this. First, there wasn’t a huge number of studies that looked at the health status of babies who had been exclusively breastfed for different periods to compare. Second, some of the studies that were included looked at a relatively small population of babies. Third, the studies differed in their quality and potential for bias, such as whether it was observational or whether it relied on self-reporting. Fourth, some studies didn’t show a difference in health status – is this because no effect was present, because the study design was weak, or because they hadn’t studied enough babies to show an effect? We don’t know.

This is not to undermine the advice of WHO and other health agencies – it is advice we’ve followed for our son – it just highlights how health advice is often formed and how it can be subject to change.

The potential for scientific judgement to be swayed over time also means scientists are generally more cautious and moderate than perhaps people expect or, indeed, want. It seems common for people to desire clear-cut and, above all, ‘correct’, advice. This is often reflected in sensational newspaper headlines that overstate tentative research findings, which is unhelpful in informing the public as to the true nature of the findings, as well as to the scientific process itself (see the BBC’s “Spoon feeding ‘makes babies fatter‘” versus this more reasoned analysis from NHS Choices).

This can, unfortunately, sometimes be used as a stick with which to beat scientists: “people were wrong about babies sleeping on their front, so how do you know you’re right now?” A temptingly persuasive argument to ignore those boffins, especially if they’re telling you things you don’t want to hear. But scientists (honest ones, at least) are always working from a position that is based on the best available evidence. If you reject this best available evidence, then you’re working from an even weaker position.

In the case of WHO’s advice on when to introduce solids, it made its judgement because all the evidence, despite some weaknesses, was pointing in that direction. So even though there is more research to be done – the report even acknowledges this and makes recommendations for new studies that should be carried out – it would be unwise to act against what the best available evidence is telling us. And it wouldn’t take a single study to come along to turn that around, it would take an accumulation of solid evidence that showed an opposite and consistent trend.

I didn’t quite manage to finish telling this to the lady on the train, though.

On the benefits of breastfeeding (and getting the policy right)

Certain benefits of feeding children with breast milk over modified cow’s milk (‘infant formula’) have been well established, such as bolstered immunity, reduced risk of diabetes, and a lower risk to the mother of developing breast cancer. A recent article in the Observer introduces the possibility of an extra benefit of breast feeding, in that it could boost a child’s later cognition (as measured by IQ score). This is likely to be of interest to any parent wanting to help their child acquire the proper mental faculties to lead a fulfilling life.

by Flickr user: muskva*

This newspaper story, however, brings to bear some important cautionary tales. First of all, the types of studies the researchers have assessed mean results should be treated with a note of vigilence. It is unclear from the article whether the conclusions are based largely on the quoted researcher’s own research or a synthesis of past, published work conducted by others (more on that later), but the Institute for Social & Economic Research’s own website suggests that this is based on primary research. The Observer article does, however, indicate that the researchers analysed “studies in the fields of epidemiology and public health”. It is likely that these would have been observational studies, where the cognitive abilities of children from groups of women who had chosen to breastfeed their babies are compared with the IQ scores of children from other groups who had chosen to use infant formula. The Essex and Oxford researchers in the article appear to claim that those children who were fed breast milk, on the whole, outperformed those children who were fed formula milk. On the face of it, a neat result.

But that brings us back to familiar adage that correlation does not mean causation. It doesn’t take much scrutiny to realise that if children from more affluent families are more likely to breastfeed, then simply growing up in more comfortable, less stressful and education-rich environment could easily account for the improvement in cognitive abilities. The team that carried out the work are clearly aware of this, and claim to have corrected the data to eliminate the effects of other factors such as the family wealth, but this always introduces additional sources of potential error. This doesn’t make the research any less worthwhile, only limits the conclusions that can be drawn from it.

These types of study are often one of the first type to be done on human populations and it emphasises the experimental boundaries. If this sort of question were to be asked for, say, a rodent, the researchers could take a group of animals from the same population and split them equally into two groups. Each group could be raised in controlled environments that are identical to each other (temperature, access to food/water, number of companions, etc.). The only variable would be that one group would be suckled with mammary milk and the other fed with milk from another source. Any difference in the performance of the offspring in subsequent behavioural tests could suggest that it was down to the mammary milk. It wouldn’t prove it, though, as it could be more subtly due to increased mother-baby contact or an unknown variable, but it would be an interesting result that would warrant further investigation.

It is, understandably, improbable that this kind of experiment would be done in a human population – who could convince a mother to agree to be randomly assigned to one of two groups, breastfeeding or non-breastfeeding? – and so investigators have to use the next best thing. This could lead to targeting interventions or different levels of support at two or more groups from the same socioeconomic background and seeing whether that affects children’s IQ scores, which would allow comparison of women from the same group but receiving different treatment. But, again care needs to be taken to eradicate or account for potential confounding factors.** These limitations bedevil many researchers trying to find out more about factors affecting child development and is part of the reason why much of the guidance about parenting is not evidence-based or is, at best, based on suggestive findings.

Anyway, the second cautionary tale comes from the way the evidence is presented. The newspaper article states that the scientists are to present their findings at a policy conference. Conferences are the vital arenas at which researchers’ current theories are presented to and subsequently challenged (robustly!) by their colleagues. Many a hypothesis has been revised, re-interpreted or rubbished at a conference. Many pieces of work, such as that in the Observer article, will not have even been published in a peer reviewed journal. This means that it has not been formally reviewed and accepted as scientifically valid by other experts in the research field.

Even after this assessment and publication by a scientific journal, it is still only one piece of evidence in a vast sea of scientific work. It often takes a review of several scientific studies tackling the same research question before any firm conclusions can be made. So with this appreciation of the scientific process, the claims in the Observer article start to look less conclusive as the headline would suggest. As I mentioned earlier, the researchers are well aware of the caveats, but a casual reader may not be. It took digging beyond the article, to the ISER’s website, before I could determine the nature of the research, as the article didn’t link to anything other than more guardian.co.uk articles.

This is a problem that is widespread in science reporting – bold claims in newspapers are often based on tentative evidence, unpublished findings or misleading press releases (see ‘churnalism’) – and is a particular problem when it comes to parenting, as it can instantly affect the decisions parents make. One only has to look at scare stories about pregnant women’s sleeping position and the risk of miscarriage, vaccine safety, mobile phone use, plus countless other examples, to see the difficulties people face in picking out the sound evidence.

It is also problematic because evidence presented in the media can influence public opinion and the policies introduced by governments. Looking back to the breastfeeding story, if it is genuinely that the act of breastfeeding or a component of breast milk that boosts the chances of a child having improved cognitive scores, then the government may wish to look at improving breastfeeding support programmes. If, on the other hand, the major influence turns out to be the societal factors, then this would require a broader set of interventions that tackle social deprivation.

A further issue with policy documents, such as the one mentioned in the Observer, is that they circumvent the usual quality control for scientific research. They are often not published in peer reviewed journals and, as such, not subjected to the same rigour or openness as work that has been critiqued by other experts in the field. This opens up the possibility of a greater degree of bias and subjectivity. In this case as well, the report is not yet available (only a working paper) so the general public has to largely rely on the media’s reporting of researchers’ interpretations, something that is riddled with potential problems.

For the record, the Observer article seems fairly non-sensational – the headline is probably overly-conclusive, there’s a lack of linking to information about the conference and research, and the caveats could be included.

The causes and consequences of speculative science reporting have been dissected brilliantly elsewhere (see Martin Robbins of The Guardian on the subject and Brian Switek at wired.com for a recent example), so I will summarise by saying that it is incumbent on scientists to not inflate the nature of their findings for the sake of publicity, the universities or research institutions to issue balanced press releases, and for science journalists not to over-spin the story.

Not that I’m asking too much.

*(under Creative Commons licence, some rights reserved: http://creativecommons.org/licenses/by-nc-sa/2.0/deed.en)

**Interestingly, another recent research article does point towards a causal link of breastfeeding with infant IQ, by seeing whether the same trends are apparent in both high-income and middle-income populations