Tag Archives: peer review

Spuriouser and spuriouser: the Greenfield effect

Baroness Susan Greenfield, Professor of Pharmacology at University of Oxford, has been in the news a bit recently. She’s being getting a lot of coverage due to her views on the (possible) effects of computer-based activities on children’s mental development. The most striking claims are that the use of social media tools, such as Facebook and Twitter, can lead to autism and/or shortened attention span, and more recently that playing computer games can cause dementia and alter risk-taking behaviours.

These claims have attracted a lot of criticism from bloggers and fellow scientists alike for, amongst other things, being totally unsupported by any peer reviewed research and contradictory to known biological effects. Baroness Greenfield has responded at various points by accusing her detractors of stifling open debate and likening them to those who denied a link between smoking and lung cancer. She also asserts that she’s ‘just asking questions’, which prompted an inventive stream of similarly spurious links on Twitter (#greenfieldism). The serious point behind the humour being that, without any evidence behind, it is just as valid to juxtapose internet use and autism as it is to link Rebecca Black and the Greek sovereign debt (courtesy of @alsothings).

A similar thing happened when Glenn Beck started ‘just asking questions‘, which lead to counter questions about Glenn Beck’s personal history. Although this was an overly extreme question with which to respond, it does highlight the danger of suggesting causal links without any empirical support whatsoever.

Putting unsupported claims in the spotlight to push personal viewpoints about computer games and social media, especially when they’re packaged as scientifically validated ideas, is a dangerous path to lay. The effect of technology use on brain development is clearly an important topic to consider, but any guidance must have some scientific proof behind it. Even a single study that has been through the peer review process and published in an academic journal is not enough to make the sort of bold claims Susan Greenfield has made . Only once hypotheses have been discussed, repeated, followed-up, tested on different groups by other scientists do theories start to become accepted (or rejected) by the scientific community. Baroness Greenfield hasn’t even got as far as the first step.

Tentative evidence can be found to suggest we need to seriously consider the effect of increased computer activity, such as whether violent computer games can alter brain activity and wiring. These types of studies, however, are fraught with potential confounding factors – do violent computer games change the brain activity of gamers or are gamers with particular types of brain activity more drawn to violent computer games? This is why a body of evidence is needed to eliminate these confounding factors and tease out the true causal link.

Parents have a hard enough time in drawing out the best advice, many of which appears to be based on personal opinion and gut-feeling, and so a scientist making raising serious doubts without going through the proper scientific process first can only add to anxiety and confusion. These effects are often long-lasting too. While Andrew Wakefield’s dodgy claims about the link between the MMR vaccine and autism has been thoroughly refuted by the scientific community, culminating in the original research paper being retracted and Dr Wakefield being struck off the General Medical Council register (see this for a brief history), doubts linger in parents’ minds and huge efforts are still being made to bring the vaccination rate up to the required level (e.g.).

Technology now available to children provides unparalleled access to information resources, creative tools and network sharing, and my inclination is that this is A Good Thing for broadening and challenging the mind (see Carmen Gets Around for a similar conclusion). Of course, I await the evidence to support this, but one thing is clear, computer technology has changed the way our kids behave forever. Just watch this YouTube clip…

On the benefits of breastfeeding (and getting the policy right)

Certain benefits of feeding children with breast milk over modified cow’s milk (‘infant formula’) have been well established, such as bolstered immunity, reduced risk of diabetes, and a lower risk to the mother of developing breast cancer. A recent article in the Observer introduces the possibility of an extra benefit of breast feeding, in that it could boost a child’s later cognition (as measured by IQ score). This is likely to be of interest to any parent wanting to help their child acquire the proper mental faculties to lead a fulfilling life.

by Flickr user: muskva*

This newspaper story, however, brings to bear some important cautionary tales. First of all, the types of studies the researchers have assessed mean results should be treated with a note of vigilence. It is unclear from the article whether the conclusions are based largely on the quoted researcher’s own research or a synthesis of past, published work conducted by others (more on that later), but the Institute for Social & Economic Research’s own website suggests that this is based on primary research. The Observer article does, however, indicate that the researchers analysed “studies in the fields of epidemiology and public health”. It is likely that these would have been observational studies, where the cognitive abilities of children from groups of women who had chosen to breastfeed their babies are compared with the IQ scores of children from other groups who had chosen to use infant formula. The Essex and Oxford researchers in the article appear to claim that those children who were fed breast milk, on the whole, outperformed those children who were fed formula milk. On the face of it, a neat result.

But that brings us back to familiar adage that correlation does not mean causation. It doesn’t take much scrutiny to realise that if children from more affluent families are more likely to breastfeed, then simply growing up in more comfortable, less stressful and education-rich environment could easily account for the improvement in cognitive abilities. The team that carried out the work are clearly aware of this, and claim to have corrected the data to eliminate the effects of other factors such as the family wealth, but this always introduces additional sources of potential error. This doesn’t make the research any less worthwhile, only limits the conclusions that can be drawn from it.

These types of study are often one of the first type to be done on human populations and it emphasises the experimental boundaries. If this sort of question were to be asked for, say, a rodent, the researchers could take a group of animals from the same population and split them equally into two groups. Each group could be raised in controlled environments that are identical to each other (temperature, access to food/water, number of companions, etc.). The only variable would be that one group would be suckled with mammary milk and the other fed with milk from another source. Any difference in the performance of the offspring in subsequent behavioural tests could suggest that it was down to the mammary milk. It wouldn’t prove it, though, as it could be more subtly due to increased mother-baby contact or an unknown variable, but it would be an interesting result that would warrant further investigation.

It is, understandably, improbable that this kind of experiment would be done in a human population – who could convince a mother to agree to be randomly assigned to one of two groups, breastfeeding or non-breastfeeding? – and so investigators have to use the next best thing. This could lead to targeting interventions or different levels of support at two or more groups from the same socioeconomic background and seeing whether that affects children’s IQ scores, which would allow comparison of women from the same group but receiving different treatment. But, again care needs to be taken to eradicate or account for potential confounding factors.** These limitations bedevil many researchers trying to find out more about factors affecting child development and is part of the reason why much of the guidance about parenting is not evidence-based or is, at best, based on suggestive findings.

Anyway, the second cautionary tale comes from the way the evidence is presented. The newspaper article states that the scientists are to present their findings at a policy conference. Conferences are the vital arenas at which researchers’ current theories are presented to and subsequently challenged (robustly!) by their colleagues. Many a hypothesis has been revised, re-interpreted or rubbished at a conference. Many pieces of work, such as that in the Observer article, will not have even been published in a peer reviewed journal. This means that it has not been formally reviewed and accepted as scientifically valid by other experts in the research field.

Even after this assessment and publication by a scientific journal, it is still only one piece of evidence in a vast sea of scientific work. It often takes a review of several scientific studies tackling the same research question before any firm conclusions can be made. So with this appreciation of the scientific process, the claims in the Observer article start to look less conclusive as the headline would suggest. As I mentioned earlier, the researchers are well aware of the caveats, but a casual reader may not be. It took digging beyond the article, to the ISER’s website, before I could determine the nature of the research, as the article didn’t link to anything other than more guardian.co.uk articles.

This is a problem that is widespread in science reporting – bold claims in newspapers are often based on tentative evidence, unpublished findings or misleading press releases (see ‘churnalism’) – and is a particular problem when it comes to parenting, as it can instantly affect the decisions parents make. One only has to look at scare stories about pregnant women’s sleeping position and the risk of miscarriage, vaccine safety, mobile phone use, plus countless other examples, to see the difficulties people face in picking out the sound evidence.

It is also problematic because evidence presented in the media can influence public opinion and the policies introduced by governments. Looking back to the breastfeeding story, if it is genuinely that the act of breastfeeding or a component of breast milk that boosts the chances of a child having improved cognitive scores, then the government may wish to look at improving breastfeeding support programmes. If, on the other hand, the major influence turns out to be the societal factors, then this would require a broader set of interventions that tackle social deprivation.

A further issue with policy documents, such as the one mentioned in the Observer, is that they circumvent the usual quality control for scientific research. They are often not published in peer reviewed journals and, as such, not subjected to the same rigour or openness as work that has been critiqued by other experts in the field. This opens up the possibility of a greater degree of bias and subjectivity. In this case as well, the report is not yet available (only a working paper) so the general public has to largely rely on the media’s reporting of researchers’ interpretations, something that is riddled with potential problems.

For the record, the Observer article seems fairly non-sensational – the headline is probably overly-conclusive, there’s a lack of linking to information about the conference and research, and the caveats could be included.

The causes and consequences of speculative science reporting have been dissected brilliantly elsewhere (see Martin Robbins of The Guardian on the subject and Brian Switek at wired.com for a recent example), so I will summarise by saying that it is incumbent on scientists to not inflate the nature of their findings for the sake of publicity, the universities or research institutions to issue balanced press releases, and for science journalists not to over-spin the story.

Not that I’m asking too much.

*(under Creative Commons licence, some rights reserved: http://creativecommons.org/licenses/by-nc-sa/2.0/deed.en)

**Interestingly, another recent research article does point towards a causal link of breastfeeding with infant IQ, by seeing whether the same trends are apparent in both high-income and middle-income populations