Wednesday, 24 June 2015

How the media spun the Tim Hunt story


 I had vowed not to blog about the Tim Hunt affair. I thought everything that could have been said had been said, and I'd made my own position clear in a comment on Athene Donald's blog, and in a comment in the Independent.
But then I came across Stephen Ballentyne's petition to "Bring Back Tim Hunt", and I was transported back five years to my first ever blog post on "Academic Mobbing in Cyberspace," a strange tale about sex, fruitbats and internet twittermobs. I started blogging in 2010 because I wanted to highlight how the internet encourages people to jump in to support causes without really examining the facts of the matter. The Ballentyne petition points to an uncannily similar conclusion.
Let me start out by saying I am not arguing against people's right to take Tim Hunt's side. As many people have noted, he is a well-liked man who has done amazing science and there are many women as well as men who will speak up for him as a supporter of female scientists. Many of those who support him do so in full knowledge of the facts, out of a sense of fairness and, in the case of those who know him personally, loyalty.
My concern is about the number of signatories of Ballentyne's petition who have got themselves worked up into a state of indignation on the basis of wrong information. There are three themes that run through the comments that many people have posted:
a) They think that Tim Hunt has been sacked from his job
b) They think he is 'lost to science'
c) They think University College London (UCL) fired him in response to a 'Twitter mob'.
None of these things is true. (a) Hunt is a retired scientist who was asked to resign from an honorary position.  That's shaming and unpleasant, but an order of magnitude different from being sacked and losing your source of income. (b) Hunt continues to have an affiliation to the Crick Institute – a flagship research centre that recently opened in Central London. (c) UCL are explicit that their acceptance of his resignation from an honorary position had nothing to do with the reaction on social media.
So why do people think these things? Quite simply, this is the interpretation that has been put about in many of the mainstream media. The BBC has been particularly culpable. The Today programme on Radio 4 ran a piece which started by saying Hunt had 'lost his job'. This was a couple of days after the UCL resignation, when any self-respecting journalist would have known this to be false. Many newspapers fuelled the flames. An interview with Boris Johnson on the BBC website added the fictitious detail that Hunt had been sacked by the Royal Society. He is in fact still a Fellow – he has simply been asked to step down from a Royal Society committee. It is interesting to ask why the media are so keen to promote the notion of Hunt as victim, cruelly dismissed by a politically correct university.
It's fascinating analysing the comments on the petition.  After deleting duplicates, there were 630 comments. Of those commenters where gender could be judged, 71% were male. Rather surprisingly, only 52% of commenters were from the UK, and 12% from the US, with the remainder scattered all over the world.
There were 93 comments that explicitly indicated they thought that Hunt had been sacked from his job, and/or was now 'lost to science' – and many more that called for his 'reinstatement', where it was unclear whether they were aware this was an honorary position.  They seemed to think that Hunt was dependent on UCL for his laboratory work, and that he had a teaching position. For instance, "Don't let the world lose a great scientist and teacher over a stupid joke." I would agree with them that if he had been sacked from a regular job, then UCL's action would have been disproportionate. However, he wasn't.
Various commentators drew comparisons with repressive fascist or Marxist states, e.g. "It is reminiscent of the cultural revolution in China where 'revisionist' professors were driven out of their offices by their prospective students, to do farm labour." And there was an awful lot of blaming of women, Twitter and feminism in general, with comments such as "Too much of this feminist ranting going on. Men need to get their spines back and bat it away" and "A respected and competent scientist has been hounded out of his job because of an ignorant baying twitter mob who don't happen to like his views". And my favourite: "What he said was a joke. If lesbian feminist women can't take a joke, then they are the joke." Hmm.
It's unfortunate that the spread of misinformation about Hunt's circumstances have muddied the waters in this discussion.  A minority of those commenting on Ballentyne's petition are genuine Hunt supporters who are informed of the circumstances; the bulk seem to be people who are concerned because they have believed the misinformation about what happened to Hunt; a further set are opportunistic misogynists who do Hunt no favours by using his story as a vehicle to support their dislike of women. There is a much more informed debate in the comments section on Athene Donald's blog, which I would recommend to anyone who wants to understand both sides of the story.






Sunday, 7 June 2015

My collapse of confidence in Frontiers journals



Frontiers journals have become a conspicuous presence in academic publishing since they started in 2007 with the advent of Frontiers in Neuroscience. When they were first launched, I, like many people, was suspicious. This was an Open Access (OA) online journal where authors paid to publish, raising questions about the academic rigour of the process. However, it was clear that the publishers had a number of innovative ideas that were attractive to authors, with a nice online interface and a collaborative review process that made engagement with reviewers more of a discussion than a battle with anonymous critics. Like many other online OA journals, the editorial decision to publish was based purely on an objective appraisal of the soundness of the study, not on a subjective evaluation of importance, novelty or interest. As word got round that respectable scientists were acting as editors, reviewers and authors of paper in Frontiers, people started to view it as a good way of achieving fast and relatively painless publication, with all the benefits of having the work openly available and accessible to all.
The publishing model has been highly successful. In 2007, there were 45 papers published in Frontiers in Neuroscience, whereas in 2014 it was 3,012 (data from Scopus search for source title Frontiers in Neuroscience, which includes Frontiers journals in Human Neuroscience, Cellular Neuroscience, Molecular Neuroscience, Behavioral Neuroscience, Systems Neuroscience, Integrative Neuroscience, Synaptic Neuroscience, Aging Neuroscience, Evolutionary Neuroscience and Computational Neuroscience). If all papers attracted the author fee of US$1900 (£1243) for a regular article, this would bring in £3.7 million pounds in 2014: the actual income would be less than this because some articles are cheaper, but it's clear that the income is any in case substantial, especially since the journal is online and there are no print costs. But this is just the tip of the iceberg. Frontiers has expanded massively since 2007 to include a wide range of disciplines.  A Scopus search for articles with journal title that includes "Frontiers in" found over 54,000 articles since 2006, with 10,555 published in 2014.
With success, however, have come growing rumbles of discontent. Questions are being raised about the quality of editing and reviewing in Frontiers.  My first inkling of this was a colleague told me he would not review for Frontiers because his name was published with the article. This wasn't because he wanted confidentiality; rather he was concerned that it would appear he had given approval for the article, when in fact he had major reservations.
Then, there have been some very public criticisms of editorial practices at Frontiers. The first was associated with the retraction of a paper that claimed climate denialism was associated with a more general tendency to advocate conspiracy theories. Papers on this subject are always controversial and this one was no exception, attracting complaints to the editor. The overall impression from the account in Retraction Watch was that the editor caved in to legal threats, thereby letting critics of climate change muzzle academic freedom of speech. This led to the resignation of one Frontiers editor**.
Next, there was a case that posed the opposite problem: the scientific establishment were outraged that a paper on HIV denial had been published, and argued that it should be retracted. The journal editor decided that the paper should not be retracted, but instead rebranded it as Opinion – see Retraction Watch account here.
Most recently, in May 2015 there was a massive upset when editors of the journals Frontiers in Medicine and Frontiers in Cardiovascular Medicine mounted a protest at the way the publisher was bypassing their editorial oversight and allocating papers to associate editors who could accept them without the knowledge of the editor in chief. The editors protested and published a manifesto of editorial independence, leading to 31 of them being sacked by the publisher.   
All of these events have chipped away at my confidence in Frontiers journals, but it was finally exploded completely when someone on Twitter pointed me to this article entitled "First time description of dismantling phenomenon" by Laurence Barrer and Guy Giminez from Aix Marseille Universit√©, France. I had not realised that Frontiers in Psychology had a subsection on Psychoanalysis and Neuropsychoanalysis, but indeed it does, and here was a paper proposing a psychoanalytic account of autism. The abstract states: "The authors of this paper want to demonstrate that dismantling is the main defense mechanism in autism, bringing about de-consensus of senses." Although the authors claim to be adopting a scientific method for testing a hypothesis, it is unclear what would constitute disproof. Their evidence consists of interpreting known autistic characteristics, such as fascination with light, in psychoanalytic terms. The source of dismantling is attributed to the death drive. This reads like the worst kind of pseudoscience, with fancy terminology and concepts being used to provide evidence for a point of view which is more like a religious belief than a testable idea. I wondered who was responsible for accepting this paper.  The Editor was Valeria Vianello Dri, Head of Child and Adolescent Neuropsychiatry Units in Trento, Italy. No information on her biography is provided on the Frontiers website. She lists four publications: these are all on autism genetics. All are multi-authored and she is not first or last author on any of these*. A Google search confirmed she has an interest in psychoanalysis but I could find no further information to indicate that she had any real experience of publishing scientific papers. There were three reviewers: the first two had no publications listed on their Frontiers profiles; the third had a private profile, but a Google search on his name turned up a CV but it did not include any peer-reviewed publications.
So it seems that Frontiers has opened the door to a branch of pseudoscience to set up its own little circle of editors, reviewers and authors, who can play at publishing peer-reviewed science. I'm not saying all people with an interest in psychoanalysis should be banished: if they do proper science, they can publish that in regular journals without needing this kind of specialist outlet. But this section of Frontiers is a disastrous development; there is no evidence of scientific rigour, yet the journal gives credibility to a pernicious movement that is particularly strong in France and Argentina, which regards psychoanalysis as the preferred treatment for autism. Many experts have pointed out that this approach is not evidence-based, but worse still, in some of its manifestations it amounts to maltreatment.  What next, one wonders? Frontiers in homeopathy?
Like the protesting editors of Frontiers in Medicine, I think the combined evidence is that Frontiers has allowed the profit motive to dominate. They should be warned, however, that once they lose a reputation for publishing decent science, they are doomed. I've already heard it said that someone on a grants review panel commented that a candidate's articles in Frontiers should be disregarded. Unless these journals can recover a reputation for solid science with proper editing and peer review, they will find themselves shunned.


*The Frontiers biography suggests she is last author on a paper in 2008, but the author list proved to be incomplete.
** Correction: Shortly after I posted this, Stephan Lewandowsky wrote to say that there were 3 editors who resigned over the RF retraction, plus another one voicing intense criticism

Sunday, 17 May 2015

Will traditional science journals disappear?


The Royal Society has been celebrating the 350th anniversary of Philosophical Transactions, the world's first scientific journal, by holding a series of meetings on the future of scholarly scientific publishing. I followed the whole event on social media, and was able to attend in person for one day. One of the sessions followed a Dragon's Den format, with speakers having 100 seconds to convince three dragons – Onora O'Neill, Ben Goldacre and Anita de Waard – of the fund-worthiness of a new idea for science communication. Most were light-hearted, and there was a general mood of merriment, but the session got me thinking about what kind of future I would like to see. What I came up with was radically different from our current publishing model.

Most of the components of my dream system are not new, but I've combined them into a format that I think could work. The overall idea had its origins in a blogpost I wrote in 2011, and has points in common with David Colquhoun's submission to the dragons, in that it would adopt a web-based platform run by scientists themselves. This is what already happens with the arXiv for the physical sciences and bioRxiv for biological sciences. However, my 'consensual communication' model has some important differences. Here's the steps I envisage an author going through:
1.  An initial protocol is uploaded before a study is done, consisting only of introduction, and a detailed methods section and analysis plan, with the authors anonymised. An editor then assigns reviewers to evaluate it. This aspect of the model draws on features of registered reports, as implemented in the neuroscience journal, Cortex.  There are two key scientific advantages to this approach; first, reviewers are able to improve the research design, rather than criticise studies after they have been done. Second, there is a record of what the research plan was, which can then be compared to what was actually done. This does not confine the researcher to the plan, but it does make transparent the difference between planned and exploratory analyses.
2. The authors get a chance to revise the protocol in response to the reviews, and the editor judges whether the study is of an adequate standard, and if necessary solicits another round of review. When there is agreement that the study is as good as it can get, the protocol is posted as a preprint on the web, together with the non-anonymised peer reviews. At this point the identity of authors is revealed.
3. There are then two optional extra stages that could be incorporated:
a) The researcher can solicit collaborators for the study. This addresses two issues raised at the Royal Society meeting – first, many studies are underpowered; duplicating a study across several centres could help in cases where there are logistic problems in getting adequate sample sizes to give a clear answer to a research question. Second, collaborative working generally enhances reproducibility of findings.
b)  It would make sense for funding, if required, to be solicited at this point – in contrast to the current system where funders evaluate proposals that are often only sketchily described. Although funders currently review grant proposals, there is seldom any opportunity to incorporate their feedback – indeed, very often a single critical comment can kill a proposal.
4. The study is then completed, written up in full, and reviewed by the editor. Provided the authors have followed the protocol, no further review is required. The final version is deposited with the original preprint, together with the data, materials and analysis scripts.
5. Post-publication discussion of the study is then encouraged by enabling comments.
What might a panel of dragons make of this? I anticipate several questions.
Who would pay for it? Well, if arXiv is anything to go by, costs of this kind of operation are modest compared with conventional publishing. They would consist of maintaining the web-based platform, and covering the costs of editors. The open access journal PeerJ has developed an efficient e-publishing operation and charges $99 per author per submission. I anticipate a similar charge to authors would be sufficient to cover costs.
Wouldn't this give an incentive to researchers to submit poorly thought-through studies? There are two answers to that. First, half of the publication charge to authors would be required at the point of initial submission. Although this would not be large (e.g. £50) it should be high enough to deter frivolous or careless submissions. Second, because the complete trail of a submission, from pre-print to final report, would be public, there would be an incentive to preserve a reputation for competence by not submitting sloppy work.
Who would agree to be a reviewer under such a model? Why would anyone want to put their skills in to improving someone else's work for no reward? I propose there could be several incentives for reviewers. First, it would be more rewarding to provide comments that improve the science, rather than just criticising what has already been done. Second, as a more concrete reward, reviewers could have submission fees waived for their own papers. Third, reviews would be public and non-anonymised, and so the reviewer's contribution to a study would be apparent. Finally, and most radically, where the editor judges that a reviewer had made a substantial intellectual contribution to a study, then they could have the option of having this recognised in authorship.
Why would anyone who wasn't a troll want to comment post-publication? We can get some insights into how to optimise comments from the model of the NIH-funded platform PubMed Commons. They do not allow anonymous comments, and require that commenters have themselves authored a paper that is listed on PubMed.  Commenters could also be offered incentives such as a reduction of submission costs to the platform.  To this one could add ideas from commercial platforms such as e-Bay, where sellers are rated by customers, so you can evaluate their reputation. It should be possible to devise some kind of star rating – both for the paper being commented on, and for the person making the comment. This could provide motivation for good commenters and make it easier to identify the high quality papers and comments.
I'm sure that any dragon from the publishing world would swallow me up in flames for these suggestions, as I am in effect suggesting a model that would take commercial publishers out of the loop. However, it seems worth serious consideration, given the enormous sums that could be saved by universities and funders by going it alone.  But the benefits would not just be financial; I think we could greatly improve science by changing the point in the research process when reviewer input occurs, and by fostering a more open and collaborative style of publishing.


This article was first published on the Guardian Science Headquarters blog on 12 May 2015