reblogged with minor modification from http://innge.net/?q=blog
The last few years have seen a revolution in scientific peer-review, with numerous initiatives to redress, what are seen as, failings of the traditional review system. One such initiative is Axios Review started by Tim Vines, former managing editor of Molecular Ecology. Hari Sridhar spoke to Tim about the philosophy behind Axios Review and what sets it apart from other efforts to improve peer-review.
Hari: What is the USP of Axios review? What sets it apart from other recent initiatives to improve peer-review, like Peerage of Science, Rubriq, Scholastica, bioRxiv etc.?
Tim: One feature of Axios that makes it different from e.g. Rubriq and Peerage of Science is that it is quite traditional. Our peer review process is very similar to a regular journal, with the tweak that reviewers are commenting on how well the paper would fit four different journals, and not just the journal it’s been submitted to. We still have an editorial board and (mostly) single blind pre-submission peer review. I think that’s helped us when we talk to journals as they intuitively understand what we’re doing, and it’s also meant that they’re more inclined to treat our round of review as their first round. I think all the new independent peer review services are filling slightly different niches and it will be interesting to see how each develops.
H: What was your personal motivation to start Axios Review?
T: I was approving a rejection decision letter for Molecular Ecology when I started having pangs of guilt about our review process. Like many others, the paper was based on decent science but it was being rejected for not being novel enough, which meant that the authors would have to reformat and submit elsewhere. In all likelihood, the decision at the next journal would also hinge on how well the paper fits that journal’s scope and novelty requirements. That got me to thinking about how we could do things more efficiently. After a few iterations I arrived at the idea of Axios.
H: Please describe how Axios Review works.
T: We’re a bit like a literary agent, but for scientific papers – we help authors improve their work and then find somewhere that wants to publish it.
Authors send us their paper, perhaps after it’s been rejected from a journal, or even as a first submission. They give us a ranked list of four target journals where they think their paper could be published. Authors often choose their journals from our target journal list, but we’re able to refer papers to most of the other major journals in our field as well.
If the paper in one of the areas we’re currently covering (Botany, Comparative Physiology, Ecology, and Evolution) we pass it to one of our editors. The editor can decide that it’s not ready for peer review and return it to the authors, normally with an encouragement to send a new version. The editor might also ask for a different list of target journals.
If the editor thinks the paper is appropriate for at least one of the target journals they select reviewers, and the editorial office sends out review requests. The reviewers write a regular review (1-2 pages on the strengths and weaknesses of the paper) and comment on how well the paper would fit each of the four target journals. Once we’ve got 2 or 3 reviews the editor makes a decision on where the paper can be referred. For example, the editor might think the paper is not a good fit to Journal 1, but that Journals 2 or 3 might be suitable.
With the authors’ permission, we then send a referral letter to Journal 2 – this is essentially a detailed presubmission enquiry that contains the reviewers’ comments, the reviewers’ names, and the paper itself. We ask the journal ‘would you like this paper to be revised based on our reviews and submitted to your journal?’ If they say no, we refer the paper to Journal 3. If they say yes, the authors revise their paper and submit by the normal route, including a response to reviewers in their cover letter. If none of the journals on the target list are keen on the paper, we work with the authors to decide where else to send the referral.
The journal can then handle the paper as it sees fit. They can send it back out for review, maybe to the same reviewer(s) Axios used, or to different people. They could also decide that the paper is strong and has been well reviewed and well revised, and accept it without further external review. The latter currently happens for about 40% of our referred papers.
No matter what the journal does, the key step is that the journal thought our reviews were good and said they wanted the paper, which changes the paper from being a direct submission (which have ~30% acceptance rates) into something like a resubmission (which have 70-90% acceptance rates). Most importantly, the journal has effectively said that the paper is likely publishable with them and so the authors’ chance of getting in is much higher.
One caveat: if the Axios editor and reviewers have a number of substantial criticisms we don’t refer the paper right away. Instead, we ask the authors to revise the paper, write a response to reviewers and send us the new version. Our editor will then take a look and make a decision on where the paper can be referred at that stage. This approach helps the authors because they’re able to address the criticisms in their paper and in their response, and the journal gets to see a strong version of the paper.
H: In a short time you have built an impressive list of Axios editors and journals (that look at Axios reviews). How did you manage that? Do you think your own reputation as a scientist played some part?
T: I was always very obscure as an academic, so that didn’t help much. Working at Molecular Ecology was my biggest source of contacts – I think I’ve handled the review process for over 10,000 papers in the last seven years. That’s a lot of emailing with authors, reviewers and editors. Having a direct experience with peer review was also a big help when talking to journals, as I’m familiar with their pain points.
H: You have decided to adopt the ‘author pays’ instead of a ‘journal pays’ model. Can you tell us a little about the thinking that went into this decision?
T: We did put a lot of thought into this. Our feeling is that there’s value for authors if we can reliably get their paper into the best possible journal within a reasonable timeframe. Our steady growth in submissions suggests that authors see this value too. On the journal side, we’ve found that journals are generally reluctant to pay for outside reviews – their editorial offices are already set up and running, so there’s less value for them in getting reviews from elsewhere as well. This is principally why we didn’t pursue the ‘journal pays’ angle. However, journals are still happy to engage with our referrals, as they see the value in us steering inappropriate submissions elsewhere and bringing them improved papers that have already been through one round of review.
H: If journals benefit from the service, shouldn’t they bear some of these costs, to reduce the burden on individual scientists?
T: You’re right that our service does benefit journals, but I don’t think blanket charge for e.g. each accepted Axios manuscript is the right way to go. Instead, our hope is that the efficiencies introduced by Axios lead to cost savings at the journal, and these are then passed along to authors in the form of reduced page charges or lower open access fees. The BioMed Central journals already do something like this, as their OA fee is reduced by $250 for Axios authors.
H: Do you worry that this “author pays” model might contribute to creating an even more unequal scientific world, where the link between scientists ability to pay and publishing in top journals becomes stronger?
T: We’ve worried about this too. We have considered offering fee waivers for authors with little or no funding, but judging by the experience at PLoS it can become very hard to decide who gets a waiver. Our approach is therefore to charge the same fee to everyone, but to keep it as low as possible. It is an order of magnitude lower than a typical Open Access fee, and should be within reach of most groups conducting biological research.
H: Initially, Axios was charging authors only after a manuscript was accepted. Now, you have moved to a system where authors pay when they receive their Axios reviews. Do you think this will change, fundamentally, how people view Axios?
T: Back when we started, we realized that it didn’t make sense to charge authors too early in the process as they simply didn’t know whether Axios would work. We therefore decided to have a ‘no win, no fee’ approach. Now that we’ve handled a few hundred papers, we’ve got data to show that our process is effective (e.g. 80% of our referred papers get accepted), and that we’re offering a useful service for authors. Since running peer-review is the most time-consuming part for myself and Hilary (our editorial office coordinator), we decided to switch to charging our fee when we’ve reviewed the paper and sent the decision to the authors. I don’t think this change has affected how people see Axios – they recognize that running a quality peer-review process means paying for a professional editorial office. Shifting the fee to the decision phase means that we’re now recouping the costs associated with that particular round of review, and that’s ultimately going to make us financially sustainable.
H: Recently, there has been a lot of debate on whether journals place too much emphasis on novelty and/or generality when evaluating manuscripts. What is your take on that and how has that shaped Axios Review?
T: Journals have two roles. First, they improve scientific papers through the peer review process. Second, they filter the literature by their scope and their novelty requirements. Both roles are very important for the community, and not easily replaced. To me, frustration about ’lack of novelty’ rejections points to the need for an intermediary like Axios – we steer authors towards journals that think their papers are novel enough, and we stop inappropriate papers from clogging up journals with more stringent novelty requirements. Authors are able to publish in the best possible journal, and journals can choose among manuscripts that fulfil to their requirements.
H: Axios Review is now two years old. How do you think it has fared so far?
T: I’m a pessimist, so I’m pleasantly surprised that we’ve had no major disasters so far. In fact, we’ve grown very well – we’ve just received our 100th submission for 2015, which is over double for the same time last year. And the 50th paper handled by Axios has just been published. Our target journal list has grown to close to 60 journals, and we’ve established good relationships with many others. Best of all, our process is working even better than I hoped, and the 80% acceptance rate for our referred papers is a real testament to the expertise of our editors and reviewers.
Disclaimer: This interview does not imply an endorsement of Axios Review by INNGE or the interviewer.
More discussion on Axios vis-à-vis other new peer-review initiatives: