I was entertained to see that Springer recently retracted a set of papers, having apparently detected a set of fake reviewers (http://retractionwatch.com/2015/08/17/64-more-papers-retracted-for-fake-reviews-this-time-from-springer-journals). The game seems to be that authors suggest reviewers who are real people but with fake emails owned by the authors. Allyson Lister, a colleague of mine, was twice the victim of this form of identity theft (http://themindwobbles.wordpress.com/2011/05/04/stolen-professional-identity/) (http://themindwobbles.wordpress.com/2011/07/19/a-new-journal-a-new-bogus-review-again-the-culprits-are-banned/) a while back so I am perhaps less surprised to hear that this is happening than some in the scientific community.

Now, of course, this is a form of fraud and is not something than I would condone. Ally lost some time chasing the situation down, but I do not think that there were any negative outcomes for her other than this. She also got to write some entertaining blog posts about the issue. But there is clearly a reputational risk as well. If the review apparently coming back from Ally had been of poor quality, then that, by itself, is an issue.

For the journals, my immediate response, though is to hope that they got the money back that they paid for these reviews. However, hidden behind the flippancy is, perhaps, a serious point.

The serious fraud office define fraud as “an act of deception intended for personal gain or to cause a loss to another party.” You can argue the former here, but they the last? The fake reviewers do not stand to gain any money from the publishers, so clearly they are not trying to cause loss to them. The best you can argue is that the authors using fake reviewers are trying to cause loss to their employers at some point in the future, gaining promotions to which they are not entitled.

The journals say that organising reviews and ensuring peer-review is a significant contribution that they are making; yet, to achieve this, they are dependent on people with who they have no existing business relationship. It’s all done on a freebie, a gentlemens agreement. Does this really make sense? If the whole system were moved toward a more formal basis, then suddenly, fake reviewers would clearly be in breach of contract, as well as clearly be defrauding the reviewers. Even the basic machinary of payment would help to prevent the fraud; instead of just requiring a name and email, journals would need to know bank accounts, tax details and so on.

All in all, I think, this supports provides another strong reason why we should pay reviewers (http://www.russet.org.uk/blog/3095).

I am not sure why, but while writing this it became clear to me that the argument would be much better supported if also represented as a sea shanty. Middle age hitting me no doubt, but here we go…

What shall we do with the fake reviewer?
What shall we do with the fake reviewer?
What shall we do with the fake reviewer?
Early in the Morning

Pay Reviewers, Sue the Fake Ones
Pay Reviewers, Sue the Fake Ones
Pay Reviewers, Sue the Fake Ones
Early in the Morning

Full Economic Costs are Rising
Full Economic Costs are Rising
Full Economic Costs are Rising
Time to Pay the Piper

What shall we do with the fake reviewer?
What shall we do with the fake reviewer?
What shall we do with the fake reviewer?
Early in the Morning

— Phillip Lord


I have discussed the issue of peer-review before, and my frustration at being asked to review for journals with high submissions fees (http://www.russet.org.uk/blog/2327), something for which we as academics gain very little credit for.

I recently fell across by an economist who acts as an editor for a journal (http://marcfbellemare.com/wordpress/10776), which has an interesting perspective. One comment he makes rings true.

When you receive a request to review, please respond, and please respond as quickly as possible. I am always baffled by how many people fail to ever respond to requests to review.

— Marc F Bellemare

I have always felt slightly guilty about this, although I do get a fairly large quantity of spam about reviewing, much of which is not particuarly directed at me and often with short deadlines. And much of which says “please log into our terrible system to accept or decline”. Which you need a password to do. Which you have to request from a different part of the terrible system. But, ultimately, I am aware that in many cases, there is a person at the end, sending a perfectly reasonable request. And I do not always reply. For me, this goes against the grain, as I try to be helpful and courteous whenever I can, as life is more pleasant this way. There are limits, of course; this is my job not my hobby, and if I gain little value from an activity, at some point the time cost is going to become unjustifiable.

Recently, there has been some interest in Publons, a new website which attempts to change this. With Publons, you send your review in and then later your send in the confirmation email from the journal about your review, and this is associated with your profile. It opens up the peer-review process because the reviews end up public, making a process that I have attempted to do for myself before (http://www.russet.org.uk/blog/2366) considerably easier. For that purpose, Publons seems like a good thing. But, also, it attempts to bring some credit to the process; over time, you build your profile up and the metrics associated with it.

Now, all of this seems like a nice idea. Reviewers gain credit. But, there again, what do these metrics mean? And how important are they to me, yet another metric from yet another social network? It’s nice to get a new Twitter follower, a facebook like or even “earn” some Eve-Online ISK. But, to justify the effort I need something more than nice.

Marc Bellemare offers a different perspective, that of an hard-working journal editor, during which he describes his feelings about the responses to requests for reviews.

Consider these quotes:

In this business, “I’m too busy” is a really bad excuse and often means “I don’t care.” Have you ever met a pre-tenure academic who was not busy? […] if you truly care about something, you will make time for it, even if it means getting up at 4 am to do it

— Marc F Bellemare

From my background, I like the idea that you should do reviewing because you care about it, because you care about the science (or economics), and you want to do a good job for yourself and your community. There again, if you have to get up at 4am to do pretty much anything, then yes, you are indeed too busy. There might be a few exceptions, I guess, mostly associated with travel which sometimes requires odd hours (I wrote some of this at 4am, and yes, it was travel), but I do not and will not work at 4am routinely. Perhaps, this makes me unusual. Maybe many of my papers have been reviewed by an exhausted colleague at 4am; it would explain the grumpy or just plain confused tone of some of them.

Interestingly, though, this rule appears not to apply to everyone.

That said, it is perfectly legitimate to decline a request to referee if you are senior in your field […] But if you are a recently minted PhD and you tell me “I just don’t have time for this right now,” […] chances are you are not going to impress anyone.

— Marc F Bellemare

Personally, I do not understand this at all. If you are a “senior” in your field, then you probably have a permanent position, it’s probably reasonably well-paid, and probably reasonable secure. If you are a “recently minted PhD”, you probably have none of these things, in addition to a significant debt that you are servicing. It’s even less clear to me why, under such circumstances, you would want to get up at 4am, however much you care. Surely, he has a stronger argument than this for why we should review? , the only other answer he gives is this:

[…] the journal’s editors will often be among the people asked to provide external letters for tenure and promotion cases, and good citizenship in your discipline tends to get rewarded.

— Marc F Bellemare

Frankly, this seems a pretty weak argument — good citizenship tends to get rewarded; just a tendency and you have to wait till the tenure committee stage of your career if you haven’t left before then? I also dislike the implicit threat (don’t review, and we’ll rubbish your tenure case), which explains why he excludes seniors, as I dislike the general tone of arrogance and entitlement in the whole article.

Putting that aside, the more important question is how much do tenure and promotion committees really care? In my experience, they are far more interested in high scoring papers for REF or that your research is sufficiently expensive (http://www.dcscience.net/2014/12/01/publish-and-perish-at-imperial-college-london-the-death-of-stefan-grimm/). The world has become metric-based (http://www.hefce.ac.uk/rsrch/metrics/), and good citizenship is a marginal issue at best.

I was entertained, therefore, to read that Nature is experimenting with actually paying reviewers for their reviews (http://blogs.nature.com/ofschemesandmemes/2015/03/27/further-experiments-in-peer-review). As the publishers say: “Authors submitting a manuscript to Scientific Reports can choose a fast-track peer-review service at an additional cost.” The costs are passed directly onto the authors of course. The practical upshot is that “Authors who opt-in to fast-track will receive an editorial decision (accept, reject or revise) with peer-review comments within three weeks”

Of course, the general feedback to this annoucement was not particularly positive (http://svpow.com/2015/04/01/pay-scientific-reports-extra-to-bypass-peer-review-altogether/), and it seems that Scientific Reports will not be carrying on with this process (http://www.nature.com/news/concern-raised-over-payment-for-fast-track-peer-review-1.17204). Concerns were expressed that it could affect the independence of review, that the reviewers would be no good (presumably because they are poor “newly-minted” PhDs), and that it would favour scientists with lots of money.

But I am less convinced that it is such a bad thing. The reviewers are not paid by the scientists directly; it’s unclear that the current system produces good reviews and the entire system favours scientists with lots of money; after all, they pay the oft-mentioned newly-minted PhDs to do the work and write their papers anyway.

Ultimately, I have come to the conclusion that the contribution of the reviewers need to be acknowledged in some way; the general way that we acknowledge work is money. The journals get paid, so why should the reviewers not do so also? And unlike publons metrics or unquantified brownie points from editors, I can spend money at the supermarket.

So, I think it is time that reviewing was put onto a reasonable footing, and the same footing that most of the rest of our lives run on. Don’t get me wrong here, I do enjoy my work: fundamentally, research retains its excitement for me, and teaching is a privilege. But I would probably do very little of either if I won the lottery and had enough to live on without. It’s my job, and I do it because I get paid.

I have, therefore, put my terms and conditions for reviewing up for everyone to see. I think that they are reasonable and proportionate. Of course, I have not worried about my reviews impressing anyone for years, and I hope that I will plan my workload so that I never have to get up at 4am. More likely, of course, the journals will just say no (one has already), and I will get asked to review less. But, either way, I will be able to respond as quickly as possible and that will remove the one piece of guilt that I feel about the whole process.

Comments welcome as always.


I’m winding by way back from a busy month with both Bio-Ontologies and ICBO, but in general I think the experience has been really positive, even if interspersing holiday and work travel has rather exhausted me. But both were in Europe and Bio-Ontologies was right next door, so I did not want to waste the opportunity.

I have a long history with Bio-Ontologies, having been a chair for many years and a informal helper before that. We steered it from an informal meeting, to having a proper programme committee, proceedings and much of the structure that it has now. I bumped into Steven Leard at the meeting, and was rather shocked to realise that the first meeting I helped out at was 14 years ago.

Strangely, though, since my last time as a chair, five or six years ago, I’ve never been once. For a few years, of course, this was quite deliberate; I was so fed up with travelling at that time of year, that I really enjoyed the rest. But since then, it has been happenstance, rather than a deliberate decision. So, it felt like a bit of a home-coming, and even if I have seen many of the people at different conferences on different occasions. Mark Musen gave a interesting keynote: I was, at the time, rather unconvinced by this hypothesis that we don’t spend enough time arguing (I mean, ontologists, really?). A more nuanced reading of what he said though, is that we should assess and re-assess our practices against the evidence of our experience. I cannot help but agree with this, and it has made me think again. More on that later, perhaps.

It was nice to go to Dublin, also, as it was my first time. Nice city, deeply integrated with it’s river. We had some nice feed, in some good resturants and cafes, and a blissful absence of Irish theme pubs. The conference venue was good also, even if it does look like a vacuum cleaner from outside.

ICBO was a different kettle of fish, though. At four days (many of the delegates go for the whole thing) it’s long, and I felt rather stretched by the end (I’m on the plane home now, after a very early start, which might be colouring my vision). This does give plenty of time for slightly longer and more detailed presentations; the workshops were small, intense and full of discussion. Likewise, the poster and demo sessions. I rather blitzed the conference with Tawny-OWL (http://www.russet.org.uk/blog/3030), and Lentic (http://www.russet.org.uk/blog/3035). In total, I gave 1 tutorial; 1 paper; 1 demo; 1 flash update on the demo and 1 feedback session on the tutorial. People seemed genuinely sympathetic and a little sad when my cute Tawny-OWL logo went 404 during the flash update. For those who missed it, the logo is online, as is the logo for lentic which lacks in cuteness, but is rather more dramatic.

I got some good feedback, was surprised to win the best demo session (I mean, it was entirely text running in Emacs, and very laggy, running on my 5 year-old netbook). The second place was James Overton’s Robot. I am told between the two of us we got a very large percentage of the vote. I think this is an interesting result, because it strongly suggests to me that, for ICBO attendees there is disatisfaction over current tooling. Ontologies are being more programmatically developed and I cannot help but feel that this is the future.

I thought I had never been to Lisbon before, but on getting there I realised I had been, about 20 years ago; the story of is long, and not that interesting so I will skip describing it here. This time I had a better look and I will not forget again. Lisbon is very nice city indeed; while it’s architectural elegance may not be quite up there with Rome (or even Milan), it’s certainly not far behind, but as a city built into and with its geography it is stunning.

In summary, an interesting month from an ontology perspective and one that I enjoyed very much. While I might have wanted for something a little less hectic (especially, as I interspersed my holidays (http://www.russet.org.uk/blog/3091)), it has left me with the sense that ontologies are both a productive part of the bioinformatics environment and a sense that there is more to come.


I think that I have missed the last few years of holiday posts. They are no doubt getting undoubtely getting increasingly in-appropriate as this blog completes it’s transition to an open lab-book. Still, I am stuck in an airport post-ICBO, so why not?

I have, to my knowledge, only been to Greece for work before, so I was not entirely sure what we were going to get, beyond the climate and food, which I figured was likely to be pretty similar to Turkey where we were two year before (actually, we could see the Turkish coast we had swam from). In this, I was correct. It’s reasonably hot. The ground is dry although there is no shortage of water (although, I didn’t know that Rhodes has almost no natural running water). The first week there was a very pleasant wind blowing, although I believe that this is unusual; it had gone by the second week.

Pefkos it a pretty small town and it is entirely tourist-driven; the accommodation is largely populated by the UK package tour operators, and it is rather “English breakfast”, but not offensively so. Our hotel backed onto the beach and, unlike Turkey, Greek beaches are not private owned; long may this last, although I have no doubt that somewhere right-wing economics are busy coming up with explanations as to why this is really a bad thing.

Rhodes itself is very beautiful, although we saw it largely by tour bus; while this avoids argument as to who is going to get stuck with the driving, the lack of control and the joyless, breathless, pillar-to-post excursion experience brings hassles of its own. Still, unlikely Turkey, the attempts to sell junk (Onyx in Turkey, Olive oil in Rhodes) to tourists was much less relentless. Likewise, the prices were reasonable, stated up-front on the menu and stuck to. Best things to see, if you are interested, were Rhodes town itself, and the butterfly valley, although this is not a major discovery. If you go to Rhodes, people will give you opinions on the 10 options pretty quick.

I believe that as a tourist town Pefkos is unusual on Rhodes in actually having Resturants at all, as most of the other towns have full-board hotels and there is no market. This seemed like a good thing, as hotel catering is rarely reliable. However, the resturant food was rather disappointing from a veggie perspective. While the Greeks seem curiously smug about their cuisine as they have discovered both the grill and the oven, their main dishes I find lack variety or composition, so I was not expecting much there. But, their mezze are great and even if they are largely replicated identically between resturants, so I expected to eat well.

The problem was that the mezze were too big; by the time you’d bought, say, some hummous, with a bit of bread, you already had half the food that you were ever going to eat. If I wanted to sample the variety they had on offer in one meal, I’d have to buy 5 dishes, then throw most of each away. I tried asking about this, but Greece (or tourist Greece) is not like Italy; the menu is a description of what you can choose from, not a starting point for negotiation. I didn’t starve, but it could have been so much better.

The best meal I got was from a buffet at the hotel, where I could pick what I wanted; oh, the irony. This at their themed “Greek night” — a pretty obscure theme for a Greek hotel, on a Greek island, in Greece, but they attacked it with gusto: they seemed genuinely disappointed that I didn’t drink the Ouzo (“it’s a spirit, with aniseed”, they explained); then, the Greek dancers, to the sounds of a bousouki player, and attempts to get us to join the knees up, after which the band switched between the top 10 bousouki standards, interspersed with the top 10 chart hits (Taylor Swift sounds better on bousouki than in the original though, perhaps, that is less surprising than it seems on first thoughts). It was all rather fun, even if haunted by the shade of Monty Python.

The backdrop to the holiday was the economic crisis; the Greeks voted no in the middle. It seemed a brave attempt to resist the bullying and hectoring; reform is needed, indeed, but it is in the international businesses who think that tax is for the little people, and with the self-appointed guardians of the economy who treat social justice and welfare as a luxury to be disposed of, as soon as the opportunity arises. Perhaps it seems unlikely, given that their leaders have already capitulated, but I hope that the Greek people can hold on to as many of these things as they can. Maybe, the Greeks can help lead the world back toward democracy and away from the current hegemony; they have a history in doing this, after all.

If they can keep their free beaches which they were kind enough to share with us on our holiday, and which I hope they will share with us again in future, that would be nice as well.

I am pleased to announce this 1.4.0 release of Tawny-OWL. It is with some surprise that I find that it is about 8 months since the last release, which is indicative of the relatively stable state of Tawny.

The main addition to this release has been further support for patterns, which now include what I am calling “facets”, as well as some general functions for grouping the entities created using a pattern with annotations. I have had a lot of discussion about the implementation of this feature and that it should be useful in practice.

The main motivation behind making this release now is the development of a Tawny Tutorial for ICBO 2015. The features are stable for the release are stable, so the release has not been rushed out the door, but I wanted to use them in the tutorial as they really simplify some aspects.

Tawny-OWL 1.4.0 is now available on Clojars.

While I have been working on a manual called Take Wing for Tawny-OWL (http://www.russet.org.uk/blog/3030) for some time, it is far from finished. In the meantime, I am giving a tutorial at this years ICBO, and the slides for this are now relatively advanced, although I have a few more sections and some checking to do. The full tutorial is available (http://github.com/phillord/tawny-tutorial), and I think it offers are fairly comprehensive guide to basic Tawny-OWL usage.

I am fairly pleased with the tutorial as it stands. It is written up as a semantic document using my lentic package (http://www.russet.org.uk/blog/3047) which seems to be working well (although I discovered a few bugs with the asciidoc support in the course of writing the tutorial!).

For anyone who is thinking of coming, the pre-requisities for the tutorial are now online. I am planning to do a large part of the tutorial in a “follow-my-leader” fashion; it’s always difficult to predict the state of the networks at these events, so it would help significantly if at least some of the people coming could work through this short document before hand.