Archive for the ‘Communication’ Category

I was entertained to see that Springer recently retracted a set of papers, having apparently detected a set of fake reviewers (http://retractionwatch.com/2015/08/17/64-more-papers-retracted-for-fake-reviews-this-time-from-springer-journals). The game seems to be that authors suggest reviewers who are real people but with fake emails owned by the authors. Allyson Lister, a colleague of mine, was twice the victim of this form of identity theft (http://themindwobbles.wordpress.com/2011/05/04/stolen-professional-identity/) (http://themindwobbles.wordpress.com/2011/07/19/a-new-journal-a-new-bogus-review-again-the-culprits-are-banned/) a while back so I am perhaps less surprised to hear that this is happening than some in the scientific community.

Now, of course, this is a form of fraud and is not something than I would condone. Ally lost some time chasing the situation down, but I do not think that there were any negative outcomes for her other than this. She also got to write some entertaining blog posts about the issue. But there is clearly a reputational risk as well. If the review apparently coming back from Ally had been of poor quality, then that, by itself, is an issue.

For the journals, my immediate response, though is to hope that they got the money back that they paid for these reviews. However, hidden behind the flippancy is, perhaps, a serious point.

The serious fraud office define fraud as “an act of deception intended for personal gain or to cause a loss to another party.” You can argue the former here, but they the last? The fake reviewers do not stand to gain any money from the publishers, so clearly they are not trying to cause loss to them. The best you can argue is that the authors using fake reviewers are trying to cause loss to their employers at some point in the future, gaining promotions to which they are not entitled.

The journals say that organising reviews and ensuring peer-review is a significant contribution that they are making; yet, to achieve this, they are dependent on people with who they have no existing business relationship. It’s all done on a freebie, a gentlemens agreement. Does this really make sense? If the whole system were moved toward a more formal basis, then suddenly, fake reviewers would clearly be in breach of contract, as well as clearly be defrauding the reviewers. Even the basic machinary of payment would help to prevent the fraud; instead of just requiring a name and email, journals would need to know bank accounts, tax details and so on.

All in all, I think, this supports provides another strong reason why we should pay reviewers (http://www.russet.org.uk/blog/3095).

I am not sure why, but while writing this it became clear to me that the argument would be much better supported if also represented as a sea shanty. Middle age hitting me no doubt, but here we go…

What shall we do with the fake reviewer?
What shall we do with the fake reviewer?
What shall we do with the fake reviewer?
Early in the Morning

Pay Reviewers, Sue the Fake Ones
Pay Reviewers, Sue the Fake Ones
Pay Reviewers, Sue the Fake Ones
Early in the Morning

Full Economic Costs are Rising
Full Economic Costs are Rising
Full Economic Costs are Rising
Time to Pay the Piper

What shall we do with the fake reviewer?
What shall we do with the fake reviewer?
What shall we do with the fake reviewer?
Early in the Morning

— Phillip Lord

Bibliography

I have discussed the issue of peer-review before, and my frustration at being asked to review for journals with high submissions fees (http://www.russet.org.uk/blog/2327), something for which we as academics gain very little credit for.

I recently fell across by an economist who acts as an editor for a journal (http://marcfbellemare.com/wordpress/10776), which has an interesting perspective. One comment he makes rings true.

When you receive a request to review, please respond, and please respond as quickly as possible. I am always baffled by how many people fail to ever respond to requests to review.

— Marc F Bellemare

I have always felt slightly guilty about this, although I do get a fairly large quantity of spam about reviewing, much of which is not particuarly directed at me and often with short deadlines. And much of which says “please log into our terrible system to accept or decline”. Which you need a password to do. Which you have to request from a different part of the terrible system. But, ultimately, I am aware that in many cases, there is a person at the end, sending a perfectly reasonable request. And I do not always reply. For me, this goes against the grain, as I try to be helpful and courteous whenever I can, as life is more pleasant this way. There are limits, of course; this is my job not my hobby, and if I gain little value from an activity, at some point the time cost is going to become unjustifiable.

Recently, there has been some interest in Publons, a new website which attempts to change this. With Publons, you send your review in and then later your send in the confirmation email from the journal about your review, and this is associated with your profile. It opens up the peer-review process because the reviews end up public, making a process that I have attempted to do for myself before (http://www.russet.org.uk/blog/2366) considerably easier. For that purpose, Publons seems like a good thing. But, also, it attempts to bring some credit to the process; over time, you build your profile up and the metrics associated with it.

Now, all of this seems like a nice idea. Reviewers gain credit. But, there again, what do these metrics mean? And how important are they to me, yet another metric from yet another social network? It’s nice to get a new Twitter follower, a facebook like or even “earn” some Eve-Online ISK. But, to justify the effort I need something more than nice.

Marc Bellemare offers a different perspective, that of an hard-working journal editor, during which he describes his feelings about the responses to requests for reviews.

Consider these quotes:

In this business, “I’m too busy” is a really bad excuse and often means “I don’t care.” Have you ever met a pre-tenure academic who was not busy? […] if you truly care about something, you will make time for it, even if it means getting up at 4 am to do it

— Marc F Bellemare

From my background, I like the idea that you should do reviewing because you care about it, because you care about the science (or economics), and you want to do a good job for yourself and your community. There again, if you have to get up at 4am to do pretty much anything, then yes, you are indeed too busy. There might be a few exceptions, I guess, mostly associated with travel which sometimes requires odd hours (I wrote some of this at 4am, and yes, it was travel), but I do not and will not work at 4am routinely. Perhaps, this makes me unusual. Maybe many of my papers have been reviewed by an exhausted colleague at 4am; it would explain the grumpy or just plain confused tone of some of them.

Interestingly, though, this rule appears not to apply to everyone.

That said, it is perfectly legitimate to decline a request to referee if you are senior in your field […] But if you are a recently minted PhD and you tell me “I just don’t have time for this right now,” […] chances are you are not going to impress anyone.

— Marc F Bellemare

Personally, I do not understand this at all. If you are a “senior” in your field, then you probably have a permanent position, it’s probably reasonably well-paid, and probably reasonable secure. If you are a “recently minted PhD”, you probably have none of these things, in addition to a significant debt that you are servicing. It’s even less clear to me why, under such circumstances, you would want to get up at 4am, however much you care. Surely, he has a stronger argument than this for why we should review? , the only other answer he gives is this:

[…] the journal’s editors will often be among the people asked to provide external letters for tenure and promotion cases, and good citizenship in your discipline tends to get rewarded.

— Marc F Bellemare

Frankly, this seems a pretty weak argument — good citizenship tends to get rewarded; just a tendency and you have to wait till the tenure committee stage of your career if you haven’t left before then? I also dislike the implicit threat (don’t review, and we’ll rubbish your tenure case), which explains why he excludes seniors, as I dislike the general tone of arrogance and entitlement in the whole article.

Putting that aside, the more important question is how much do tenure and promotion committees really care? In my experience, they are far more interested in high scoring papers for REF or that your research is sufficiently expensive (http://www.dcscience.net/2014/12/01/publish-and-perish-at-imperial-college-london-the-death-of-stefan-grimm/). The world has become metric-based (http://www.hefce.ac.uk/rsrch/metrics/), and good citizenship is a marginal issue at best.

I was entertained, therefore, to read that Nature is experimenting with actually paying reviewers for their reviews (http://blogs.nature.com/ofschemesandmemes/2015/03/27/further-experiments-in-peer-review). As the publishers say: “Authors submitting a manuscript to Scientific Reports can choose a fast-track peer-review service at an additional cost.” The costs are passed directly onto the authors of course. The practical upshot is that “Authors who opt-in to fast-track will receive an editorial decision (accept, reject or revise) with peer-review comments within three weeks”

Of course, the general feedback to this annoucement was not particularly positive (http://svpow.com/2015/04/01/pay-scientific-reports-extra-to-bypass-peer-review-altogether/), and it seems that Scientific Reports will not be carrying on with this process (http://www.nature.com/news/concern-raised-over-payment-for-fast-track-peer-review-1.17204). Concerns were expressed that it could affect the independence of review, that the reviewers would be no good (presumably because they are poor “newly-minted” PhDs), and that it would favour scientists with lots of money.

But I am less convinced that it is such a bad thing. The reviewers are not paid by the scientists directly; it’s unclear that the current system produces good reviews and the entire system favours scientists with lots of money; after all, they pay the oft-mentioned newly-minted PhDs to do the work and write their papers anyway.

Ultimately, I have come to the conclusion that the contribution of the reviewers need to be acknowledged in some way; the general way that we acknowledge work is money. The journals get paid, so why should the reviewers not do so also? And unlike publons metrics or unquantified brownie points from editors, I can spend money at the supermarket.

So, I think it is time that reviewing was put onto a reasonable footing, and the same footing that most of the rest of our lives run on. Don’t get me wrong here, I do enjoy my work: fundamentally, research retains its excitement for me, and teaching is a privilege. But I would probably do very little of either if I won the lottery and had enough to live on without. It’s my job, and I do it because I get paid.

I have, therefore, put my terms and conditions for reviewing up for everyone to see. I think that they are reasonable and proportionate. Of course, I have not worried about my reviews impressing anyone for years, and I hope that I will plan my workload so that I never have to get up at 4am. More likely, of course, the journals will just say no (one has already), and I will get asked to review less. But, either way, I will be able to respond as quickly as possible and that will remove the one piece of guilt that I feel about the whole process.

Comments welcome as always.

Bibliography

Godwin’s law says that sooner or later every argument on the internet ends up with someone being called a Nazi. Interestingly in the Open Access debate, this has turned out not to be true; in this case, it’s been communists instead.

Perhaps the best known example of this was the Jeffrey Beale rant, which uses the term “anti-corporatist” and “collectivizing”. A more direct communist angle comes from Moshe Vardi at ACM. He defends the idea that publishing is expensive because of all the fixed costs, something seen from the ACM before, and which I have commented on (http://www.russet.org.uk/blog/1924). He also makes the tired confusion that “free software” is the same thing as “anti-copyright”.

Now, personally, I do not particularly mind being called a communist — I do have left-leaning political views, but in the US “communist” is essentially used as a short-hand for being personally responsible for Stalinist purges.

For me OA is just a small part of the issue. I do want OA because I get irritated that I have to read papers at work because at home I can’t login, as well as the slightly more philanthropic ideal that scientific knowledge should be free.

But I also want the entire publication process to be easier (http://www.russet.org.uk/blog/2248). I want a single submission format, free from the stupidity of multiple formats. I want peer review to be open and acknowledged. I want the process to take less time, less effort and less money. I want colour figures, I want animation (http://www.russet.org.uk/blog/2189). OA does not necessarily provide this: both OUP (10.1093/bioinformatics/bts372) and PLOS (10.1371/journal.pone.0075541) were painful. However, arXiv for example, provides cheap, rapid and simple publication at $7 a paper. WordPress manages billions of articles as a loss leader.

Open-Access is only part of the issue. Scientific authoring is a hard, arduous and difficult process. Scientific publishing should not be. This is the issue.

Bibliography

I have finally got around to releasing kblog-include, a plugin that I first alluded sometime previously (http://www.russet.org.uk/blog/2367). This plugin allows WordPress to transclude content from arXiv and potentially any OAI-PMH repository. When used from arXiv the date, authors and title are set (and advertised if kblog-metadata is installed also), and the abstract is added in place.

I’ve been using this for sometime now; in fact my last article (http://www.russet.org.uk/blog/2948) is an example.

Feedback welcome, as always.

Bibliography


Abstract

Semantic publishing can enable richer documents with clearer, computationally interpretable properties. For this vision to become reality, however, authors must benefit from this process, so that they are incentivised to add these semantics. Moreover, the publication process that generates final content must allow and enable this semantic content. Here we focus on author-led or "grey" literature, which uses a convenient and simple publication pipeline. We describe how we have used metadata in articles to enable richer referencing of these articles and how we have customised the addition of these semantics to articles. Finally, we describe how we use the same semantics to aid in digital preservation and non-repudiability of research articles.

  • Phillip Lord
  • Lindsay Marshall


Plain English Summary

Academic literature makes heavy of references; effectively links to other, previous work that supports, or contradicts the current work. This referencing is still largely textual, rather than using a hyperlink as is common on the web. As well as being time consuming for the author, it also difficult to extract the references computationally, as the references are formatted in many different ways.

Previously, we have described a system which works with identifiers such as ArXiv IDs (used to reference this article above!), PubMed IDs and DOIs. With this system, called kcite, the author supplies the ID, and kcite generates the reference list, leaving the ID underneath which is easy to extract computationally. The data used to generate the reference comes from specialised bibliographic servers.

In this paper, we describe two new systems. The first, called Greycite, provides similiar bibliographic data for any URL; it is extracted from the URL itself, using a wide variety of markup and some ad-hoc tricks, which the paper describes. As a result it works on many web pages (we predict about 1% of the total web, or a much higher percentage of “interesting” websites). Our second system, kblog-metadata, provides a flexible system for generating this data. Finally, we discuss ways in which the same metadata can be used for digitial preservation, by helping to track articles as and when they move across the web.

This paper was first written for the Sepublica 2013 workshop.

Our original intention with Greycite (http://greycite.knowledgeblog.org) was to build a tool which can provide bibliographic metadata for any URL, to support my own kcite referencing tool (http://knowledgeblog.org/kcite-plugin). While it still fulfils this function, it also turns out to be a useful, general-purpose tool for investigating the metadata in various web pages. And this reveals some interesting results. We discovered a nice example of this recently while adding RIS support.

The paper in question comes from EMBO reports (10.1038/embor.2013.11). At first sight, the RIS for this page taken from Greycite looks reasonable.

TY - ELEC
UR - http://www.nature.com/embor/journal/v14/n3/full/embor201311a.html
Y2 - 2013-04-18 12:54:54
TI - The economics of creative research
JO - EMBO reports
PY - 2012
DA - 2012-02-08
DO - 10.1038/embor.2013.11
AU - Cou|[eacute]|e, Ivan
ER -

However, something strange is going on with the author; poor old Ivan Couée’s name has been rather broken. So, why is this happening? Looking at the underlying HTML the first thing that hits you is a lot of space; there are over 50 empty lines at the beginning of the file; still, this is only a problem for people strange enough to be reading the HTML.

However, eventually we get to the metadata, first dublin core and what we describe as Google Scholar (since this is where we found it). And there we have it; greycite is reporting the metadata as it is. The author’s name is represented with |[eacute]| as a letter.

<meta name="dc.language" content="en" />
<meta name="dc.rights" content="&#169; 2012 Nature Publishing Group" />
<meta name="dc.title" content="The economics of creative research" />
<meta name="dc.creator" content="Ivan Cou|[eacute]|e" />
<meta name="dc.identifier" content="doi:10.1038/embor.2013.11" />
<meta name="dc.date" content="2012-02-08" />

<meta name="citation_publisher" content="Nature Publishing Group" />
<meta name="citation_authors" content="Ivan Cou|[eacute]|e" />
<meta name="citation_title" content="The economics of creative research" />
<meta name="citation_date" content="2012-02-08" />
<meta name="citation_volume" content="14" />
<meta name="citation_issue" content="3" />
<meta name="citation_firstpage" content="222" />
<meta name="citation_doi" content="doi:10.1038/embor.2013.11" />
<meta name="citation_journal_title" content="EMBO reports" />

As far as we can tell this is an error; HTML attributes or extended character sets are entirely valid, but |[eacute]| does not appear to be a valid representation. Interestingly enough, there also appears to be some slightly buggy code in the PRISM metadata, which I am sure should not be this.

<meta name="prism.issn" content="ERROR! NO ISSN" />
<meta name="prism.eIssn" content="ERROR! NO EISSN" />

My guess is that the problem is at the point of website generation rather than deeper in the bowels of the publishing system; grabbing the metadata for this article from CrossRef by content negotiation (http://www.crossref.org/CrossTech/2011/04/content_negotiation_for_crossr.html) shows the correct name.

{"volume":"14",
 "issue":"3",
 "DOI":"10.1038/embor.2013.11",
 "URL":"http://dx.doi.org/10.1038/embor.2013.11","title":"The economics of
        creative research",
 "container-title":"EMBO reports",
 "publisher":"Nature Publishing Group",
 "issued":{"date-parts":[[2012,2,8]]},
 "author":[{"family":"Couée","given":"Ivan"}],
 "editor":[],"page":"222-225",
 "type":"article-journal"}

We emphathise with the publishers here. Getting character sets correct is the bane of everyones life; given the state of computing when multi-lingual character sets appeared, we guess it is not an example of premature optimisation, but an example of an optimisation you wish had never happened. The world would be an easier place if everything that used unicode from the start.

The current metadata for this paper can be seen on greycite or in detail. Hopefully, this will be updated in time!

This post was written by Phillip Lord and Lindsay Marshall

Update

Spelling mistake corrected, bibliography added. Thanks to Christian Perfect for bug report.

Bibliography