« For local people - new models for newspapers | Main | Open Conference video interviews »

09/06/2010

Comments

AJ Cann

A few links:

Unpaid peer review is worth £1.9bn
http://www.timeshighereducation.co.uk/story.asp?sectioncode=26&storycode=402189

Cost of peer review exceeds the cost of giving every researcher a grant
http://scienceoftheinvisible.blogspot.com/2009/04/cost-of-peer-review-exceeds-cost-of.html

Peer Review - A Love Poem by Google
http://scienceoftheinvisible.blogspot.com/2010/03/peer-review-love-poem-by-google.html

twitter.com/cdmilligan

Your decision only to review for open access journals is completely consistent with your publishing practice, perfectly justified.

When I peer review, part of the motivation for me is to reinforce my faith in the system - I treat other researchers papers in the same way I'd like my own to be judged.

I was wondering how practical deciding to publish only in open access journals has proved - it is easy when you are the primary author, but have you compromised when someone else leads, or have you been able to argue your case successfully.

(I couldn't find your original post on publishing in open access journals: perhaps you answered this point there).

Erik Duval

I review quite a bit too: an additional benefit for the reviewer is early access to research results. But it often becomes a bit complicated to refer to the paper you reviewed later on, because few journals or conferences do decent review follow-up. The result is that, as a reviewer, you often don't know what other reviewers think about the paper, wether it eventually got accepted and published (possibly with a different title), etc. It seems to me that there is a lot of wasted opportunity there...

BTW, do you only review for journals where also the review process is open? I have rather mixed feelings about that, so am curious to hear about your experiences...

Martin

@AJ - thanks for the links, that 1.9billion figure looks high (as you said on twitter), but may take into account further costs such as the editorial time, changes by the writers in response to review, etc. I also only took the modal time for review which was 4 hours, the median (they didn't give the mean) was 6, which would bump mine up a bit.

@Colin - I couldn't find my post where I said I would only write for open journals either, which means I didn't write one (strange, I was sure I'd blogged it somewhere). To be honest, I don't write many papers now, the blog being the my main outlet so it hasn't proven to be too big an issue. I haven't really encountered probs with co-authors so far - our recent paper on digital scholarship was in an open journal, and everyone seemed happy with that, but then I tend to be working with like-minded people.

@Erik - no, I haven't gone that far. I have mixed feelings too, and while I'm happy to experiment, I don't think I should impose a best practice in that respect yet. But I do feel that open access to the content is a minimum now if people want me to provided free content.

Neelakantan Nair

You write about this from the reviewers perspective. I tend to see it from the student/common man's perspective. I had blogged about this problem a while earlier here:

http://technicallypossible.wordpress.com/2010/06/04/academic-papers-should-be-free/

I'm new to the whole research side of things and just getting a hang of things. In computer science the problem is probably more, I am starting to feel. Because as far as I have seen, conferences are the ones with more reputation than journals. But most papers which are available for free in some form if you search hard enough through university databases mostly :)

leighblackall

To extend the illustration, Reed Elseiver reported a pre tax profit in the Academic Journal devision of ~ £247m in 2008. (see comments to this post for the link)

leighblackall

Sorry, hypertext html didn't go through. Here it is again: http://leighblackall.blogspot.com/2009/12/academic-publishing-scam.html

Bob L. Sturm

This is a thought provoking post, but I think there is more to the "return on investment" than just quality control for closed-access journals. How about the way the peer review process (usually) provides thorough and free feedback to researchers? I have rarely submitted a review and recommendation for rejection that didn't result in a stronger and acceptable resubmission. And in my own submissions I have rarely received any feedback that was not helpful in making my research and its presentation more thorough, convincing, and broad. Ironically, the worst review I have received is an accept with nothing more stated; the best I have received is a reject, with an insightful explanation of why my approach is flawed (kind of like getting to peek in the back of the book at the answers to the odd numbered questions). Also, as a peer reviewer, I usually get to see how the other reviewers see the problems, and what they found that I missed, or vice versa. I feel that this is how I am paid back by performing peer review, whether or not the publishing company provides open-access or not.

Martin

@Neelakantan - yes, exactly my point, I wasn't really focusing on the reviewers, ultimately it is the users I am concerned with - why do all this free activity and then let it be locked away from people who can afford access?

@Leigh - that's some good invective there - you are right to angry. I hadn't really appreciated the point you make about those outside the US/UK are also subsidising these companies which are largely based in those two countries.

@Bob - that's a different point. I agree about the value of peer-review to the individual and the community (although I think there is an interesting discussion to be had about whether this is the only way to achieve these results now). The publishers don't do anything to facilitate this process (that is done by academics, for free). I would take your conclusion and look at it the other way - would any of the benefits you name not exist for an Open Access journal? In which case, why do the work that allows a large multi-national to profit and lock away content, when you could get the same benefits and have the content open to all?

pluton

what about no more old-fashion peer review at all? Scientists would still write papers on their preferred subjects, then upload them on an open archive (arxiv, HAL...) with access to the useful piece of code (open languages preferred, like Python). Readers would be able to reproduce the results (I have to admit that this is not possible in many experimental fields), compare with their own methodology and comment the paper on a open forum on the side. That would be continuous social peer-reviewing instead of the current process that is not really capable of filtering mistakes. That would also filter out papers for which the strategy seems interesting but cannot be reproduced.

Martin

@Pluton - yes, I've advocated post-review or simply just let the good stuff rise up, approach. That is a slightly different argument - I think the whole peer-review process is up for debate. But deeply embedded cultural practices change slowly (and we may decide that peer-review really is the best approach for research), and a first step seems to be at least to ensure that all publications are openly available.

Austin Bond

Thanks for the very interesting post Martin. I wonder though whether your estimates are high enough? I was advised recently by an academic that only about one in every ten articles she reviewed was approved for publication (whereas your figures are based on the number of published journal articles only). I support the idea of post-review ... I wonder how many good ideas have been lost by overly stringent reviewers? I remember a post I read a while ago suggesting that many of our notable educational theorists may not have passed peer review if they were submitting their articles today.

Just another Internet Poster

if we all did...

1. secrets for anything profitable might be witheld.

2. petitions for subsidies would favor the 1.9 side rather than the .2 side for taxation run by a comittee which would likely [historically] require administrative fees t oproduce glossy pamphlets justifying their existence, purpose, and PSA's for the potato farmer who pays their rent.

3. the quality of open-source scientific, medical, information technology, and agricultural information might improve.

4. people who are not qualified might become compensated for reviewing information that is over their heads.

R3beccaF

Good Xtranormal take on this issue
http://www.youtube.com/watch?v=GMIY_4t-DR0
'Your manuscript has been accepted by the journal I own. Just sign here...'

The comments to this entry are closed.

Flickr

  • www.flickr.com
    This is a Flickr badge showing public photos and videos from edtechie99. Make your own badge here.

Twitter Updates

    follow me on Twitter