« The REF - a user's guide | Main | Remote conference participation - flash debate »




testing comments again

Will Reader

Pretty much agree with all of that, Martin. I was completely unsurprised (as were you, I guess) that the REF was so conservative. I do see, however, why they base their evaluations so heavily on journal articles as the peer review process provides appropriate checks and balances for academic rigour, novelty and so on. This much I'm sure is obvious.

My feeling is that the whole business of academic 'publishing' has to be revised. The standard peer-review process is glacially slow and cannot keep up with the pace of understanding new media. It is likely to take at least 2-3 years, often longer between deciding on a research question and the final paper being published. So we should expect to find papers on Twitter published in 2011 when we will have all moved on to some other medium. If wiki-based peer-review works for high-energy physicists why can't it work for other disciplines?

Maybe it is because high-energy physics is more of a community. The goal of the community is to get good ideas out fast to progress the field. I get the feeling that other disciplines such as psychology (the discipline with which I'm most familiar but the same probably applies to other social sciences) have a goals that are more focused on individual research groups. Whether this is due to psychology's lack of a weltanschauung (there, worked that one in) or due to other reasons I'm not sure. But I do know that it we want change it is us that has to do it.


Hi Will, yes, I have a 'reviewing peer review' post brewing. The delay is not the only problem - there is the issue of the peer review process becoming something of a game also I feel - we know what we are supposed to say as reviewers and as authors we make some perfunctory changes and then it goes through. Having open comments (as in pre-publication on blogs) is a lot better I think because people can see the objections and your responses. The peer-review process also limits the type of papers we get - the process seems designed to strip out anything resembling interest in a paper. While this is suitable for some work (eg medical research), in other places it's ok to have an opinion.
And while it is up to us to change it, if the REF is the means by which you get funding and promotion then it actively discourages this kind of experimentation and change.
I'm kinda ranty about it aren't I?


Hi Martin,

Thanks for your thorough - if depressing - post.

My question is, what can we *do* about it?

Lobbying via social media is preaching to the converted, but it might give an indication of the strength of feeling and the sheer number of people who see the need for change.

Are there more modern and appropriate systems operating elsewhere in the world that can be used to showcase best practice, or perhaps we can link the debate more pro-actively to related high profile issues such as tuition fees/quality of university services?


Hi Lisa
well commenting on WriteToreply is a start I guess. Also I think we can start making our case within institutions. If we have highly regarded people (eg Michael Wesch) who we can cite as examples of people who would be completely overlooked by REF metrics it begins to show the flaws. Also if we can work at developing our own models (open publishing, metrics) then we have some models to offer as alternatives.

The comments to this entry are closed.


  • www.flickr.com
    This is a Flickr badge showing public photos and videos from edtechie99. Make your own badge here.

Twitter Updates

    follow me on Twitter