The RAE - time to make a stand?

I had to do some admin today for my RAE submission (Research Assessment Exercise - the formal process for judging an individuals and institutions research output in the UK). As well as being tedious, the sheer futility of it depressed me.  In some areas I'm sure the traditional academic journal is useful, but it is almost entirely irrelevant in educational technology (see my previous rants on this: here and here).  I do read academic research articles (if they're available online - I never trawl through print ones), but they represent only one part of what constitutes my scholarly network, and an ever-decreasing one at that. The debate, articles, opinions, experiments, developments and people I want to engage with are found predominantly in internet native fora now, primarily the blogosphere, but also social networking sites, wikis, podcasts, video clips, open educational resources, etc.

I've mentioned before that blogging has seen a decline in my academic publishing output. This is not really due to time pressures, but is more related to motivation. There were three main reasons to publish in academic journals previously: the formal recognition required for promotion, CVs and the RAE; as an outlet for creativity; to gain recognition and reputation amongst your peers.

Now blogging satisfies these last two, and to a much greater extent than the very narrow confines of academic articles. I used to wade through the tedium of the proper format just in order to write the one paragraph that was of interest, because I had no other outlet. Now I do, so why would I put myself through that very poor fit of writing style when I can find a more natural outlet through this blog?

It's not just that the RAE is irrelevant that irritates me (although I would like to know what return on investment it provides given the enormous amount of time that has been spent by my university and others in preparing for it, with committees, reviewers, systems, processes all being put in place and then the distributed time from each individual), but more that it might be actually harmful. There are two arguments you can make against it. The first is that it is measuring the wrong thing, or at least not recognising lots of other activities (and yes they did say they were expanding the criteria this time, but I doubt 'my blog' would carry much weight, they key being that the people behind the process don't understand the digital world). One could therefore lobby for the recognition of blogs as a formally recognised publication. For RAE rating read Technorati rating. The second argument you can make against it is that it is just the wrong thing to do. In classic Heisenberg mode, the very act of measurement changes what they are measuring. You are not recognising good research or scholarly activity but rather good RAE game playing. And life's too short for that.

The academic publishing racket is a good example of this last complaint. By measuring academic publications, the RAE was getting a proxy for research ability and output, or so it would hope. But by the very act of measuring it, the RAE has legitimised the existence of journals in a time when they are finding their editorial and business model increasingly irrelevant. Many journals exist now not because of the dissemination of research, but because of the RAE.

There are some arguments for why you need something like the RAE, such as how do you standardise so that someone can move between jobs and how do you flow research money to universities? I'm less and less convinced by them, for the first one I would argue that in a connected world you should be able to find out much more about an individual than the RAE will tell you. If I ever apply for another job (not very likely anyone would want me) then I would like to just tell them to look at my blog. If they like that, then they'll like me.  The money is the real issue of course, and the finances of higher education have always possessed the easy logic of quantum physics for me, but I'm sure there are other, more efficient, methods than this one.

As a Prof I wonder if I should make a stand on this - I think it's very damaging for younger academics, who feel they have to engage in it to gain promotion. This is at the detriment of doing interesting things. Developing software, blogging, exploring web 2.0 applications (which don't fit the funding model well - they're more suck it and see, rather than write a 50 page proposal, get ten partners, then see in a few years time), innovating in teaching - all these would be better things to do for academics in my area (maybe the same isn't true in Classical studies, say, although the last one should be).

It all feels very last century, or last 5 centuries for that. The lack of recognition for the large socio-technological change we are going through (it's really a paper exercise despite attempts to dress it up - at heart what they want are traditional, printed journals that sit on dusty shelves), doesn't reflect well on academia. My friend is a graphic designer, and when he started in the 1980s he was always coming home with the top of his finger cut off. He explained that this was caused by the scalpel when he was cutting out photographs in order to create montages and images, which would be glued together and then photographed again. His skill at this was well recognised in his company. Within 5 years all those desks, darkrooms, scalpels and glue pots were gone, and the Mac reigned supreme. The RAE feels like we're still being judged on our scalpel skill.

Comments

[Juliette]

Hmmmm, replacing RAE rating by Technorati rating would just result in a different sort of gaming surely? Being able to choose between submitting either your quota of papers or a blog for the RAE would be interesting though. You'd be battling against the fact that in many discipliness journals (or in reality, preprints and seminars based on research papers) are still very much how research is communicated.

The core problem is how you evaluate quality of research which is a very subjective and usually specialist thing. There definitely ought to be some way of doing peer review without all the trappings and protocols of journals though.

[Martin]

Hi Juliette
I wasn't very clear - I wasn't suggesting that we do replace RAE with Technorati, because as you say that would lead to the same problem.
I suggested there were two complaints you might make against the RAE:
i) that it is okay but measures the wrong things.
ii) that it is fundamentally a pointless (and damaging) exercise.
I think I've come round to ii), ie any such process is flawed. Particularly if one takes an 'everything is miscellaneous' view (I know, I've been applying this to everything recently).
What is the point of the RAE? You could argue that it gives us a reliable, standardised measure of someone's research quality. In Wienberger's analysis this is like second order order, ie separated metadata from the actual data (the researcher's output). But in a highly connected world, everything is metadata. I don't need to know how other people rank someone's research, I can find it easily and make my own assessment.

[Juliette White]

I think the problem is maybe actually with the current funding model. Once you have that, you have to have some means of measuring whatever you regard as research quality in order ot make decisions about who gets how much money. If money wasn't involved, everybody could just form their own opinions as you say.

It's the same reason I guess why job ads often ask for qualifications - it's because the person making the decision either doesn't have expertise or time to make their own complete judgment. And if you want to be fair, you've got to treat people who aren't friends with the same folk as you on the same terms as those who are. Most private companies don't admittedly, but as a public sector organisation you have much more responsibility in that direction.

Anything like the RAE where you quantify qualitative information is obviously silly though, but that hasn't stopped people marking students' work for however many years!