Technorati as scholarly metric

The other day my technorati authority finally nudged past 100. This is significant for me because in my annual staff appraisal meeting last year one of the goals I set myself was to achieve this. I did this because I wanted to test the whole staff appraisal system, and as part of my campaign to get blogging recognised as a scholarly activity, in the Open University at least. Having measurable outputs then is a means of bringing blogging in to the fold - like publishing a number of journal articles, or delivering a number of presentations. It's measurable, therefore it's real in the eyes of official processes.

Having achieved my goal (and ahead of schedule too!) I thought I'd reflect on it, in case anyone is attempting something similar. There are a number of questions one should ask about this use of Technorati as one measure of academic performance.

Firstly, is it reliable? This itself breaks down in to two parts: Is Technorati itself working reliably, and then is it a reliable measure of academic output. Taking the Technorati element first, I'd say - sort of. In my experience it sometimes gives odd results - my authority dropped when In posted two new items. Why would it decrease with new posts? It also hasn't picked up some people I know have linked to me, this may be because they are not listed on Technorati themselves. And it also takes ages to update sometimes. For some reason my blog is shown twice, with two different authorities. So, there may be some glitches, but overall it acts as a reasonable proxy to blogging reputation.

The second element was whether it is a good measure of academic output. Clearly Technorati itself doesn't care what I blog about. I could have increased my authority by blogging about Paris Hilton, or by indulging in blog wars and linkbaiting. That doesn't make it a good educational blog, so clearly your Technorati authority needs to be only one component.

After reliability we come on to desirability. Would it be good to have your Technorati authority (or any such measure) used as part of your scholarly output? There are two viewpoints to take on this, firstly from the educational establishment, and secondly from the blogging community. In terms of the establishment I'd argue that although there are issues around reliability, the same is true of traditional measures also. It is relatively easy to get publications out in some journals, to publish lots of papers from the same basic idea/research by only changing a small element between each and to get research students to do all the work. To combat this we end up with the convoluted and tortuous measures of esteem that are found in the RAE.

From the blogger's perspective the reaction might be 'please don't ruin blogging by making it one of your measurable factors.' Having seen what this has done to research publications (ie transformed them from a genuine activity to a game to be played for points), the last thing you'd want as a blogger is to have this area of creativity, engagement and enjoyment trampled over by herds of bean-counters. One can imagine what it would be like - points for rate of blogging, for your ranking, for the number of subscribers, etc. All of which would transform behaviour and ruin it. But, having said that if we bloggers truly think it is the best place for academic discourse, then getting it recognised as such may be important for encouraging new academics who need to keep an eye on their career path.

Overall then? For me it's been worthwhile having a specific target, and having some form of measurement I can feed into the system to have that activity recognised. Perhaps the thing to remember is that it doesn't matter what it means, after all is an authority of 100 good? Should it be 200, 700, or some other number? What does it mean in terms of reach? But there is a measurement in the system, and the system likes measurements, so it can rest easy.

Comments

[AJ Cann]

Interesting as always Martin, but I can see (at least) two problems in this use of Technorati as a metric (not in your case, but in general).

First, this system is not immune to gaming. Set up a load of interinked blogs under assumed names and, hey presto... Or only link to people with an implicit (or explict) understanding that they'll link to you.

Second, I'm increasingly preferring to comment on other blogs rather than to link to them from my own. Needing a high Technorati authority could tend to interfere with the conversation, or at best, make it very stilted and split up across many sites.

I know you wouldn't sink to such depths, but there are plenty of people who might...

[AJ Cann]

And - am I allowed to add mine up? :-)

http://skitch.com/ajcann/fxk9/technorati-blogs-for-ajcann

[Jon]

Interesting post! A fascinating experiment, setting it as an explicit goal, and a great "post-action reflection" ...

> I'm increasingly preferring to comment on other blogs rather than to link to them from my own. Needing a high Technorati authority could tend to interfere with the conversation, or at best, make it very stilted and split up across many sites.

I think this is a really important point. The blogosphere as a whole functions well when everybody's commenting on each other's blogs. As well as the open question of how to browse multi-blog conversations effectively (trackbacks are useful but problematic), your question of how to come up with measurements that incent this kind of behavior is an excellent one.

jon, commenting on somebody else's blog :-)

[Tony Hirst]

Hi Martin-
I'd be a bit wary of this. Part of the Technorati ranking juice is the number of people 'citing' your posts in their blogs (technorati's 'blog reactions' view of your blog).

If you look at your blog reactions list, there are quite a few self-citations (Technorati is getting confused by the path in the urls, i think?)