Chasing the Hawthorne effect

In a response to a comment from Alan Cann I glibly mentioned the Hawthorne effect and that maybe it wasn't the evil it is usually portrayed as. I've been thinking about this since, and wanted to explore the idea. It goes something like this:

The Hawthorne effect says that because you start experimenting that can change the behaviour of the people you are observing (see also the related Pygmalion and halo effects) . It bedevils educational technology, because say you introduce podcasting on a course and then measure student performance or satisfaction then you can't be sure that any change is really due to a pedagogic benefit of the podcasts, it could just as well be due to the novelty of having podcasts, or of being part of the experiment. There are ways of controlling it, but a lot of educational technology research is quite small scale, done with one teacher and a cohort, and this is particularly prone to the effect.

The teacher effect is probably more significant, in that if you are the teacher who introduces the technology then the very fact that you have done so will modify your behaviour. It may make you more motivated (it's fun again!), it may change what or how you teach, it may make you more reflective. In short it can rejuvenate the way you are teaching a course because you have to think about it in a different way. Now this is likely to have an effect on the students, almost regardless of what the actual technology is.

So generally researchers get sniffy about such studies. But my point is that if it does have this effect, then regardless of the cause - isn't that a good thing? Introducing a new technology (or it could be a new approach, doesn't have to be technology related) every couple of years keeps you on your toes as an educator, makes you think about the design of your course, your approach, the content. It's rather like all those studies that show that introducing new IT systems doesn't actually improve productivity, but you have to do it to keep up. If you hadn't introduced them then the organisation wouldn't  be able to compete. So maybe there isn't this great pedagogical motivation for using the new technologies, but their incidental effects are more significant.

So rather than dismissing the Hawthorne effect maybe we should be actively pursuing it, and asking yourself 'How have I Hawthorned recently?'

Comments

[AJ Cann]

I think you hit the nail on the head Martin. An interest in pedagogic development will have a positive effect on teaching, even if it's not directly through the intended means.
Of course, this problem could be formally tackled by well designed trials with control groups. Unfortunately, it is rare that we have the opportunity to exercise this luxury in most educational institutions. If we could only abandon the "not invented here" syndrome and move towards larger scale national trials, that problem could be overcome, just as it is with drug trials.