Blogs easier to read than formal publications

Stunning new research (okay, ten minutes by me this morning) has shown that blogs are much easier to read than formal academic publications, even when they are covering the same topics.

Yesterday Alan Cann put a few bloggers through FLESH, a piece of software that takes text and scores it according to the Flesch Reading Ease Score and Flesch-Kincaid Grade Level. For the reading ease score, the higher the score then the easier the text is to read. For the grade level it indicates how many years in education a person needs to understand the text. There are some concerns about this method, and the grade level in particular seems dubious to me, and then we have to accept that the software is interpreting it correctly, so lots of room for doubt, but putting that aside for now, I thought I would compare the blog scores Alan obtained with samples of published papers from three of the writers (me, Alan and Grainne).

The papers I used were: Alan's Assessed Online Discussion Groups in Biology Education, Grainne's Using Compendium as a tool to support the design of learning activities (forthcoming) and my Learning Objects, Learning Design, and Adoption through Succession.

Here are the scores for reading ease:

Author	Blog Reading Ease	Paper reading ease
Conole	50.79	32.48
Weller	63.85	34.82
Cann	59.66	36.8

And for reading level:

Author	Blog Reading Ease	Paper reading ease
Conole	12.19	14.2
Weller	9.47	14.04
Cann	9.02	13.55

I'll come on to the reservations later, but at face value this seems to be saying that there is a marked increase in reading difficulty when the same authors write in the formal publication style. Variation for author is controlled, and so to an extent is variation in subject matter, since the papers are all subjects we blog about (this could be controlled more rigorously though, I admit). Which means the variation is down to the medium, or style imposed by that medium. If general understanding is the goal, then this mini-research suggests blogs are better at achieving it.

Now the reservations:

• Obviously a very limited set, would need to be expanded to many different authors and subject areas.
• Reliability of Flesch - are there some peculiarities in the system that mean academic papers always come out as harder to read e.g. use of references.
• Reliability of the software - how reliably is the software interpreting the Flesch method?
• Interpretation of results - what do these really mean?

If someone was looking for a Masters project  that looks like a neat investigation to me (no doubt someone has already done it, so if you know, let me know).

Comments

[Grainne Conole]

OOOhh yes this is fun :-) But some serious points underneath this too i think. Of course it would be interesting as you say to expand this across people, disciplines, content etc. I guess the issue is can we disentangle good/bad english from a particular style/genre used in a particular discipline??

[James Aczel]

Is there any research comparing The Sun with Channel Four News in terms of "general understanding"?

[Non Scantlebury]

Interesting post for me as a nascent anarchist librarian. It made me reflect on the role of blogs versus academic papers in terms of disseminating knowledge and contextualising its provenance.
OK there may be some indication from this proof of concept that blog writing styles could potentially be easier to 'read' but if we do away with referencing what others have found because including these makes it 'harder' to read, then aren't we in danger of dumbing down big time here?
But maybe that's not what you meant by your comment?

[Martin]

Non, in this case I didn't mean do away with referencing - I just meant that a reference section might 'trick' the FLESH software because according to its rules they wouldn't be very well formed sentences. I don't know if this was true, I was just suggesting as something that would need further investigating to make a proper comparison. Having said that I have suggested that the very formal style of referencing we insist on teaching students is anachronistic - see http://nogoodreason.typepad.co.uk/no_good_reason/2007/11/is-academic-ref.html - that's not dumbing down, just saying we shouldn't be so strict.