Better angels of our edtech nature

Over the summer I finally got around to reading Steven Pinker's The Better Angels of Our Nature. In case you don't know about it, Pinker makes the argument that violence of all forms has declined - between states, domestic, national and criminal. It's a lengthy book, but he goes through the arguments very carefully and brings a range of research together to make a compelling case.

What he does very well is take accepted arguments or facts and challenge them - for instance that the 20th century was the most violent one in history. He demonstrates how these are often based on a combination of a rose-tinted view about the past (we were all happy farmers) and a misreading of data.

I was often struck when reading it how lazy people are - and not just journalists or lay perceptions, but academics also. Certain beliefs become an accepted starting point, and when you really examine these you often come to some surprising, counter-intuitive findings.

In education and education technology I think this is also true. Many claims appeal to our "common sense" or widely held beliefs about how society is today, without really backing this up. For example, the raft of 'technology is destroying society' type books or beliefs often only draw on indirect data at best, but usually rely on anecdote. Take Sherry Turkle's argument that we're losing the art of conversation and becoming more isolated. What is this really based on? There is evidence that families are spending more time together, yet you would be hard pushed to find anyone who had this as a starting point in an argument.

Or take the argument that education is broken (which I've moaned about before). This often starts from the 'fact' that truancy is at an all time high, therefore schooling isn't working, therefore we need to do something radical. I haven't analysed truancy rates in sufficient detail (in fact there doesn't seem to be much meta-analysis of truancy rates), but I know the manner in which they are recorded varies considerably. One must also ask the following questions:

• Is any change now a statistical one, or within the realms of normal variation?
• Are historical comparisons valid (ie are they comparing the same measures)?
• Can an increase in truancy rates be accounted for by an increase in population or targeted school attendance (eg if you are working harder to make sure certain groups are registered in school in the first place, will you get more truancy)?
• Is it an increase in more pupils being truant, or the same number of truancy pupils being truant for longer? (eg this article suggests7% of pupils account for one third of all truancy numbers).

And so on - I'm not suggesting truancy isn't an issue, but merely making the point that we adopt lazy interpretations in education, and then make quite sweeping demands for change based on them.

And, naturally, I can't resist one more digital natives swipe on all this. The whole digital natives argument was dogged by this failure to make any real historical comparison and instead rely on anecdote and over interpretation.

Because ed tech is often dealing with new things, it is easy to do this. And I definitely don't advocate doing nothing until you have all the data 20 years from now when you can declare 'yes, the internet will be significant.' It's perfectly okay to have hunches, intuition, to draw from experience and to have a working hypothesis. But equally, because this stuff is all new, we should be forceful in making sure claims are really founded in evidence.

So here's an activity you can do with students, or just for the fun of it: Take a claim or widely held belief in educational technology and examine the precepts that underlie it. How solid are these? Are any of them subject to the kind of Pinker analysis and review?

For now, I would just heed caution to any argument that is based on something that is 'obvious to everyone'.

 

Comments

[twitter.com/cogdog]

A really good read in response to Turklism is "The IRL Fetish" and the notion of digital dualism, all too often we do in education (and elsewhere) to boil complex spaces to binary choices on the extremes
http://thenewinquiry.com/essays/the-irl-fetish/

I will say it is obvious to everyone that you are an awesome blogger ;-)

[Lawrie]

Nice piece Martin, I think we know that headlines are what gets attention, the drive by ed tech shooting type session at a conference is more likely to get a high attendance than a well researched presentation. "the VLE is dead" will get more attendees than "a statistical analysis of LMS use in 5 universities". The problem comes when policy influencers / makers start listening to headlines rather than content.

Interestingly I'm at eduwiki conference 2012, and the work presented is well referenced and evidence based. At odds with most headlines about Wikipedia in education.

[mweller]

Thanks Alan, hadn't seen that piece. The point you make about extremes is one I was going to make also (but, you know, couldn't be arsed). One interpretation of what I suggest here is that all such claims are rubbish, but it's usually more nuanced than that. For example, one such claim might be that 'games are a more engaging way to teach young people'. We could then find problems with this, but that wouldn't mean games are not an effective way to teach some people some things.
So, yes, it ain't all or nothing.
PS - everybody knows that cogdog blogs at a higher quality rate than previous generations

[Holden]

One of my favorite books, and one of my favorite subjects. I got so interested in this subject I actually wrote a free textbook about it. The model we use in my statistical literacy class is called COMPARABLE and the basic outline is here: http://www.makingfaircomparisons.us/introducing-the-fair-comparison-checklist-comparable/

Under that framework:

* Your question one is an "R" question (could this be nothingn but randomness at play?)

* Your Question 2 is an O/L question (How were the variables defined/operationalized? and, Could a longitudinal analysis tell us more?)

* Your question three is an "A1" question (What was accounted for/controlled for?)

* Your question four is another "O" question (or maybe an "A2" -- What alternative measures can we think of that represent the same issue? Do they tell the same story as the measures chosen, or a different one?)

Check out the link, I think you will like it.

In any case, there are multiple reasons I got into teaching this, but one is the non-critical way that people deal with edtech. I'm a proponent of analytics, for instance, but I see time and time again that people put up a slide on Purdue's Course Signals project, and uncritically state a headline increase in retention. But a short dive into Purdue's own data shows that the bulk of retention increase had nothing to do with Course Signals at all...it was an across the board increase...

"Digital Natives" are also a great example. Do they use technology in a slightly more social pattern than adults because they are a different generation? Or could it be that all generations, when younger, use technology more socially (thinking of my youth and the time my sister spent on the telephone). Could it be that they use facebook more than twitter, not out of a generational preference, but because facebook serves the needs of a young person, and twitter serves the needs of a professional more? Etc. etc.