It has become accepted practice amongst those who know about MOOCs to sniff at completion rates. Focusing on them (hell, even mentioning them) demonstrates just how constrained you are by the old ways of thinking daddio. I find this particularly from the cMOOC crowd, and I've stopped talking about them, because as David Kernohan suggests, to even talk about them is like saying you hate learning.
The commonly used argument against completion rates (or even worse 'drop-out rates'), is that they aren't relevant. Stephen Downes has a nice analogy, (which he blogged at my request, thankyou Stephen) in that it's like a newspaper, no-one drops out of a newspaper, they just take what they want. This has become repeated rather like a statement of fact now. I think Stephen's analogy is very powerful, but it is really a statement of intent. If you design MOOCs in a certain way, then the MOOC experience could be like reading a newspaper. The problem is 95% of MOOCs aren't designed that way. And even for the ones that are, completion rates are still an issue.
Here's why they're an issue. MOOCs are nearly always designed on a week by week basis (which would be like designing a newspaper where you had to read a certain section by a certain time). I've blogged about this before, but from Katy Jordan's data we reckon 45% of those who sign up, never turn up or do anything. It's hard to argue that they've had a meaningful learning experience in any way. If we register those who have done anything at all, eg just opened a page, then by the end of week 2 we're down to about 35% of initial registrations. And by week 3 or 4 it's plateauing near 10%. The data suggests that people are definitely not treating it like a newspaper. In Japan some research was done on what sections of newspapers people read. There is an interesting gender split but also the sections are quite evenly divided:
Men top 5 sections:
- Headlines (62.0%)
- Domestic News (55.4%)
- Sports (55.4%)
- Economy (53.3%)
- International News (47.8%)
Women top 5 sections:
- TV listings (71.4%)
- Headlines (65.3%)
- Domestic News (53.3%)
- International News (50.8%)
- Crimes and Accidents (39.2%)
For MOOCs to be like newspapers then you'd expect 65% to read the topics in week 1 and, say 54% the topics in week 7. This doesn't happen. Now, it could happen, if MOOCs were designed that way, and you thought that was appropriate for your subject matter. But to say it does happen is simply incorrect.
Now for any individual this may not matter, you've dropped out when you felt like it, and maybe that was a meaningful experience (or maybe it was a painful experience because you felt out of your depth, but we don't like to talk about that either). But for MOOCs in general as a learning approach it really does matter. Most MOOCs are about 6-7 weeks long, so 90% of your registered learners are never even looking at 50% of your content. That must raise the question of why are you including it in the first place? If a subject requires a longer take at it, beyond 3 weeks say, then MOOCs really may not be a very good approach to it. There is a hard, economic perspective here, it costs money to make and run MOOCs, and people will have to ask if the small completion rates are the most effective way to get people to learn that subject. You might be better off creating more stand alone OERs, or putting money into better supported outreach programmes where you can really help people stay with the course. Or maybe you will actually design your MOOC to be like a newspaper.
Kernohan raises the point that it is in the commercial interest of MOOC companies to dismiss drop out rates. A good question to ask yourself when someone says completion rates don't matter is "if they had 90% completion rates, would they still be telling me they don't matter?".