The Ed Techie

MOOC completion rates DO matter

It has become accepted practice amongst those who know about MOOCs to sniff at completion rates. Focusing on them (hell, even mentioning them) demonstrates just how constrained you are by the old ways of thinking daddio. I find this particularly from the cMOOC crowd, and I've stopped talking about them, because as David Kernohan suggests, to even talk about them is like saying you hate learning.

The commonly used argument against completion rates (or even worse 'drop-out rates'), is that they aren't relevant. Stephen Downes has a nice analogy, (which he blogged at my request, thankyou Stephen) in that it's like a newspaper, no-one drops out of a newspaper, they just take what they want. This has become repeated rather like a statement of fact now. I think Stephen's analogy is very powerful, but it is really a statement of intent. If you design MOOCs in a certain way, then the MOOC experience could be like reading a newspaper. The problem is 95% of MOOCs aren't designed that way. And even for the ones that are, completion rates are still an issue.

Here's why they're an issue. MOOCs are nearly always designed on a week by week basis (which would be like designing a newspaper where you had to read a certain section by a certain time). I've blogged about this before, but from Katy Jordan's data we reckon 45% of those who sign up, never turn up or do anything. It's hard to argue that they've had a meaningful learning experience in any way. If we register those who have done anything at all, eg just opened a page, then by the end of week 2 we're down to about 35% of initial registrations. And by week 3 or 4 it's plateauing near 10%. The data suggests that people are definitely not treating it like a newspaper. In Japan some research was done on what sections of newspapers people read. There is an interesting gender split but also the sections are quite evenly divided:

Men top 5 sections:

• Headlines (62.0%)
• Domestic News (55.4%)
• Sports (55.4%)
• Economy (53.3%)
• International News (47.8%)

Women top 5 sections:

• TV listings (71.4%)
• Headlines (65.3%)
• Domestic News (53.3%)
• International News (50.8%)
• Crimes and Accidents (39.2%)

For MOOCs to be like newspapers then you'd expect 65% to read the topics in week 1 and, say 54% the topics in week 7. This doesn't happen. Now, it could happen, if MOOCs were designed that way, and you thought that was appropriate for your subject matter. But to say it does happen is simply incorrect.

Now for any individual this may not matter, you've dropped out when you felt like it, and maybe that was a meaningful experience (or maybe it was a painful experience because you felt out of your depth, but we don't like to talk about that either). But for MOOCs in general as a learning approach it really does matter. Most MOOCs are about 6-7 weeks long, so 90% of your registered learners are never even looking at 50% of your content. That must raise the question of why are you including it in the first place? If a subject requires a longer take at it, beyond 3 weeks say, then MOOCs really may not be a very good approach to it. There is a hard, economic perspective here, it costs money to make and run MOOCs, and people will have to ask if the small completion rates are the most effective way to get people to learn that subject. You might be better off creating more stand alone OERs, or putting money into better supported outreach programmes where you can really help people stay with the course. Or maybe you will actually design your MOOC to be like a newspaper.

Kernohan raises the point that it is in the commercial interest of MOOC companies to dismiss drop out rates. A good question to ask yourself when someone says completion rates don't matter is "if they had 90% completion rates, would they still be telling me they don't matter?".

Battle for Open webinar

As part of Open Education week, the OER Research Hub organised some webinars. One was around my Battle for Open idea/forthcoming book. It was my first attempt to condense the book into a presentation. The areas I covered were: the roots of open education; Open access publishing; OERs; MOOCs; Open scholarship; The Silicon Valley narrative; some warnings, and conclusions.

For the 4 areas of openness (OERs, MOOCs, OA and open scholarship) I tried to set out the success of the open approach and also the key areas of battle. 

You can watch/listen to the webinar here. The slidedeck is below:

The Battle for Open from Martin Weller

The dangerous appeal of the Silicon Valley narrative

For my book I've been writing about why it was that MOOCs came to such prominence in the popular press in a way that OERs didn't. One key aspect is that they fit the Silicon Valley narrative.

The model of Silicon Valley provides such a powerful narrative that it has come to dominate thinking far beyond that of computing. For instance Staton declares that the degree is doomed because Silicon Valley avoids hiring people with computer science degrees, and prefers those with good community presence on software developer sites. From this he concludes this model is applicable across all domains and vocations. It hardly needs adding that Staton is the CEO of an educational company.

There are several necessary elements to the silicon valley narrative: firstly a technological fix is both possible and in existence; secondly that external forces will change, or disrupt, an existing sector; thirdly that wholesale revolution is required; lastly the solution is provided by commerce.

The education is broken meme satisfies the third condition of the silicon valley narrative. If it is accepted as broken, then only a revolution is sufficient to resolve it. MOOCs appeal to the first and second of these conditions. They are a very technologically driven solution, particularly in their xMOOC instantiation. Thrun famously worked at Google after all. The artificial intelligence promise of adaptive learning systems and sophisticated automatic assessment is both appealing in that it seems futuristic and aligns with the silicon valley technological solution approach.

Although Thrun, Koller and Ng all worked at Stanford, and so could thus be seen as part of the establishment, Thrun in particular has been cast as the education outsider. In order to satisfy this need for an external party coming to the aid of the sector, the Sal Khan has often been proposed as the godfather of MOOCs.

Another important aspect that appeals to silicon valley, entrepreneurs and journalists alike is that of disruption. It is a term that has been applied much more broadly than its original concept, to the point where it almost meaningless, and rarely critically evaluated. Dvorak complains that it is essentially meaningless, stating that “There is no such thing as a disruptive technology. There are inventions and new ideas, many of which fail while others succeed. That's it.” There remains however a disruption obsession inherent in the silicon valley narrative. As Watters argues, disruption has become somewhat akin to a cultural myth amongst silicon valley: “when I say then, that “disruptive innovation” is one of the great myths of the contemporary business world, particularly of the tech industry, I don’t mean by “myth” that Clayton Christensen’s explanation of changes to markets and business models and technologies is a falsehood… my assigning “myth” to “disruptive innovation” is meant to highlight the ways in which this narrative has been widely accepted as unassailably true.”

Nobody wants to just create a useful tool, it has to disrupt an industry. Education, perceived as slow, resistant to change and old-fashioned is seen as ripe for disruption. Christensen, Horn and Johnson themselves have deemed it so, stating that “disruption is a necessary and overdue chapter in our public schools.” Hence the Avalanche report justifies itself by claiming that all of the key “elements of the traditional university are threatened by the coming avalanche. In Clayton Christensen’s terms, universities are ripe for disruption.” In his criticism of the impact of OERs, Kortemeyer states that they “OERs have not noticeably disrupted the traditional business model of higher education”, because for something to be successful, only disruption counts.

We can see many of these elements in essays on MOOCs. Let us take Clay Shirky’s essay “Your Massively Open Offline College Is Broken”, as it generated a lot of interest and was considered to be a thoughtful analysis.

In terms of our narrative essentials, Shirky even has the “education is broken” meme in the title of his piece, and later states it boldly: “I have a different answer: School is broken and everyone knows it.” He sets out a reasonably convincing case about the finance issues associated with higher education, although he does not question finance models for higher education in general. Shirky cites a book "Don't go back to school" which interviewed 100 people who had dropped out of school and gone on to be successful. Largely they then self-teach themselves using internet resources, an example of the Silicon Valley model being applied broadly.

In his previous essay, Napster, Udacity and the Academy he compares the impact of MOOCs on higher education with that of the MP3 on the music industry. This conforms to the silicon valley narrative, proposing a revolution and disruption: “Higher education is now being disrupted; our MP3 is the massive open online course (or MOOC)”. It also suggests that the commercial, external provider will be the force of change, stating that “and our Napster is Udacity, the education startup”.

All of the elements can also be seen in Clark’s piece where he declares that (referring to Khan) "It took a hedge fund manager to shake up education because he didn’t have any HE baggage." It appeals to the silicon valley narrative to have a saviour riding in from outside HE to save it. If the influence of those inside higher education such as Wiley, Downes, Siemens, etc is acknowledged then that weakens the appeal of the story.

David Kernohan performed a semantic analysis of eleven popular MOOC articles. Taking Kernohan’s articles to conduct simple word counts, then the word “disrupt” (or derivative) occurred 12 times, “revolution” 16, and “company” 17. Obviously this is a selective choice of terms (“open” appears 48 times for comparison), but the presence of these terms indicates a particular framing of the MOOC story that allies with the silicon valley narrative.

We can now see why MOOCs proved so popular with journalists. Firstly they seem to offer a solution to the education is broken meme, which had been gaining currency. Secondly, they met all the criteria for the silicon valley narrative: they proposed a technological solution, they could be framed as the result of external forces, and they provided a revolutionary model. Nearly all the early MOOC articles framed them as disruptive to the standard higher education model. And they were established as separate companies outside of higher education, thus providing interest around business models and potential profits by disrupting the sector. This heady mix proved too irresistible for many technology or education journalists.

This analysis also reveals why other open education initiatives haven’t garnered as much attention. They often seek to supplement or complement education, thus ruining the education is broken argument. Similarly, they are often conducted by those who work in higher education, which undermines the narrative of external agents promoting change on a sector that is out of touch. And lastly, they are supported by not-for-profit institutions, which does not fit the model of new, disruptive businesses emerging. If one wanted to make an argument for disruption, then open textbooks could make a convincing case, since they undermine an established business with digital, low-cost alternatives, but as projects like OpenStax are not-for-profit, they do not fit the silicon valley narrative as neatly as MOOCs.

One further aspect of the silicon valley and disruption narrative is that it demands a ‘year zero’ mentality. It is a much more convincing story if someone can be said to have invented a new way of working. Because complete genesis invention is rare, most work is tinkering with old ideas and improving them, this often requires either a wilful ignorance of past work, and an imaginative reworking of it. 

Redefining MOOC completion rates

A few posts back I posted Katy Jordan's data on completion rates for MOOCs. It's set me thinking that we're probably being harsh in terms of how we define enrollment on MOOCs, which in turn makes completion rates look worse than they actually are. In formal education there are different ways of defining who has enrolled on a course. I'm pretty sure there is a cooling off period, so if a student drops out within the first week or so, they don't count as having enrolled in the first place (can anyone confirm this from their uni?).

So, taking MOOC enrollment figures to be the number who signed up for a MOOC even if they never come in to it is always going to give harsh figures. It's a bit like saying university courses enrollments are all those who come to an open day. Maybe a better figure would be the number of students active after 1 week. This is your baseline figure as those are the students who have actually started the course.

Another graph Katy plotted was the average number of students enrolled across weeks:

I stupidly didn't get the data from Katy before she went on maternity leave, so actual analysis will have to wait until she's back. But just using this graph as a rough guide we can do some calculations. At the end of week 1 it looks like to me that there are about 55% of students still active from the week 0 point. So, if we took this 55% as our starting figure, we know that the average completion rate was 12.6%. This would now be 22.9% (I have probably done something very wrong with the maths here, but bear with it for rough estimate). Now, approximately 23% completion with open learners isn't too bad.

When I ran T171 we had around 12,000 students and our completion rates were around 40-50% with open access learners. And that was with expensive and intensive tutorial support, so I know how hard it is to get open entry level 1 learners to complete (mind you it was a 28 week course, if I'd stopped it at 7 weeks the completion rate would have been nearer 70-80%).

But MOOCs can't have it both ways, they've been boasting about those massive enrollment figures, which take into account everyone who raised an eyebrow at their course. We know from Katy's data that the median enrollment (from our data) was 18941. So if we take my rough 55% number, this comes down to 10417. Which is certainly not 100K, but it's not too bad either.

So, as always, when you dig into it, the picture is more subtle:: MOOCs don't enrol as many as first thought but their completion rate isn't as bad either. That's not a very snappy headline though.

The year of no s**t Sherlock

With apologies for the potty mouth title. If you want a really good review of the year from an ed tech perspective then I suggest reading Audrey Watters' series of posts. One thing that I found myself doing repeatedly this year was staring open-mouthed at my screen as I read yet another 'discovery' or 'innovation' from US based silicon valley start-ups relating to education. It seems that if you want to be noticed then having no shame in pretending you have invented something is a real advantage. And journalists really do love a 'new and shiny' story. Here are some of my faves:

The SPOC - don't want your MOOC to be massive and open, then try a Small Private one. Or as we've called them for the past 15 years "an online course".

Hybrid Pedagogy - Clayton Christensen discovered “a fundamentally new concept [in] the world of disruptive innovation.” Or as we call it "blended learning"

MOOC based learning and innovation on campus - Coursera announced that it was working with universities to deliver MOOCs on campus. Or as we call it "elearning".

Micro-MOOCs - a micro-MOOC is either a platform to host video and content (or as we call it "the web") or "short e-courses" (or as we call them "OERs")

Creating quality courses is hard and expensive - who knew?

Retention with open access is difficult - again, why did no-one tell them this?

Massive open online content - again, as we call them "OERs"

I hope next year isn't as dumb.

Design responses to MOOC completion rates

Well, my previous post on data for MOOC completion rates caused a bit of a kerfuffle on Twitter. It was interpreted by some as saying "ONLY completion rates matter". And also of not taking into account other factors such as what learners who don't complete get from a MOOC. That seems rather like criticising Alien for not being a rom-com to my mind - they're doing different things. This research was showing one aspect with the quantitative data available. It is part of a bigger picture which ethnographic studies, surveys and more data analysis will complete. It wasn't attempting to be the full stop on MOOC research.

Anyway, here is another graph that Katy created, showing attrition rates of active users (those that come into the course and do something, not just those who complete assessments) across disciplines:

That's a pretty consistent pattern. If we saw it nature we'd give it some name like "The MOOC attrition law". My interest is as a course designer, so given that the drop-off pattern seems fairly robust, what does it mean for design? (Doug will have issues about the power-lawness of this) 

I think there are two responses (but maybe you can think of more).

Design for retention

The first is to say that completion is a desired metric. There may be courses where you really do want as many people as possible to complete. Imagine you were running a remedial maths course for instance, then it won't help your learners much if they only cover a third of the subject matter they need for whatever purpose (Bridge2 Success got learners through maths so they could get onto an employment program, so completion was very important here).

In this case you need to address the 'problem' of drop-out, because it is a problem for you. There might be a number of ways you do this: by adding in more feedback, using badges to motivate people, creating support structures, supplementing with face to face study groups, breaking your longer course into shorter ones, etc. The point is that you design in features that aim to improve completion.

Design for selection

The second design approach is to say that completion isn't an important metric. Here you accept the MOOC attrition law and design the experience with that in mind. I have some sympathy with Stephen Downes when he says no-one finishes a newspaper but we don't talk about people 'dropping out' of a newspaper (I've heard him say this but can't find a link - anyone?). So even to talk about drop-out is to map the wrong metaphor to MOOCs. His analogy breaks down a bit however because not all readers drop out at page 7 of a newspaper, they dip into different sections. People tend to drop out of MOOCs by week 3. It's not as if they're coming in and doing a bit from week 7 and a bit from week 5, and then leaving. They're simply not getting to those later weeks. And even if you are of the 'completion doesn't matter' camp, I'm sure most course designers don't think the content in week 5 is half the value of that in week 1. 

So, in this design approach you might break away from the linear course model, to allow people to do the 'newspaper' type selection. A course might be structured around themes for instance, and each one around largely independent activities (I tried to design H817open a bit like this). So in this case completion really doesn't matter, learners take the bits they want. 

In both cases I would suggest that the completion rate data is useful for you. In the first case you know what type of completion rate to expect, and in the second one it drives you to be more innovative in design approach. And that's the point about the research - it helps inform decisions.

By the way - this is my fifth blog post in 5 days. Just in case Jim Groom berates me for not blogging often enough...

Completion data for MOOCs

As I mentioned in the previous post, I am doing some Gates funded research on MOOCs. My part was learning design analysis, while Katy Jordan has been looking at factors influencing completion rates. All this work is Katy's, I take no credit for it. She would blog it, but is about to have her first baby any day now, and strangely that has taken priority over blogging about MOOCs, so she said I could blog it on her behalf.

There will be a paper that details the full results and methodology, so I'm just giving some highlights here. Katy collected completion data from 221 different MOOCs. The range was limited to ones where we could get this data, so is biased by a couple of big data sets from Coursera and Open2Study. This last one in particular has good completion rates so may have biased the overall results somewhat. 

If we start with enrolment figures first then, the picture looks like this:

(N = 220; Median = 18941; Minimum = 95; Maximum = 226,652). Although the really big numbers get the headlines, there are a lot of courses in the 0-10000 range.

Completion rates across this sample are:

(N =  129; Median = 12.6%; Minimum = 0.7%; Maximum = 52.1%). This bimodal distribution may be a result of those Open2Study courses. Those guys appear to be doing something right in terms of completion.

Does enrollment numbers relate to completers? The answer is yes, there is a significant negative correlation, ie the more people who enrol then the lower the percentage who complete. This single graph might well put paid to the idea that we can educate the world with massive courses:

Yeah, I hear you say, but it's not completion that's important but how many people are active. I'm afraid that shows exactly the same picture, ie there is a significant negative correlation between number of active users and the percentage who complete:

Does the reputation of the university matter? The answer is: possibly. There is a positive correlation between the Times Higher Ed score of a university and enrolments:

But this could just be because more elite universities have joined the elite clubs of MOOC providers. As with all correlation we can't imply causality, it may not be the university reputation that leads directly to the higher enrolments but a related factor. Anyway, it turns out that reputation has no effect on completion rates:

Have enrolments increased or decreased over time I hear you ask. Well, there is a negative correlation over time with enrolments, so numbers on individual MOOCs have declined. This may be that with more MOOCs available people can find the one that suits them better, instead of everyone doing AI, but if you wanted to be all sensationalist (I'm looking at you, journalists) you could say it indicates that the MOOC bubble has burst:

Have completion rates changed over time? Yes, in the opposite direction, there is a positive correlation with completion and date. This could mean people are getting better at MOOCs, and if the trend continued those completion rates would be less of an issue. But I suspect it is related to the point above, in that there are more MOOCs around so people are taking ones more closely related to their interests, so more are completing:

(see http://www.katyjordan.com/timeline for more detail)

Does the course length have an impact on completion? You bet, a negative correlation with course length, so the longer your course, the lower the percentage who see it through. This is probably obvious, but it shows that if you get to 12+ weeks it's probably just some bloke in a shack in Arkansas left:

This raises the question, should we create MOOCs of 2-3 weeks in length (many of Open2Study courses are this long, which may account for the good completion rates)? And if we're creating 2 week MOOCs, how are those different from OERs? UPDATE - see Doug's comments below. He makes the point that obviously the longer a course goes on, the more people will drop out. So we can't say length causes drop out, although even in this respect, you might be better off having 2 x 3week MOOCs than 1 x 6 week one, simply because you catch more people (if completion is important that is).

And a tantalising one to leave you with: does gender influence completion rates? We can't tell as only a few data sets included breakdown by gender, 13 in this case. So we couldn't draw any conclusions from such a small sample, and Katy will probably tell me off for including this, but I do so as a plea to others to release data. From the sample we had, there is a possible positive influence on the percentage of female students and completion, ie women tend to complete more than men, who no doubt wander off to watch football during week 3:

We looked at a number of other factors including assessment type, MOOC type and platform, but the variation in sample sizes makes conclusions from these difficult to form. This data raises a number of interesting questions, which being correlations it can't answer on its own. For me these are:

1) Can we improve those completion rates from around 10%?

2) Are learners getting better at MOOCs?

3) Should we limit ourselves to short MOOCs?

4) What demographic data may influence completion rates?

5) The big one is, does completion even matter? (If it does, then MOOCs probably aren't the solution to your problem)

The Learning Design of MOOCs

I got some Gates funding for the MOOC Research Initiative to look at two things: completion data, and learning design MOOCs. The first part allowed Katy Jordan to finish the work she had started in mapping various factors from over 200 MOOCs that influence completion. You can see more of her work here, and I'll blog on that later.

My part has been using the tools we've developed at the OU for learning design, building on Grainne Conole's work. We use two main tools: the Activity Planner and the Module Map. The first maps student activity across 6 categories, eg assimilative, productive, etc. This is a good way to think about how you want to teach a particular topic. The second one looks at the resources you will need in order to create those activities, and maps them under four headings: Content, Guidance, Demonstration & Communication.

So far I've only done the Module Map applied to 14 MOOCs, chosen from a range of providers and to give both x and cMOOC types. I have created two representations. The first is the Resource Type which shows the range of different media used (video, article, text, audio, etc). Here's an example:

The second representation is more interesting and maps the number of resources used under each of the four categories given above. Here is an example:

Now there are lots of caveats around this. For a start it's a subjective interpretation (Sheila MacNeill and I did the mapping). Secondly it counts number of resources, not length. So 3 x 10 min videos counts as 3 whereas 1 x 30 minute video is just 1. But nonetheless, I think it gives a good indication of the overall weighting in a course. Here is an example of what I think is a fairly typical MOOC, which actually came up for both x and cMOOCs:

Here is an example of a more extreme end of the xMOOC spectrum, with a heavy emphasis on content:

And this is what a more cMOOC looks like (this is DS106):

I was keen to emphasise that it's not the case that one design is 'better' than another, but rather they are doing different things. Sometimes you might want the content approach, and other times you might want a more activity/community centric approach.

In my talk someone suggested they are like food labelling, and that's how I'd like to develop them. You could imagine four or five 'types' so learners, and designers, could decide what type of MOOC this one was. 

In the next phase we will develop some more of the activity profiles for MOOCs to examine in more detail what learners are doing (or what the course designers think they should be doing). But anyway, it feels like a fruitful way of examining them. You can find all the maps on this wordpress blog I've created.

The iceland of Dallas

 

<Dallas deathstar in the snow - this may, or may not, be a metaphor>

I was at the MOOC research initiative conference in Dallas, Texas last week. As Jim and others have reported, we got caught in icemageddon, but that's a whole other (war) story. I'll be doing a few posts about the conference. It was a fantastic meeting, well done George Siemens, Amy Collier and Tanya Joosten for putting it together. I got to have some great conversations, and meet people I've know online for years. Which is by way of apology for my first post being a bit negative.

This one concerns one aspect of the conference that I am having difficulty articulating, so I'm going to try and work it through in this post. There was a data strand to the presentations, and I went to a few of these. There is some fascinating stuff being done, particularly when you have analytics on so many learners. But I also had a vague sense of unease about some of these.

They were often presented by super-smart, young computer-science researchers from privileged universities: Carnegie-Mellon, Stanford, Harvard. I don't have anything against super-smart, young or computer science people, or anyone from those universities (some of my best friends are young people :). But there was something a bit, well, cold, about it all (and not just because of the wind chill factor of -10). People were nodes, and they could be manipulated to move from peripheries to the center by tweaking certain elements. It was easy to forget you were talking about learners, and not sales of baked beans.

But I think we need this research, it's useful and can tell us a lot about what's going on. Candace Thrall was stuck at the airport so couldn't give her presentation, but a colleague gave it on her behalf. She mentioned that one of the transitions we were going through was from a theory-led one to an evidence based one. Prior to this Jim Groom was telling me about Mike Caulfield suggesting we were in post-theory now, where only the big-data mattered (this was from a book I think, if anyone knows which one, let me know). 

UPDATE: Mike has done a great post elaborating on the post-theory debate here. It explains what I was trying to get at in this post.

I felt I had a glimpse of that post-theory world, and I wasn't sure I liked it. We may have been too theory-heavy before, where the evidence was inconsequential, because hey, we have a nice theory. But the pendulum swing to lack of theory where we only care about the evidence seems to lose sight of the people in the system. So I guess my plea to the super-smart, young computer science researchers at ivy league institutions who are now getting into to education is - don't ignore the bearded old guy with a bunch of theories in his back pocket, we need those too. 

Like Dallas, education should be warm and welcoming, and the danger is that data fetishisation will make it like the post-apocalyptic Dallas I experienced.

Stop me if you think you've heard this one before

 

<Image http://www.flickr.com/photos/hanuman/1860273153/>

No doubt most of you will have read the (unintentionally hilarious) interview with Udacity founder and the media's poster child for MOOCs, Sebastian Thrun. If you haven't the short version (minus the ego fanning and competitive cycling) is that Thrun has realised that not many people complete MOOCs, and that making them pay is a good incentiviser, so he's making Udacity an elearning corporate training company.

And there it is. After all that hype. All that "Napsterisation of higher education", the "end of universities", the "10 global providers of education" nonsense, what do we have? A corporate elearning company. As TS Eliot observed, the world ends not with a bang, but a whimper.

Everyone will blog about it (I expect there will be a wave of "the end of MOOCs" pieces by the very people who wrote "the end of universities" ones), and I can barely bring myself to add to the noise, but if you take a step back, it really is a fascinating, and telling case study in what happens when companies try to do openness. 

What is both interesting and depressing about it is the sheer predictability of it all. I commented a while back that FlatWorld Publishing provided a good warning. When the going gets tough, openness is the first casualty. Only last week George Siemens was railing against how people had opted for the easy option because openness was complex and messy. Thrun says it's because he is worried that the Udacity product was 'lousy', but you can bet those venture capitalists were whispering in his ear "where is the return on our investment?".

A couple of points worth noting: Thrun seems to have 'discovered' that open access, distance education students struggle to complete. I don't want to sound churlish here, but hey, the OU has known this for 40 years. It's why it spends a lot of money developing courses that have guidance and support built into the material, and also on a comprehensive support package, ranging from tutors, helpdesk, regional study centres and so on. But of course, none of the journalists and certainly not the new, revolutionary people at Udacity wanted to hear any of this. They could solve it all, and why hadn't higher education thought of this before? As Audrey Watters said to me on twitter:

 

@mweller "disruptive innovation" means never having to say 1) you're sorry 2) you're wrong 3) you're ignorant 4) all of the above

— Audrey Watters (@audreywatters) November 14, 2013

(Audrey has an excellent post on the Thrun interview that you should read)

I also like the way the article depicts Thrun as bravely digging into the data: "he was obsessing over a data point that was rarely mentioned in the breathless accounts about the power of new forms of free online education: the shockingly low number of students who actually finish the classes". That's right, no-one had noticed. If only someone had, say, plotted all of the MOOC completion data...

Anyway, where does this leave us? Does it mean MOOCs are dead? Not really. It just means they aren't the massive world revolution none of us thought they were anyway. And it also suggests that universities, far from being swept away by MOOCs, are in fact the home of MOOCs. You see, MOOCs make sense as an adjunct to university business, they don't really make sense as a stand alone offering. One wonders if the likes of Shirky will be writing about how wonderful the university model of open education is. So in the end, far from being a portent of doom of the university model, MOOCs are a validation of universities and their robustness.

It's not the end of MOOCs, they just make more sense when you view them as part of the OER continuum. Actually I don't think Udacity's product is lousy - they have some really fine open material. It's their business model that's lousy. To quote the Smiths song of the title: "Nothing's changed, I still love you, only slightly less than I used to."