The Ed Techie

Completion data for MOOCs

As I mentioned in the previous post, I am doing some Gates funded research on MOOCs. My part was learning design analysis, while Katy Jordan has been looking at factors influencing completion rates. All this work is Katy's, I take no credit for it. She would blog it, but is about to have her first baby any day now, and strangely that has taken priority over blogging about MOOCs, so she said I could blog it on her behalf.

There will be a paper that details the full results and methodology, so I'm just giving some highlights here. Katy collected completion data from 221 different MOOCs. The range was limited to ones where we could get this data, so is biased by a couple of big data sets from Coursera and Open2Study. This last one in particular has good completion rates so may have biased the overall results somewhat. 

If we start with enrolment figures first then, the picture looks like this:

(N = 220; Median = 18941; Minimum = 95; Maximum = 226,652). Although the really big numbers get the headlines, there are a lot of courses in the 0-10000 range.

Completion rates across this sample are:

(N =  129; Median = 12.6%; Minimum = 0.7%; Maximum = 52.1%). This bimodal distribution may be a result of those Open2Study courses. Those guys appear to be doing something right in terms of completion.

Does enrollment numbers relate to completers? The answer is yes, there is a significant negative correlation, ie the more people who enrol then the lower the percentage who complete. This single graph might well put paid to the idea that we can educate the world with massive courses:

Yeah, I hear you say, but it's not completion that's important but how many people are active. I'm afraid that shows exactly the same picture, ie there is a significant negative correlation between number of active users and the percentage who complete:

Does the reputation of the university matter? The answer is: possibly. There is a positive correlation between the Times Higher Ed score of a university and enrolments:

But this could just be because more elite universities have joined the elite clubs of MOOC providers. As with all correlation we can't imply causality, it may not be the university reputation that leads directly to the higher enrolments but a related factor. Anyway, it turns out that reputation has no effect on completion rates:

Have enrolments increased or decreased over time I hear you ask. Well, there is a negative correlation over time with enrolments, so numbers on individual MOOCs have declined. This may be that with more MOOCs available people can find the one that suits them better, instead of everyone doing AI, but if you wanted to be all sensationalist (I'm looking at you, journalists) you could say it indicates that the MOOC bubble has burst:

Have completion rates changed over time? Yes, in the opposite direction, there is a positive correlation with completion and date. This could mean people are getting better at MOOCs, and if the trend continued those completion rates would be less of an issue. But I suspect it is related to the point above, in that there are more MOOCs around so people are taking ones more closely related to their interests, so more are completing:

(see http://www.katyjordan.com/timeline for more detail)

Does the course length have an impact on completion? You bet, a negative correlation with course length, so the longer your course, the lower the percentage who see it through. This is probably obvious, but it shows that if you get to 12+ weeks it's probably just some bloke in a shack in Arkansas left:

This raises the question, should we create MOOCs of 2-3 weeks in length (many of Open2Study courses are this long, which may account for the good completion rates)? And if we're creating 2 week MOOCs, how are those different from OERs? UPDATE - see Doug's comments below. He makes the point that obviously the longer a course goes on, the more people will drop out. So we can't say length causes drop out, although even in this respect, you might be better off having 2 x 3week MOOCs than 1 x 6 week one, simply because you catch more people (if completion is important that is).

And a tantalising one to leave you with: does gender influence completion rates? We can't tell as only a few data sets included breakdown by gender, 13 in this case. So we couldn't draw any conclusions from such a small sample, and Katy will probably tell me off for including this, but I do so as a plea to others to release data. From the sample we had, there is a possible positive influence on the percentage of female students and completion, ie women tend to complete more than men, who no doubt wander off to watch football during week 3:

We looked at a number of other factors including assessment type, MOOC type and platform, but the variation in sample sizes makes conclusions from these difficult to form. This data raises a number of interesting questions, which being correlations it can't answer on its own. For me these are:

1) Can we improve those completion rates from around 10%?

2) Are learners getting better at MOOCs?

3) Should we limit ourselves to short MOOCs?

4) What demographic data may influence completion rates?

5) The big one is, does completion even matter? (If it does, then MOOCs probably aren't the solution to your problem)

The Learning Design of MOOCs

I got some Gates funding for the MOOC Research Initiative to look at two things: completion data, and learning design MOOCs. The first part allowed Katy Jordan to finish the work she had started in mapping various factors from over 200 MOOCs that influence completion. You can see more of her work here, and I'll blog on that later.

My part has been using the tools we've developed at the OU for learning design, building on Grainne Conole's work. We use two main tools: the Activity Planner and the Module Map. The first maps student activity across 6 categories, eg assimilative, productive, etc. This is a good way to think about how you want to teach a particular topic. The second one looks at the resources you will need in order to create those activities, and maps them under four headings: Content, Guidance, Demonstration & Communication.

So far I've only done the Module Map applied to 14 MOOCs, chosen from a range of providers and to give both x and cMOOC types. I have created two representations. The first is the Resource Type which shows the range of different media used (video, article, text, audio, etc). Here's an example:

The second representation is more interesting and maps the number of resources used under each of the four categories given above. Here is an example:

Now there are lots of caveats around this. For a start it's a subjective interpretation (Sheila MacNeill and I did the mapping). Secondly it counts number of resources, not length. So 3 x 10 min videos counts as 3 whereas 1 x 30 minute video is just 1. But nonetheless, I think it gives a good indication of the overall weighting in a course. Here is an example of what I think is a fairly typical MOOC, which actually came up for both x and cMOOCs:

Here is an example of a more extreme end of the xMOOC spectrum, with a heavy emphasis on content:

And this is what a more cMOOC looks like (this is DS106):

I was keen to emphasise that it's not the case that one design is 'better' than another, but rather they are doing different things. Sometimes you might want the content approach, and other times you might want a more activity/community centric approach.

In my talk someone suggested they are like food labelling, and that's how I'd like to develop them. You could imagine four or five 'types' so learners, and designers, could decide what type of MOOC this one was. 

In the next phase we will develop some more of the activity profiles for MOOCs to examine in more detail what learners are doing (or what the course designers think they should be doing). But anyway, it feels like a fruitful way of examining them. You can find all the maps on this wordpress blog I've created.

The Battle for Open

<This is how we do protest in Wales>

We have a new issue of JIME out, which is a special issue on the OER 13 conference. There are some excellent papers, so go read them. Then, when you've read all those, read my opinion piece, The Battle for Open.

I'm going to write a book about this over the next few months, so if you think it's an awful idea, please let me know. The article is effectively the first chapter of that book, which sets out the overall argument. My argument goes something like this: Openness has been victorious in many ways, but at this point of victory the real struggle begins. If you look at openness in research, teaching, publication scholarship, then it's hard to argue that openness hasn't been successful over the last few years in establishing itself as a core approach in higher education. It isn't something just a few oddballs bang on about now, it has moved to the centre of discourse (and, more importantly, funding).

And yet it doesn't feel like it should. We aren't seeing David Wiley leading triumphant processions down Wall Street. The MOOC invasion and backlash is the most visible part of all this, but I think that is just representative of a wider story. I try to articulate why I see it as a 'battle', giving three reasons:

1) There is real conflict between different visions

2) There are considerable spoils to be won

3) The victor writes history, so there is a battle for narrative taking place

What I want to do then in the book is explore this victory/battle dilemma for a number of areas, and then come to some conclusions. My main hypothesis is that open education isn't an interesting subset of education now, it is education. And so if you care about education you should care about the direction that openness takes. We have made mistakes in the past by outsourcing issues, such as selling off university presses or buying in LMSs. This meant education no longer controlled or had much of a say in core parts of its key functions. We are repeating this mistake with MOOCs and commercial companies. Openness cuts across all of these and so I see it as the key principle that educators and universities need to engage with.

As you can probably tell, it'll be a struggle to control the polemic, but I hope it will be a reasoned, well argued book with plenty of research and evidence, and I'll keep the spitty, table-thumping to a minimum.

The iceland of Dallas

 

<Dallas deathstar in the snow - this may, or may not, be a metaphor>

I was at the MOOC research initiative conference in Dallas, Texas last week. As Jim and others have reported, we got caught in icemageddon, but that's a whole other (war) story. I'll be doing a few posts about the conference. It was a fantastic meeting, well done George Siemens, Amy Collier and Tanya Joosten for putting it together. I got to have some great conversations, and meet people I've know online for years. Which is by way of apology for my first post being a bit negative.

This one concerns one aspect of the conference that I am having difficulty articulating, so I'm going to try and work it through in this post. There was a data strand to the presentations, and I went to a few of these. There is some fascinating stuff being done, particularly when you have analytics on so many learners. But I also had a vague sense of unease about some of these.

They were often presented by super-smart, young computer-science researchers from privileged universities: Carnegie-Mellon, Stanford, Harvard. I don't have anything against super-smart, young or computer science people, or anyone from those universities (some of my best friends are young people :). But there was something a bit, well, cold, about it all (and not just because of the wind chill factor of -10). People were nodes, and they could be manipulated to move from peripheries to the center by tweaking certain elements. It was easy to forget you were talking about learners, and not sales of baked beans.

But I think we need this research, it's useful and can tell us a lot about what's going on. Candace Thrall was stuck at the airport so couldn't give her presentation, but a colleague gave it on her behalf. She mentioned that one of the transitions we were going through was from a theory-led one to an evidence based one. Prior to this Jim Groom was telling me about Mike Caulfield suggesting we were in post-theory now, where only the big-data mattered (this was from a book I think, if anyone knows which one, let me know). 

UPDATE: Mike has done a great post elaborating on the post-theory debate here. It explains what I was trying to get at in this post.

I felt I had a glimpse of that post-theory world, and I wasn't sure I liked it. We may have been too theory-heavy before, where the evidence was inconsequential, because hey, we have a nice theory. But the pendulum swing to lack of theory where we only care about the evidence seems to lose sight of the people in the system. So I guess my plea to the super-smart, young computer science researchers at ivy league institutions who are now getting into to education is - don't ignore the bearded old guy with a bunch of theories in his back pocket, we need those too. 

Like Dallas, education should be warm and welcoming, and the danger is that data fetishisation will make it like the post-apocalyptic Dallas I experienced.

Stop me if you think you've heard this one before

 

<Image http://www.flickr.com/photos/hanuman/1860273153/>

No doubt most of you will have read the (unintentionally hilarious) interview with Udacity founder and the media's poster child for MOOCs, Sebastian Thrun. If you haven't the short version (minus the ego fanning and competitive cycling) is that Thrun has realised that not many people complete MOOCs, and that making them pay is a good incentiviser, so he's making Udacity an elearning corporate training company.

And there it is. After all that hype. All that "Napsterisation of higher education", the "end of universities", the "10 global providers of education" nonsense, what do we have? A corporate elearning company. As TS Eliot observed, the world ends not with a bang, but a whimper.

Everyone will blog about it (I expect there will be a wave of "the end of MOOCs" pieces by the very people who wrote "the end of universities" ones), and I can barely bring myself to add to the noise, but if you take a step back, it really is a fascinating, and telling case study in what happens when companies try to do openness. 

What is both interesting and depressing about it is the sheer predictability of it all. I commented a while back that FlatWorld Publishing provided a good warning. When the going gets tough, openness is the first casualty. Only last week George Siemens was railing against how people had opted for the easy option because openness was complex and messy. Thrun says it's because he is worried that the Udacity product was 'lousy', but you can bet those venture capitalists were whispering in his ear "where is the return on our investment?".

A couple of points worth noting: Thrun seems to have 'discovered' that open access, distance education students struggle to complete. I don't want to sound churlish here, but hey, the OU has known this for 40 years. It's why it spends a lot of money developing courses that have guidance and support built into the material, and also on a comprehensive support package, ranging from tutors, helpdesk, regional study centres and so on. But of course, none of the journalists and certainly not the new, revolutionary people at Udacity wanted to hear any of this. They could solve it all, and why hadn't higher education thought of this before? As Audrey Watters said to me on twitter:

 

@mweller "disruptive innovation" means never having to say 1) you're sorry 2) you're wrong 3) you're ignorant 4) all of the above

— Audrey Watters (@audreywatters) November 14, 2013

(Audrey has an excellent post on the Thrun interview that you should read)

I also like the way the article depicts Thrun as bravely digging into the data: "he was obsessing over a data point that was rarely mentioned in the breathless accounts about the power of new forms of free online education: the shockingly low number of students who actually finish the classes". That's right, no-one had noticed. If only someone had, say, plotted all of the MOOC completion data...

Anyway, where does this leave us? Does it mean MOOCs are dead? Not really. It just means they aren't the massive world revolution none of us thought they were anyway. And it also suggests that universities, far from being swept away by MOOCs, are in fact the home of MOOCs. You see, MOOCs make sense as an adjunct to university business, they don't really make sense as a stand alone offering. One wonders if the likes of Shirky will be writing about how wonderful the university model of open education is. So in the end, far from being a portent of doom of the university model, MOOCs are a validation of universities and their robustness.

It's not the end of MOOCs, they just make more sense when you view them as part of the OER continuum. Actually I don't think Udacity's product is lousy - they have some really fine open material. It's their business model that's lousy. To quote the Smiths song of the title: "Nothing's changed, I still love you, only slightly less than I used to."

What sort of open do you want?

<Image http://www.flickr.com/photos/42386632@N00/8528725328/ >

I've been thinking about openness in education a lot recently (my plan is to write a book on this, more on that later). And I've slowly, probably years after everyone else, come to the conclusion that it's a mistake to talk about openness as if it's one thing. There may have been a time when it was, when all the forms of openness blended easily into one indistinguishable lump, but that's not the case now. Not only are there different aspects of openness, but I'm beginning to feel that some may be mutually exclusive with others, or at least prioritising some means less emphasis on others. 

What do I mean by this? Well I could list the different types of openness in education: OERs, MOOCs, open access, open scholarship, etc. But instead it's more useful to consider the motivation for openness, why has someone adopted an open approach in the first place? Here are some possibilities:

Increased audience - you want as many people as possible to be able get at your resource, be it an article, book, course, video or presentation. The main aim here is to remove barriers to people accessing it. This means it has to be free, easily shareable, online, and with easy rights.

Increased reuse - related to the above, but slightly different, you want people to take what you have created and combine it with other elements, adapt it and republish. The same considerations are required as above, but with an extra emphasis on minimal rights and also creating the resource in convenient chunks that can be adapted. Whereas the first motivation might mean releasing your song online, the second motivation might lead you to sharing each of the tracks that constitute the song separately under a CC-BY licence.

Increased access - this is different from the first motivation in that you want to reach particular groups who may be disadvantaged. You have open access in that you do not require formal entry qualifications. You may decide that free is also the best way to gain increased access, but that may not follow. If you want to work with learners who often fail in formal education then simply making a resource free doesn't really help. What they need is material aimed at this audience and specific support. Open access is not just about price.

Increased experimentation - one of the reasons many people adopt open approaches is that it allows them to do different things. Whether that's use different media, create a different identity, or experiment with approach that wouldn't fit within the normal constraints of work, an open method allows this. If this is your goal, then the emphasis is on getting an audience that will feedback on this and maybe participate in this experimentation.

Increased reputation - being networked and online can help improve your, or your institution's, profile. Openness here allows more people to see what you do (the motivation of increased audience) but your main aim is to enhance your reputation. If you were an academic who really wanted to be on the keynote circuit then operating in the open, publishing openly, creating online resources, being active in social media and establishing an online identity might be a good way to achieve this. Here openness is a method by which you realise a different goal, but it could be anything. The emphasis here then will be on networking and using openness to establish identity.

Increased revenue - while we may have suspicions about open washing and using openness as a route to commercial success, it's true that an open, or part-open model can be an effective business model. The freemium approach works this way, where a service is open to a large extent, but some users pay for additional services. If this is the goal then openness works by creating a significant demand for the product.

Increased participation - you may need input from an audience, but can't pay to access them. This could be crowdsourcing in research, or getting feedback on a book or research proposal. Being open allows others to access it and then provide the input you require. Openness here can be quite targeted, you want to reach a particular audience and get them involved, not necessarily as large an audience as possible.

Let's take an example in education and consider it from these different motivations. Let's imagine your university (or a university you know) wants to create a MOOC. They've heard all about them and think they should be doing something in this area (this actually covers about 90% of university's approach to MOOCs). They seek your advice, so you go around to a bunch of different stakeholders and you ask them "what is the aim of the MOOC? What do you want from it?"

Now, the person from marketing says they want to increase the university's online profile and reputation. From this perspective you propose a MOOC in a big hitting subject, featuring a big name academic. The subject will be "Life on Mars". It'll be expensive, high end production, acting as a showcase for the university and getting it in the press.

When you speak to the Dean of the Science faculty they say they are concerned about student recruitment on their post grad course. They want it to bring in high fee paying overseas students. The model that might work here is one that makes the first 6 weeks open, and target a specific audience, who can then sign up. You suggest offering badges and allowing people to build up to taking the full masters.

You then speak to an academic who is really keen to try a student-led approach. They feel frustrated by the customer-led focus of conventional teaching and see in MOOCs an opportunity to try some more radical pedagogic approaches that they have been blocked from implementing. They don't see it as particularly massive in terms of audience, but it will be a rich learning experience for those who do it, as they will be creating the curriculum. You propose a MOOC based in Wordpress, and featuring a range of technologies with learners co-creating the content.

Later you have a conversation with a funding council who want to bring under-represented groups into science. They will need a lot of support, but they are willing to fund the provision of mentors and support groups in the community. You suggest a MOOC based on adapting existing materials, with carefully targeted support and minimal technical barriers.

And so on - you can see that from each of these perspectives the resultant MOOC would be a very different beast. It would be open in each of these scenarios, but with a different emphasis on the form that openness should take.

So, now that openness has to a large extent won out as an approach, the question is no longer 'do you want to be open?' but rather 'what sort of open do you want?'. 

Earning the right to preach

I watched the tweets from the WISE13 conference with interest. One that caught my eye (and rather made it water) was Gordon Brown extolling the value of education and its benefits for society. You can see his talk below:

It's a good talk, impassioned and well reasoned. I agree with everything he said. But I find it strange that he is saying it. Brown's record on international development is good, it's something he really believed in. But when he was Prime Minister, he abolished what's called ELQ funding in the UK. This meant that you couldn't get funding to study if you already had an equal or higher qualification. This was enormously damaging for many of the people you want to avail themselves of education - those who have been made redundant and want to reskill for examples. 

I appreciate this is a minor concern compared with global education for all, and particularly the social power of getting girls into school. But it's the principle - you don't get to talk about the wonderful power of education if when you had the chance you looked at the budgets and thought "nah, education's not worth it, we can make a saving there." 

Everyone (well maybe not some of the Taliban) thinks education is a good thing. It's an easy totem to gather around and make impassioned calls for. Education has a lot of social credit. We shouldn't let people use it without having shown their credentials, and Brown botched his chance when he had it to demonstrate that he really believes in the transformative power of education. Of course, compared with what's come afterwards, Brown's crime is minor in comparison, but even so, you don't get to use education now as your platform. Or at least not without some contrition. The same goes for openness - you have to earn that badge through action.

Open media and the new Stasi

"Trust me, I'm a journalist..."

I came across a disturbing story last week, particularly for academics in the UK who use social media. A UK academic who blogs and tweets as Plashing Vole reported that a journalist from the Sun on Sunday had contacted him, telling him they were going to run an article about him. It concerned a few anti-government tweets he had made, in one jokingly making a Nazi comparison. An MP was calling for him to be sacked. In the end they didn't run the piece, but it must have been a stressful few days.

This worries me on a number of levels. Firstly, I often encourage academics to establish an online identity, and that part of that identity is to give a bit of themselves, not just bland broadcast. If we are now entering a phase when any opinion or joke will be used against you then this advice might be harmful.

Secondly, I think most of us will have a 'there but for the grace of God' moment on reading his account. I am mindful of being offensive in any online communication, but I've made jokes & expressed opinions so if someone really wanted to I'm sure they could construct a negative story about me and use a few carefully selected tweets to prove a case. The same is true for any of us I think - a friend of mine was berated by someone because they tweeted about biscuits during work hours.

But the thing that bothered me most of all was the sense of surveillance and suppression of opinion. The East German secret police used to have people in every workplace reporting to them, and they would drill holes in walls to spy on citizens. Any anti-state sentiment that was expressed would be recorded and that person would find their life difficult, or they'd just be disappeared. The consequence was that no-one would express any dissenting view in public, which was, of course, the aim. It is the same here, it doesn't matter that they didn't run the story, they only need to threaten to often enough to make academics scared of expressing an opinion publicly. And that is a very dangerous society to live in.

The possible saving grace is that I would hope universities are strong enough to stand up to this nonsense, and also, I think it's such a non-story, and would be of so little interest to their readers that it's not a practical policy. But who knows?

MOOCs as 1st year undergrad replacement

For top-secret research I am undertaking, I'm looking at a range of MOOCs, both xMOOCs, cMOOCs and flavours inbetween (although, definitely not ridiculous variations such as SPOCs). Here's some breaking news - they are all pretty good. Take away all the hype, commercial bubble and rabid arguments on both sides and you are left with some good teaching material. 

As I've been going through them (admittedly not as thoroughly as a student), I've begun to think that a mix of them would probably represent a good grounding in a topic, equivalent to a 1st year of an undergrad degree. It wouldn't teach some of the other skills you develop, I'll come to that later. Let's take an example, say I want to study a degree in Psychology. The following MOOCs would give me a good knowledge base:

• Udacity's Introduction to Psychology - good overview of the main areas of Psychology, covers experimental design, major topics, ethics, etc
• Coursera Introduction to Statistics - a basic intro to stats 'for everyone'
• EdX Descriptive Statistics - good to get students to understand interpreting and presenting data clearly
• Saylor's Cognitive Psychology - going a bit deeper into a sub-discipline (and reinforcing bits from the intro course)
• Saylor's Social Psychology - ditto for above
• Lancaster's Linguistics course - a kind of elective, maybe choose from a range of topics eg philosophy

Now, I think that would give you a good grounding in knowledge. I know from doing my first degree in Psychology that the first year is really spent bringing everyone up to speed. A second year could then start on the assumption that all of the above is known to all students. This is where a conventional (campus or distance) university can step in. The MOOCs only take you so far. They're good at getting across content, but not so good at developing skills. As a Psychology graduate there are key skills you need to develop (the elusive qualities of 'graduateness'), such as critical thinking, reading and interpreting scientific literature, debate and communication skills, experimental design, etc. These are really best developed by interaction with other learners and experts in a more structured, focused manner than most MOOCs offer.

So here's a model for a university wanting to offer something different - come to us with certificates in all of the above MOOCs and we will enroll you on a shortened two year degree programme. Because we want to be competitive our fees (assuming a UK uni here) are set at £7K per annum (compared with the usual 9K) and that means your degree will cost you £14K, not the usual £27K. That begins to look like a good offer, and I would be willing to bet that there would be no difference between these students at graduation than those that have studied a three year programme.

Of course there are a whole host of objections to this model, for instance it can undermine universities, it plays along with the broken funding regime, a three year degree programme is the right length of time for personal development, and so on. I wouldn't disagree with any of these. And I wouldn't suggest that this is the only model that should be pursued, but rather it is an example of how changes in education, and open education in particular, could offer a wider diversity of university models.

One parting thought - if this model was used successfully I wonder how long before the MOOC providers started charging for their courses to be used in this way?

Autumn issue of JIME

In times of fake open access journals, and open access being used as a means of making even more money by publishers, it's nice to know that some things are true to the simple values of open access... yes, there is a new issue of JIME out. It may not have the bells and whistles of a funded journal, and maybe we can't give it as much time as we'd like, but it's free to publish, peer-reviewed and open to all. 

In this issue there is quite a range of papers, some have a 'design' theme, but it's not a themed issue. Here is the editorial, I'm sure there's something there to pique everyone's interest:

This edition of JIME sees a varied collection of papers covering a diverse range of topics. It starts with a perspective piece by the Director of the Institute of Educational Technology at The Open University, Josie Taylor, addressing the institutional approach to pedagogy and technology here at the OU. This piece was originally given as a speech internally and focuses on the OU experience, but it will be of interest to others, particularly as they seek to blend new technologies and pedagogies. Its discussion of the learning model and the connection and journey between informal and formal learning is of particular contemporary interest.

There are three papers concerned with different aspects of design: designing online learning or blended courses - and the role of learning design in developing such courses. The first of these is an article by Terumi Miyazoe and Terry Anderson, which provides a much needed theoretical basis for examining OERs and MOOCs. Using the Interaction Equivalency Theorem, they address issues such as the scalability of different MOOC models.

In the third article Canan Blake and Eileen Scanlon focus on designing for collaborative learning and provide a mix of the theoretical and practical in examining the nature of collaborative online learning and its relation with the learning environment. They note that, whilst much prior research has focused on empirical studies to understand the mechanisms, as well as the benefits of collaboration, this is not always helpful in addressing the challenge of designing good collaborative learning activities.

The fourth article, by Anna Mavroudi and Thanasis Hadzilacos, picks up on both themes of collaboration and design again in its discussion of group work in designing for adaptive learning. This paper presents an empirical case study of collaboration on a complex design text, which like Blake and Scanlon's paper above, is in the context of Open and Distance Education, and which has implications for design guidelines in contemporary teaching contexts.

A less theoretical and more practical focus is given in the final article, where Nick Pearce and Sarah Learmonth investigate the use of Pinterest as a tool for anthropology students, suggesting that its non-linear format aids the integration of material in an interdisciplinary subject.

This issue also includes a book review written by Bethany Alden about Joseph P McDonald's book "Going online with Protocols: new tools for teaching and learning".