The Ed Techie

Digital Scholarship - the DJ set

I have a vague idea of maybe one day doing a keynote as a DJ set (I think Brian Lamb did this once), but of course, I have never done a DJ set, and it would be a painful affair. So in lieu of that, I decided to create a DJ set to accompany my book. I used Soundcloud, to produce the set below. Lots of tracks were disallowed because they had restricted rights, so my 30 odd track set came down to 12. I've added in a little explanation in between. My mic started dodgy and then packed up altogether, so the quality of these (both in technical and content terms) isn't great, and I don't think I have a career as a late night DJ in the offing, but you get the point. All the songs seem to have been truncated at 3 minutes too, so it's not ideal. 

Of course, the main point is not the content of the set, but the existence of the set itself. I mentioned previously about the boundaries of the book being blurred and this is a further example. I can create a DJ set and share it globally, to accompany a book (about digital scholarship for chrissakes, hardly the funkiest subject!). I mean, come on, that is kinda cool isn't it?

Anyway, I hope you enjoy it, I'll maybe try and find an alternative means of doing it later and get a better mic. Look upon this as a first draft. Maybe a DS106radio set beckons...

 

Digital Scholarship by mweller

Job selection and the paradox of choice

<Image by Ted Percival http://www.flickr.com/photos/tedpercival/3115404982/>

I read Barry Schwartz's The Paradox of Choice recently (after Erik Duval recommended it). The title kind of tells you all you need to know, but his basic argument is that, contrary to what we might think, greater choice isn't always a good thing. As we get more and more choice, we become either paralysed by the possibility of making the wrong choice, or we revert to standard, old practice, ignoring the choice we have.

This came to mind recently when both selecting people for a job interview and talking to someone else who was trying to shortlist. The 'problem' is that in times of financial hardship you get a lot of applicants for every job. In order then to shortlist and also to meet regulations on fair selection, you need to apply a score to the job criteria. With so many applicants, you are effectively looking for reasons to reject, not to include. For example if you have 100 applicants for a post, you can only interview 6 maybe in a day. That's a 94% rejection rate you need to get to in order to progress. 

So, what happens is that you reject people who don't get a perfect score on every criteria. This may mean you get an excellent candidate. But compare what would happen if you only had 20 applicants, say. There you might reject half of them as not having the right skills, but the remaining 10 you would whittle down by looking for who was possible to include. This means you may include people who look interesting, for example, but who haven't quite got the appropriate experience or not exactly the right skills.

The danger of the paradox of choice in job selection is that it essentially narrows your field. You inevitably end of selecting people who are like existing members of staff, because they will most readily meet the criteria. In the long term this may be harmful to the diversity within an organisation. You don't even get to interview people who might be interesting, and offer some alternative views. It might also be harmful for a society going through an economic crisis, because people find it difficult to switch between careers as their experience will not be as readily matched to the criteria.

I know that people can make a case for equivalent experience, but the combination of dealing with an excess of applicants, fair selection criteria and the paradox of choice, almost inevitably leads to the end result of employers getting a narrower range of candidates, not a wider one.  

I'm not sure anything can be done about this, I'm not blaming anyone, rather highlighting how this combination of effects may conspire against the long term goals of an organisation.

Space - The purpose of education

<image http://www.flickr.com/photos/see-through-the-eye-of-g/5414817728/ by GollyGForce>

This is my contribution for the Purpos/Ed blog

In this post my default interpretation of education is that which I’m closest to, namely formal, tertiary education, although some of my points will be valid for other levels of education (primary and secondary) and informal, personal learning.

I’ve been writing about digital scholarship a lot recently, so one way to think of the purpose of education is to turn this on its head, and rather than consider what functions scholars perform, to suggest that the purpose of education is to have a place where these functions reside. Not all of them relate directly to teaching, but they can be seen as a collective set of functions that best make sense clustered around an education system. Boyer defined the four functions of scholarship as discovery, integration, application and teaching. Examples of these would be research, interdisciplinary analysis, public engagement and delivering lectures, respectively. All of these could be done separately and outside of the education system we know, but they are all so interlinked that it makes sense to have them combined within the same institutions and often within individuals who will span all four functions. Although research is often given priority (particularly in promotion), it is really the teaching function that acts as the glue holding these scholarly functions together. So one answer to the question ‘what is the purpose of education?’ is to say that it provides a means by which these functions can exist.

But I think if I was to have to give a simple answer I would say the purpose of education is space. For an individual student it provides space to learn without the need for immediate return or application, space to think, and for many young people for whom higher education is a formative experience, space to discover the sort of person they want to be. For society it provides space and distance from commercial pressure, where expertise can develop independently. Without this space the demands of the instant can overwhelm other considerations. This maintains a crucial mix within society. We need people who are commercially oriented, who react to trends quickly and make fast decisions. We also need those who consider many variables, who take a long view and who remain independent, just as any good group needs a mixture of hawks and doves. The danger of the commercialisation of education is that it seeks to cast everyone in the same mould, and thereby undermine a key function of education.

Your attitude to space will depend on your context: for the person buying a beach front apartment, then space is expensive, to the individual trapped in a packed lift, its absence is suddenly paramount. Getting the balance between these perspectives will be the challenge for education and society over the coming years.

 

Open writing and the integrity of the book

This is the last of my reflections on the book writing process, for now anyway, so you can all stand down now.

In this post I want to explore a bit of the tension between being generally open and the book as artifact. I have blogged small sections of the book as I have gone along, and also parts of the book are based on existing posts. I have wondered whether this open pre-publication approach will undermine the book itself. In an earlier post I mentioned how the book could be seen as being one element in a wider set of resources, and how my online network had been so valuable in the writing process. So I'm aware of the benefits of this way of working, but there may be a downside too.

Here are some of my concerns, or questions:

The loss of the gestalt effect - people may feel they have got all of the main ideas from the various posts over the years, and therefore not feel a need to read the book. This may well be the case, but I feel there is a Gestalt effect with most books (and hopefully mine), that comes from reading a sustained argument with different themes and evidence. That's kind of the point of writing a book, it is more than the sum of it's chapters. So this may be lost by having shared bits and pieces prior to the book itself.

Over-familiarity - sharing some of the main sections and ideas prior to the book publication may mean that when people come to read it they dismiss it along the lines of 'there's nothing new here'. Your own blog posts effectively undermine the authority of the book.

You only get one shot - Grainne has reflected on open book writing on her blog, and I have some sympathy with Martin Oliver's response: "Please, don’t condemn me to having to wade through pages of peoples’ blogs in order to find the one or two good ideas in there! The prospect of blog entries substituting for slow publication isn’t something that thrills me". if you hadn't noticed already, there's quite a lot of stuff out there, so it is perhaps unreasonable to expect people to come to your ideas twice. If you only get one shot at getting these ideas across, maybe it's the book or the blog, not both.

One of the themes of the book is that digital scholarship is in something of a paradox, or there is at least a tension, between the possibilities of new technologies and the preservation of some practices that have served knowledge construction and dissemination so well for centuries. In a very small way, this conflict between being open during the process and maintaining the integrity of the book as an artifact in itself is representative of the bigger tensions in scholarship. 

Overall I would say that openness is beneficial, and in fact I can't really work any other way. Some of my concerns above may well turn out to be ill-founded, or at least overshadowed by benefits. But it would be incorrect to claim the relationship between openness and book writing is one of unending bliss. If they were on Facebook, they would describe their relationship as "it's complicated".

 

5 things I think about Learning Analytics

<Ed Techs go in search of data in the wild>

I am at the 1st Learning Analytics conference in Banff, which has been interesting. I came not sure of what it was, or what my take on it was. The conference has been good, very interdisciplinary in nature (for which you can read 'I didn't understand some of it'). I'm still not sure about a lot of it, but here are five things that have occurred to me over the course of the past few days. I don't believe they're strong enough to say I've learnt them, but rather they are things I now have come to a tentative viewpoing on.

1) Analytics for learners is distinct from analytics of learners. Erik Duval gave a good talk which listed some approaches to empowering learners by providing data and visualisations to them. I think this is potentially very powerful, both in formal and informal settings. Showing learners where they are in respect to others, or giving them recommendations in terms of resources or people, or providing an overview of their activity that helps identify aspects they need to work on, etc. All of these seem like useful things to do. What I am less sure about is the analysis of learners in institutional systems because it will allow them to make efficiency savings.

2) There is a range of intentionality and effort. Tony was talking about the types of things you can build by simply taking one piece of data and then plugging this into various tools. The example that stuck for me was the way by simply picking up on a hashtag he could create a representation of a community, construct a blog roll, mine delicious links for of the people in that community to find resources, etc. So from one simple action we can at least infer a good representation of a community. And to join it all the individual needs to do is tweet with that hashtag. This is distinct from the development of very specific tools, such as a virtual machine that sits on a student's machine and they do their course work within it, or a tool like Cohere. Here the proposition is that we will give you a tool which is useful, and from it we will be able to gain richer data. But the effort required is much greater. I'm not suggesting one approach is better than another, merely pointing out that they are quite distinct. I think our instinct is to usually go for the latter for rich data but the former gets at a wider network.

3) Analytics may reveal some uncomfortable truths. A couple of presentations suggested that when you look at the data then you see that students either don't spend very long in your carefully created content and hop off to Google to find their own, or that doing some activities had no impact on exam performance. This prompted David Wiley to ask 'why do we do them then?', and I suspect analytics may reveal many such uncomfortable moments for course designers, in a way that end of course surveys don't currently. You know that resource you spent lots of money on and thought was great? Students spend 15 seconds on it. That big activity we devised? Has no impact on performance. The real question then becomes, what do you do with this knowledge? It may be that you still wish to persist with some elements because we believe in them pedagogically, or because they develop certain skills. But it may also mean that we find the way we think students study our courses and the way they actually do are quite separate.

4) A little knowledge may be a dangerous thing. Related to the previous point, I also worry that interpreting this data into appropriate action will be a difficult task. And at some point the data may fall into the hands of a senior manager or accountant who will look at it and make a simplistic conclusion. One can imagine decisions such as halving the course staff only drops five percent points in average, or reducing course content by one third results in only 10 more students dropping out, so let's implement the cuts. But there is a bigger picture beyond the pure data, and so viewing it in a holistic and contextual manner is essential.

5) We shouldn't put off ethics issues. There are obviously some rather Orwellian overtones to learning analytics. I don't think this need be the case, but there is a strong commercial interest in this and there is also an element of 'let's record everything because we can'. But I think there are also powerful benefits for the learners too, and whenever we are online we are casting off data anyway. For the academics and researchers we should at least draw up a set of principles from the start that we adhere to. If others want to use data for nefarious means, that's up to them, but here are the things we believe in. For example this could include the premise that analytics is performed with the benefit of learners as a priority, that we are always transparent, that the learner owns their data, etc (Erik pointed to the 4 principles of the now defunct Attention Trust which are a good starting point).

So that's the thoughts I have for now. I also learnt that Banff is a) beautiful and b) flipping cold.

References are like airport security checks

<Image http://www.flickr.com/photos/notionscapital/4228752706/ by Mike Licht>

This is the penultimate post in my week-long series of reflections on the book writing process (you have been reading them all, right?).

I've moaned before that referencing is largely an anachronistic practice which is located in dealing with physical objects. As I said in that post, there are two main reasons that I can see:

1. To properly acknowledge the work of others. The act of referencing provides a clear framework for avoiding plagiarism since it positively encourages students to reference others and thus removes ignorance as an excuse.
2. To allow readers to locate any sources for themselves. This acts as both a check on the author (they can't make up references or misrepresent them), and also promotes knowledge sharing.

The first reason, that of acknowledgement, is obviously valid. But in an online context this is achieved by linking - the link is the acknowledgement, nothing else is required. And indeed, proper acknowledgement is one of the cultural norms of blogging, twitter, etc. We don't have problems with acknowledgement.

But in compiling the references section for my book, I was struck again by the redundancy of official referencing systems. You want to know where a book was published? Really? Isn't Amazon enough? You need a full reference for a quote when the quote itself is the reference in Google? Is an accessed date for every online reference really necessary? 

But I'm a good boy and I have a properly structured reference section of around 230 sources. It seems especially strange as most of my references are online, so I have to find a way of structuring the link into a recognised format, so something like "As Anderson notes.." becomes "Anderson, C. (2008) ‘Freemium math: what’s the right conversion percentage?’ <http://www.longtail.com/the_long_tail/2008/11/freemium-math-w.html> [Accessed 11^th February 2011]". I'm not sure what additional information is conveyed in the second version of this.

This then goes offline into a printed book, fair enough, and will then go back online. But what bugs me is the supposed supremacy of print in this. For an online reference of a journal article, the URL is not sufficient but rather the full journal reference is required, because that legitimises it.

And we treat referencing as such an immutable practice. This is a classic of example of where the method has become confused with the intention. What we want is to find sources and for that work to be acknowledged. In a print world a standard method of describing these characteristics to enable location made sense. It makes little sense in an online world. 

This reminds me of airport security checks, such as questions around whether you packed yourself or are carrying guns, body checks, etc. Any questioning of the practice is dismissed with very serious concerns about terrorism. Just as querying the reference system is met with furrowed brows and deep sighs about plagiarism and proper scholarship. But like those airport checks this adds delay, is cumbersome and only inconveniences people who are happy to play by the rules anyway. It does nothing to catch the hardcore plagiarists, and doesn't guarantee a reference will still be available any more than a link does. Yet we have no choice and have to persist in it.

If I had the courage of my convictions, here is what I should have submitted for my reference section:

JFGI

Book analytics

This is the fourth in my musings on the book writing process. When my book comes out it will be freely available online, in HTML (I think). I don't know how the site will be constructed, but it strikes me that as an author this is potentially very interesting.

I'm at the Learning Analytics conference at the moment, and Tony has been talking about Google Analytics. I run this on my blog, and it's always interesting, but the type of data it might give a book author is more revealing. On my blog it tells me where people have come from, that most people tend to come from search, and come to one post only. That's probably what you'd expect.

But analytics on a book is a different thing. As an author you have no idea how people actually read your stuff or what they find interesting. Potentially (it will depend on how the site is constructed) it could show me the following:

1. How long people read
2. What elements are most popular
3. What search terms bring people to my book (which could be seen as a proxy for what the audience really is)
4. What sections are being linked to (and therefore are presumably of most use?)
5. If people come back, which would be some indication of interest and relevance

You'd never know this stuff from a conventional book, which is one of the advantages of working this way. What would be interesting then would be if I did a rewrite of the book, how much this analytics might influence the process. If one chapter hardly ever got read, should I just excise it? Conversely should I expand a section that gets lots of links? 

What I won't know is if there are different reading patterns for the physical book and the online version. I suspect people will tend to read the paper version in longer chunks, and the online one in a less linear fashion. But paper doesn't give me any data back, dammit.

 

A tale of two books

Continuing my thoughts on the book writing process, this post is about how the process itself has changed, for me at least, over recent years.

I start my book by comparing the process of writing this book with my last one about 6 years ago. Below is a list of some of the tools and resources I used to write this book:

1. Books – via the library but increasingly as ebooks and on Kindle, and one audiobook
2. E-journals – my university library has access to a wide range of databases, but I also made frequent use of others through tools such as Google Scholar and Mendeley
3. Delicious/social bookmarking – as well as searching for key terms I would ‘forage’ in the bookmarks of people I know and trust, who make their collections available
4. Blogs – I subscribe to over 100 blogs in Google Reader, which I try to read regularly, but in addition I have cited and used many posts from other blogs
5. YouTube, Wikipedia, Slideshare, Scribd, Cloudworks and other sites – text is not the only medium for sharing now, and for certain subjects these ‘web 2.0’ services offer useful starting points, or overviews, as well as insightful comment
6. My own blog – I have kept a blog for around five years now, and it provided a useful resource for items I have commented on, and drafts of sections of this book. I also have a scrapbook type blog I keep using Tumblr where I post any interesting links or multimedia, and revisited this for resources I had harvested over the past few years. The blog was also a means of posting draft content to gain comments and feedback, which could then be incorporated into further iterations of writing
7. Social network – my Twitter network is especially useful for gaining feedback, asking for suggestions, and on a daily basis, as a filter and collection mechanism for sharing resources
8. Work and personal network – undoubtedly working in an intellectually lively environment, and having face to face discussions with colleagues has been invaluable
9. Google alerts – I set up alerts for a few key phrases, which would then provide me with daily email updates on new content containing these key words. This allowed me to find new resources, track conversations and stay abreast of a field which was changing as I wrote the book
10. Seminars and conferences - my attendance at face to face conferences has declined due to other commitments but I regularly attend, or dip into conferences remotely (see Chapter 10 for a more detailed exploration of the changing nature of conferences)

 

If I compare this with the tools I used when I wrote my last book in 2004, then many of these services did not exist or were in their infancy. Of this list I probably used books, journals and face to face conferences, with maybe some initial exploration of blogs. 

In many ways the changes are not dramatic – books and journal articles still constitute a large part of the information sources I draw upon, although blogs in particular have increased in significance. And the output of all this is still that most traditional of information sources, a book. But the changes are also significant for three reasons, I would suggest.

Firstly, the quantity of this information that is available online has increased considerably. I could access nearly all of it from my desk at home, there was no need to visit a physical library or bookstore. The digitisation of relevant content was almost total for the information sources I required for this book, compared with about half of that in 2004.

The second factor is the significance of my online network in the writing process. I have around 3,000 followers in Twitter and around 2,000 subscribers to my blog (often they are the same people), which represents a wide pool of experience to draw upon. Sometimes I would put out a direct call to this network, along the lines of ‘Does anyone have a good example of …’ In other cases I would post drafts of the content to my blog and receive comments and links to relevant material. Even without these direct appeals this distributed, global peer network represents an invaluable information source, comprising links to resources, commentary on issues, extended debate, use of new methods and technology, and contributions in the form of blog posts, videos and audio.  

This last item leads me to the third significant difference from the previous book, which is the range and variety of content that I drew upon. Even six years ago the type of content was largely limited to journal articles and books. Now, this has diversified to include blog posts, videos, draft publications, conference presentations and also the discussion, comment and debate surrounding each of these. The change from 2004 is partly a result of the first factor, quantity. There is just more of this stuff around. But it is also a result of a shift in attitude (at least on my part), in the legitimacy of these other forms of output, and their central, vital role in everyday scholarly activity.  

The comparison of writing these two books is instructive, I feel, because it gets to the heart of digital scholarship: it is both a profound change and a continuation of traditional practice. This can be seen with the final output also: the previous book existed only in traditional, paper format, and the copyright to this was owned by the publishers. The new book is available in the traditional format, but also online, freely available under a Creative Commons licence. In addition there is a set of resources, such as videos, presentations, blog posts, etc which relate to the book, and then comment and reaction to these. And this is becoming more common: my colleague Grainne Conole, is writing a book by blogging chapters and gaining feedback in a site called Cloudworks (http://cloudworks.ac.uk/cloudscape/view/2155). Conor Gearty, a Professor of Law at the London School of Economics, is writing a book by posting a weekly video which sets out his theme and encourages discussion (http://therightsfuture.com/). The boundary between Gearty’s book and a course, and the comments of the participants is deliberately blurred.

This conflict between what, from one perspective, seems a substantial change in practice, and from another, appears to be a conservative, minor adjustment to standard approaches, characterises not just book production, but any aspect of scholarly activity, including research, knowledge dissemination, public engagement and teaching. Both radically different and yet familiarly traditional seems to be the story of digital scholarship at the moment, and it is the tension between these two forces that my book sets out to explore.