UK United Kingdom

The Teaching and Learning Foundations of MOOCs

In Daphne Koller’s TED talk about Massive Open Online Courses (MOOCs), she discusses what she believes are the pedagogical foundations of MOOCs. For her, and also restated on the Coursera website (the company she helped found), those foundations include the effectiveness of online learning, mastery learning, peer assessments and active learning.

Of course the evidence cited to support these foundations is scant and is quoted without giving any context to the research. MOOC critics also rarely produce any actual evidence when dismissing MOOCs as being pedagogically unsound or when stating that they offer a worse experience than face-to-face education on campus.

Given that my university (the University of Western Australia) was about to embark on hosting our own MOOCs and combine them with a flipped-classroom mode of teaching for our own students, a colleague (Martin Forsey) and I decided to examine the actual research evidence supporting (or not) the notion that MOOCs are based on sound pedagogical foundations. We published this in First Monday and what follows is a summary of our findings.

The MOOC fingerprint

The starting point for us was categorising the attributes that define a MOOC as they are presented on platforms such as edX, Coursera and Udacity. These were: online delivery of content, lectures formatted as short videos combined with formative quizzes; automated assessment and/or peer and self-assessment and an online forum for peer support and discussion. The pedagogical benefits of these characteristics of MOOCs translated into: the effectiveness of online learning, retrieval learning, mastery learning, enhanced learning through peer and self-assessment, enhanced attention and focus due to “chunking” content into small packages and finally peer assistance, or out-of-band learning.

Having drawn up this list of MOOC characteristics and their corresponding pedagogical effects, we reviewed the literature for empirical research to examine whether there was evidence to support any pedagogical benefits.

The Effectiveness of Online Learning

The most obvious characteristic of MOOCs is that they are delivered online. Reviewing the research, there is little evidence that online learning is any worse than face-to-face modes of teaching and there is plenty of evidence to suggest that it may in fact be better for some of the reasons that we are about to cover. Of course, this is not the general perception amongst US faculty staff at least. In a recent survey, 66% of faculty staff believed that online outcomes where inferior to traditional modes of teaching. The underlying subtext of why people believe that online teaching is less effective has not been extensively examined but certainly one would assume that it is related to a fear of technology generally which includes a fear of being replaced by that technology.

Retrieval Learning

MOOCs emphasise modes of learning that include retrieval learning and mastery learning. Retrieval learning acts to enhance long-term memory of facts by prompting recall of information from short-term memory. Each time this happens, it serves to improve overall learning as connections are made and strengthened each time there is a retrieval of information. In MOOCs, retrieval learning is enhanced through the practice of using frequent formative quizzes. Again there is evidence from multiple sources to support the notion that the use of tests and quizzes enhances learning over simply allowing students to listen or read content.

Mastery Learning

In regard to the benefits of mastery learning, this has a long history within education dating back to Bloom’s coining of the term in 1968. The basic premise is fundamental to the benefits of a site such as Salman Khan’s Khan Academy. Students are allowed to progress at their own pace until they achieve mastery of a particular topic and this may involve revisiting concepts that the topic is reliant on. Mastery learning is also enhanced by one-on-one tutorial style teaching and there have been arguments made that watching videos that can be controlled by the student gets closer to that mode of teaching than sitting in a lecture theatre.

Although there is evidence for the principle of mastery learning, there isn’t any evidence that MOOCs as they are currently implemented actually support this mode of learning. For the most part, MOOCs are being presented in the traditional timeline of different topics each week and are not especially structured to reinforce mastery learning compared with the Khan Academy for example.

Peer and Self-Assessment

When it comes to peer and self-assessment, there is general agreement that it is an effective means of marking. Assignments that are peer or self-assessed agree closely to those marked by instructors and tutors.

The evidence for this process having a learning benefit however is equivocal. Anecdotally, one would assume that giving students another task that engages them with the content and presents model answers would only serve to solidify concepts and knowledge. It is likely however that students do this anyway when instructors return marked work and hand out a marking template.

Short Videos

Stanford University lecturer Peter Norvig claimed that his decision to use short videos in MOOCs was inspired by the approach taken by the Khan Academy in keeping video length to 10 minutes or less. The argument was made that 10-15 minutes fits into the optimal time that students can maintain attention. However, on examination, the research that supports this assertion is particularly week. Also, Salman Khan has explained that the reason his videos are 10 minutes or less was because this was a limit imposed by YouTube when he first started uploading and not because of any pedagogical insight. There may be a benefit to short format videos, however more research will be necessary to provide evidence to support that notion.

Online Forums and Peer Support

Finally, there is the importance of the online forum to learning. There is little evidence to suggest that online forum discussions necessarily promotes learning in its own right. This is actually also the case for face-to-face discussions as well. What forums do achieve is that they facilitate the mechanics of a course by providing a means of unblocking students who are struggling either with content or some other aspect of the course. Online forums also can provide a knowledge base which accumulates over time. They are not currently being used in this fashion however and so this is something that MOOC providers could review in terms of the functionality offered by their discussion forums.

It is possible that when a course is constructed in a way to support group collaboration, online forums provide a mechanism for new knowledge to be created. This is the tenant of so-called “connectivist” style MOOCs, however again this awaits research to support this assertion.

So, are MOOCs an effective way to learn?

Overall, the evidence is that there is no reason to believe that MOOCs provide any less a valid learning experience than face-to-face courses. In many ways, they are simply a restatement of online learning environments which are optimised for large class sizes and modes of learning suited to todays digital milieu. When used for students enrolled in a university degree, they are usually combined with on-campus learning opportunities in a “flipped-classroom” style of presentation which brings the advantages of both environments.

What is exciting about the MOOC environment is that it will provide a rich opportunity to gather data that will tell us what does and doesn’t work and how students learn most effectively in as engaging an environment as can be provided. This will almost certainly mean that the current MOOC format will evolve rapidly over time as it is driven by this research supported by real data.

The review paper on which this article is based is available on First Monday

Join the conversation

12 Comments sorted by

  1. Jeremy Hall

    PhD student

    "The underlying subtext of why people believe that online teaching is less effective has not been extensively examined but certainly one would assume that it is related to a fear of technology..."

    I would guess part of the reluctance comes from the belief that students (and academics) shouldn't miss out on the "real" personal interaction that comes with a face to face course. For better or worse I imagine this attitude is becoming old-fashioned... but it might disadvantage those who value the interpersonal experience as well as the learning.

    1. Colin McIvor

      Knowledge Management Aficiando

      In reply to Jeremy Hall

      having been to one of londons older and more prestigious universities i can tell you I sat in front of a "video" during lectures that you could neither pause or rewind or in many cases speed up. There were some great lecturers there but there were also some to who personal interaction was seen as an annoyance getting in the way of their pure research ( and there is nothing wrong with pure research - just a lot of people involved in pure research shouldnt teach) - i think what we need is to give the lecturers and students involved a whole range of new skills in how to consume and provide the content.

  2. J O'Rourke

    Community Worker and Student

    Though i wouldn't go so far as to say it was "particularly week", i don't think this sort of analysis is nearly far ranging enough. For instance, it is surely likely that certain disciplines are going to be more suited to the online environment than others. From my experience, philosophy subjects, for example, are not terribly suited to the online format at all.

    I also think it's wrong to assume the belief that online teaching is inferior is merely a product of a fear of technology. A number of…

    Read more
  3. Dr. Brendan Moloney

    logged in via LinkedIn

    A great article! I think David's description of the great benefits of MOOCs are insightful and relevant to contemporary discussions on education. There is no doubt that the insights educators can gain about their students is a phenomenally beneficial feature of MOOCs (and online learning, generally).

    An area where there seems to be resistance to online learning comes from those suspicious of teaching with technology.

    Undoubtedly, teaching face-to-face offers a range of psychological benefits for lecturers that are not easily transferred to online work. It is not easy to convince people who enjoy the real life nature of teaching to sit in front of a screen to transmit education.

    As technology evolves (holograms, anybody?), it is likely that technology may pick up educators of the 'face-to-face' bent.

    Yet, as David notes, we are in a disruptive phase and for many, many students and academics the result will be better learning.

  4. Gavin Moodie
    Gavin Moodie is a Friend of The Conversation.

    Adjunct professor at RMIT University

    Thanx for this, which is valuable.

    I agree that the evidence that 10-15 minutes is the optimal time that students can maintain attention is particularly weak. (As other posters have noted, the article has *week*.) Bligh (1972: 73, alternatively 2000: 53) has a nice graph illustrating it, but again the evidence seems weak.

    I think the full paper should read:

    'A difficulty with the analysis of MOOC structure and its pedagogical foundations is the question of how *different* a MOOC is to…

    Read more
  5. Irene Ogrizek

    logged in via Facebook

    "The underlying subtext of why people believe that online teaching is less effective has not been extensively examined but certainly one would assume that it is related to a fear of technology generally which includes a fear of being replaced by that technology."

    There are a lot of statements being made by pundits, particularly in America, about how online courses will replace professors, or at least professors in the traditional sense of the term. Edx is saying it has a computer grading system…

    Read more
  6. Craig Watkins

    logged in via Facebook


    Congratulations on a great little article. Lots has been written on the subject by others and we will see much more yet before the dust settles (a few years away I guess). However, I feel that your article has a balance that is perhaps a rare find. Indeed, I can point to recent articles by supposedly informed people on The Conversation that have left me cold with respect to objective and rational analysis. I suspect that your rationality stems from the practical nature of your technical…

    Read more
  7. Dennis Alexander

    logged in via LinkedIn

    A useful contribution. Forgive my being a little dubious about your final comment: "This will almost certainly mean that the current MOOC format will evolve rapidly over time as it is driven by this research supported by real data." Bloom, Biggs, Knowles and Kolb have been around for thirty or more years with precious little take up by the Higher Ed teaching community until relatively recently, and some might say then only grudgingly. It might be a trifle optimistic to think that the best pedagogical aspects of MOOCs will pervade and persevere throughout the arena rapidly or persistently. There is always the chance that cheap and easy will crowd out standards and quality once the business financials and credentialling aspects get rolling. I don't want to be cynical, but history is not on the side of the angels in higher education.

  8. James Hill

    Industrial Designer

    Some anecdotal evidence from a former student's perspective.
    I chose to study geology for the last two years of high school precisely because there was no teacher and mainly because as a young adult I was well and truly sick of being treated as a school "child".
    The drawback was that without the face to face experience of a teacher I was deemed unable to study geology at the level that would at that time guarantee a commonwealth scholarship.
    Nevertheless the textbook proved adequate to the task…

    Read more
    1. J O'Rourke

      Community Worker and Student

      In reply to James Hill

      In fact you can take the origins of the debate back even further in the Western tradition, to Socrates, who refused to write anything down because it would only be a copy, an inadequate expression of the living logos, and Plato, who thankfully did write stuff down so we actually know about the logos at all.

  9. Lee Emmett

    Guest House Manager

    I admit I'm a newcomer to the idea of Massive Open Online Courses, so this article was very informative. An observation though: if a course is delivered to thousands, perhaps hundreds of thousands of students each year, then how is it evaluated: in dollar returns to the university? or in performance quality of students' responses? How would a professor get to know exactly who responded to an exam if it's done in the privacy of a home - would it be possible for a paid professional to substitue for an undergraduate?