Sections

Services

Information

UK United Kingdom

Proving knowledge by degrees: MOOCs and the challenge of assessment

Massive Open Online Courses (MOOCs) have certainly got higher education folks talking. These free online courses, often from prestigious universities, have prompted one obvious question: why should students…

Traditionally, students took exams on site at a university. But how does assessment work with online courses? Exam image from www.shutterstock.com

Massive Open Online Courses (MOOCs) have certainly got higher education folks talking. These free online courses, often from prestigious universities, have prompted one obvious question: why should students pay to go to university when they can get quality courses for free?

The short answer is that universities don’t just teach: they give credentials. Universities have long been the gatekeepers to further study and professional careers. Degrees, grades and exam results – all derived from assessment tasks – are signals of knowledge, and past and predicted performance.

So far, I have found many MOOCs to be pretty disappointing. Information is delivered in traditional ways – a 50 minute video of a lecture is still a 50 minute lecture.

Some are great learning experiences; many are not. Discussions mean reading and typing on screen. And the assessment – the credentialing piece of the puzzle – is often reduced to automated quizzes to cope with scale.

But innovative MOOCs are now testing new ways of interacting, assessing, and providing credentials.

Online assessment

The primary question for many in the online education space is: how do we know the person performing assessment online is the person who receives the credential? And how can you prevent plagiarism or cheating online?

In the offline world, we can do this relatively easily with supervised testing – the kind universities have in big exam halls at the end of term. It’s pretty good, but not watertight. But when assessment is online there are many more loopholes.

Increasingly there are new innovative ways to prevent cheating during online exams and assessment, including online proctoring of exams where humans monitor the student taking an online test using web cams; browser lockdowns to prevent “googling” for answers; keystroke pattern recognition on a computer keyboard; and plagiarism detection software.

Offering online proctored exams, like ProctorU is one way of getting around the MOOC cheating conundrum. ProctorU

There are also test centres where students can go to do a face-to-face exam for an online course — some MOOCs now provide credit if the student undertakes a supervised exam at a testing centre.

Evidence and assessment

As we go down the online path, innovation is not only needed for credentials but assessment too. Of course, there are many different forms of assessment – some more adaptable to online environments than others. And some are better at conveying a better picture of achievement.

For example, it’s a good idea to test a student pharmacist’s knowledge of drugs and their interactions, and probably an even better idea to assess their abilities to apply that knowledge, engaging with patients under pressure in a real or simulated pharmacy.

The latter is more “authentic” because it more closely replicates what happens in the profession — but it’s more expensive and challenging to do online and offline.

Perhaps instead of focusing on how we test students, a more purposeful question might be: presuming we know what outcomes we need students to achieve, and at what standard, what evidence will enable us to judge that this student is ready to graduate? In other words, assessment tasks are opportunities for students to create evidence of learning achievements in an array of formats.

Portfolios are one option and can include selected learning evidence from a range of sources, including reports, montages, case studies and analyses, supervised and take home tests, individual and group problem-solving tasks, placement reports and so on. Online portfolios let owners curate the evidence and share selected pieces with particular audiences (such as assessors and potential employers).

New digital badging technologies can be added to portfolios to enable peers, mentors and teachers to give credit for learning achievements (a bit like LinkedIn endorsements but based on actual artifacts rather than reputation). Good student portfolios should train students in the habit of matching assertions about their abilities with evidence.

An even more authentic (and authenticated) assessment option is the oral defense – on- or offline. Just like a job interview, this requires the candidate to field unforeseen questions and scenarios, and demonstrate their emotional intelligence as a future simpatico professional.

After all, this is often how new employees are recruited. We might do well to design some of our assessing and credentialing “gates” along similar lines.

New ways in difficult times

Money is getting too tight to mention in higher education. A large chunk of every dollar spent on teaching is on assessment processes – creating exams, printing papers, supervising, collecting, marking, moderating and uploading results.

Re-imagining assessment tasks to be more authentic, authenticated and cost effective has the potential to change the heart of the enterprise. That’s a great challenge in the online space, but MOOCs are already pioneers, and it’s exciting to be part of it.

Articles also by This Author

Sign in to Favourite

Join the conversation

15 Comments sorted by

  1. Jeremy Williams

    Professor & Director, Asia Pacific Centre for Sustainable Enterprise, Griffith Business School at Griffith University

    An excellent piece that should be essential reading for anyone remotely connected with assessment in higher education.

    Perhaps when contemplating assessment design, an important question for academics to ask themselves would be whether the activity in question will lead to something that is 'curatable' as evidence of learning.

    report
  2. Luftmensch

    logged in via Twitter

    I have three qualifications from an Australian university. One undergraduate and two post graduate. I have found two MOOCS - Coursera and Edx to be outstanding. 'Introduction to International Criminal Law' through Case Western Reserve on Coursera was an outstanding course. The lectures were dynamic with studio quality filming. In lecture video quizzes and quality assessments with all material being highly current. 'Justice,' a course on ethical reasoning is run through Harvardx with the online MOOC…

    Read more
  3. Craig Watkins

    logged in via Facebook

    A riged emphasis on assessment of each individual course perhaps makes little sense for MOOCs. Instead more emphasis on "terminal" course assessment may be sensible. Students are indeed only cheating themselves if they attempt to find short-cuts on the way through. The MOOC environment should rightly not worry about such situations as much as the traditional university setting.

    report
  4. Michael Foster

    logged in via Twitter

    I guess we are covering a couple of items in here that could each have had their own article.

    1) How do you prove that you've learned the material?

    I'm finding a new site called Accredible (http://www.accredible.com/) great for showing your portfolio. You add in your work, notes, results, my timesheets and just about anything else you find relevant including feedback from peer scoring to social network or forum involvement. Your peers or other professionals can then endorse you.

    After doing…

    Read more
  5. Gavin Moodie
    Gavin Moodie is a Friend of The Conversation.

    Adjunct professor at RMIT University

    In the old days exams were 'closed book', ie, one wasn't allowed to consult any book aside from log tables and a dictionary. Perhaps some are still closed book, but most are open book, reflecting more closely the realities of life. So why ban googling in exams? I would, however, block emailing questions and answers.

    Even if the interwebs are banned in invigilated exams, invigilated exams requiring text production should be sat on computers. This would not only reflect more closely the realities of contemporary text production and far simplify the distribution of exam questions and answers, it would also make answers far easier to read.

    report
  6. Peter Hindrup

    consultant

    At high school, 60 odd years ago the deputy head/teacher, told us he was quite happy to allow everybody to bring their books into an exam. He said that the same people would get through, as the assessment would simply be more rigorous to counter the fact of having the information to hand.
    I once convinced the same teacher that all exams ought to be without warning. You walked into the classroom, were told to leave your books at the door, and the exam was on.
    The marks generally, were so woeful that they had to rerun the exam.

    report
  7. Dennis Alexander

    logged in via LinkedIn

    If one is working with Learning Outcomes, whether expressed as: competencies; SKA clusters; or things students will know and/or be able to do as a consequence of the course of study, the design of the scope and sequence of learning experiences is based on the formative and/or summative assessment tasks that indicate whetehr and/or to what degree those learning outcomes have been achieved. A really useful series of webinars was held on the topic led by a team from Scotland last year with much of it available from http://www.transformingassessment.com/ . The really useful thing about this is that the online environment, to some extent, has now become the leader in assessment of learning and benefits are flowing into the face-to-face environment.

    report
  8. Stephen McCormick

    Research Fellow (Mathematics) at University of New England

    Do you know off-hand of any MOOC courses that use testing centres to offer credentials?

    I've thought for a long while - before I'd even heard of MOOCs - that people should be able to educate themselves and then be tested on the requirements for the award of a degree. Particularly so those from low socio-economic backgrounds can try to earn degrees. At the moment it is incredibly difficult for some people to compete for positions in a course, or commit to scheduled classes; however, they could quite easily learn everything a graduate has (especially with the aid of MOOCs) without ever stepping into a classroom.

    report
    1. Luftmensch

      logged in via Twitter

      In reply to Stephen McCormick

      Hi Stephen,

      University of New England is offering free online education with free self assessment. If you choose to try for credit you can pay for the challenge exam across a number of assessment centers. The exam costs $495. You can also pay per hour one on one video tutorials and group tutorials.

      You can check it out below.

      https://www.uneopen.com/app/about/a_id/17/session/L3RpbWUvMTM3MTU5NTc5NC9zaWQva196NUI0dGw%3D

      report
    2. Stephen McCormick

      Research Fellow (Mathematics) at University of New England

      In reply to Luftmensch

      Thanks for the info. I didn't realise anybody was doing that yet.

      I really can't understand why they are charging so much though. $500 to sit a single exam seems to be against the spirit of it though. I would bet the person marking the exam isn't paid more than $20 for it; even after taking admin and a profit into account, I can't see how they can justify more than $150.

      And $150 for a video conferencing tutor for an hour? How the hell do I get a gig like that?

      report
    3. Michael Foster

      logged in via Twitter

      In reply to Stephen McCormick

      Here here. $500 is ridiculous. That's the amount of money that is quickly making the university sector a little bit irrelevant.

      report
    4. Luftmensch

      logged in via Twitter

      In reply to Stephen McCormick

      Ha! Ha! I thought the same thing about video conferencing tutorials.

      I'm not sure how they came to the decision about the cost of the exam. To study a single subject face to face in an enrolled undergraduate is between approx $800 and $1,400 dollars depending on the band level. This is the same whether you take the course on campus or via distance education. Some universities have foundation studies where if you didn't get the required TER, you can still take a course in the degree program as…

      Read more
    5. Stephen McCormick

      Research Fellow (Mathematics) at University of New England

      In reply to Luftmensch

      They offer the exams during the regular exam periods, so it's unlikely that the university will incur any real costs in having a few extra students sit in their exam halls.

      I won't even say how little I get paid to mark a single exam...

      Oh well, one day.

      report
    6. David Thompson

      Marketing Research

      In reply to Luftmensch

      $495 for a single one semester undergrad course is gouging. It's almost the same as the HECS cost of that course on campus and face-to-face

      report
  9. Trish McCluskey

    logged in via Twitter

    An interesting piece here http://bit.ly/194yb0r on what constitutes cheating in a default digital world. The most interesting and challenging assessment I have encountered in the many MOOCs I have dipped into over the last year was the Uni Edinburgh e-learning and digital cultures MOOC (EDCMOOC) which required participants to design a digital artefact to be experienced and assessed on the web. The artefact needed to demonstrate the themes encountered during the course and contain a mixture of text, image, sound and video links created in a digital environment of choice. This task was a couple of suburbs outside my comfort zone but the peer support and learning from other participantsin around the globe through the medium of social media was phenomenal. My point is that utilizing multiliteracies and media rather than traditional text based assessment has a lot to offer in an assessment for learning model !

    report