The recent decision to ban multiple-choice questions at an Australian university has sparked debate about the purpose of assessment in higher education. While there are many problems with the ways in which multiple-choice questions are used in universities, the problems with the inauthentic nature of multiple-choice questions could similarly be levelled at most forms of assessment.
Assessment has traditionally been seen as a way of comparing the performance of students during and at the end of semester. While there are criticisms of this way of looking at assessment, what is actually being measured in this process is another big question.
Assessment of recollection
Traditionally, assessment has been conducted in higher education to test whether students can recall content. While a base level of factual information is needed before moving on to types of higher-level thinking required of experts and professionals, remembering is not enough.
As I have discussed previously, the ubiquitous availability of information on our smart phones and across our 3G networks means just remembering something isn’t enough anymore. We need to know how to evaluate the information.
If we are really going to be honest about this, unless graduates are constantly using the information, the majority of people usually remember only isolated facts from their undergraduate degrees several years later. This was best exemplified by Father Guido Sarducci’s humorous notion of the Five-Minute University. This is where, for example, you could be taught in five minutes everything you will remember about economics five years after graduation.
Assessment of thinking
A better aim for higher education assessment is to attempt to get at students’ thinking processes. It is possible to get some sense of how students arrived at an answer by asking them to provide a written response rather than allow them to pick (possibly at random or via semi-educated guess) from a set of pre-defined options. The same logic is at play here as when students are asked to show their working as part of a maths problem.
No matter how much students are asked to write, one thing remains consistent: the best that assessors can do is hope students are thinking about issues and concepts in appropriate ways. It is not possible to measure thinking, so no matter what method is used, some element of judgement by the assessor is required.
Moving away from multiple-choice questions therefore does not necessarily mean that it is easier to tell what students are thinking.
Assessment for learning
A more recent notion is the idea that assessment can be for learning rather than just conducting assessment of learning. As discussed elsewhere, the testing effect provides strong evidence for the enhancement of learning through exams. This is based on the same principle as flash cards - that we put what we think we know and remember through some sort of test to assist with encoding it into long-term memory.
Broadly, the notion that learning is enhanced through the testing of knowledge makes intuitive sense in practice settings and is supported by laboratory-based research. Much of the evidence for the testing effect has been gathered through the use of multiple-choice questions so it would appear that this format could be good for learning.
Assessment of student becoming
When considering the purpose of a university education at the most fundamental level, it isn’t just about remembering or thinking. The ultimate aim of university education as it has been historically conceived is a process of becoming.
Students come to university so that they can become lawyers, architects, historians and scholars with all the associated cognitive skills. While this process of becoming is at the core of the purpose of university education, it is nonetheless difficult to assess. The very definition of becoming is about subjective experience and not an objective reality that can be easily quantified in some way.
This notion of becoming is particularly hard to get at with a bank of multiple-choice questions. That may have led in part to their continued derision as an assessment approach.
What is really being measured?
Assessment tasks completed by students are not precise psychometric tools that can give teachers deep insights into the ways that students think or who they are becoming. At best, assessment tasks can be designed to give a teacher a reasonable chance of using their expert judgement to determine whether students have met the intended learning outcomes.
In order for this judgement to become more valid and reliable, it is not just the questions asked of students but the curriculum as a whole that needs to be considered. If a teacher is relying on a multiple-choice exam at the end of semester to give some inkling about what their students are thinking and who they are becoming, there is something fundamentally wrong with the way the entire subject/unit is designed.
A well-designed subject will give a teacher many opportunities to observe the progress of their students. This can be challenging with very large groups of students but it can be done through the use of effective learning design.
Ultimately, prohibiting one type of exam question is only dealing with a symptom. Until teachers move away from the notion that university education is about remembering things and start designing entire subjects and degree programs that address thinking and becoming, the problems observed with the types of assessment used will persist.