A conceptual leap lies between schools that test kids in order to measure their performance and schools that assess kids to see how teachers are performing, says Professor John Hattie, Director of the Melbourne Education Research Institute at the University of Melbourne’s Graduate School of Education. The why and how of making that leap is the topic of a public lecture that Professor Hattie will give tonight at the University of Melbourne. He speaks here about how re-thinking measurements and assessments can drastically improve education and school retention rates.
Professor John Hattie, Director of the Melbourne Education Research Institute at the University of Melbourne’s Graduate School of Education
On the cusp of an incredible breakthrough
The measurement community is on the cusp of making an incredible breakthrough. We’re moving away from the old, traditional assessments that we’ve been doing to measure kids, to looking much more at the processes they’ve been using and they way they collaborate, they way they interact - with the intent of informing teachers. It’s a whole switch from assessing kids to assessment for teachers.
What we’ve been obsessed with in the last 20 or so years is assessment of teachers, and that has not been very successful and it’s had very little impact. But if we turn around and say the fundamental role of assessment in schools is feedback for teachers: about who they teach well, how they teach well, and about what [parts of the curriculum].
Over the last 12 years I’ve been developing the New Zealand system which is based on this. It’s voluntary in New Zealand but the uptake is 80% plus [among schools] - because it’s telling the teachers who they taught well, how they taught well, and about what. It’s not about telling the kids what they know and don’t know. Assessment is a way of making learning visible to the teachers.
There’s nothing wrong with measuring performance as we’ve traditionally measured it, but to add to that we should start to measure the kind of errors kids make, the kind of processes they use when they do maths or reading or whatever - and it’s not just a total score.
Immediate, continual flow of information about performance
Here’s a classic example: about ten years ago the computer scoring of essays made a major breakthrough and we now can measure content. One of the things we did is we took the [automated] feedback from the computers and we took the feedback that teachers wrote on the essays, gave it to kids, gave it to teachers - blind - and said which do you prefer? It was overwhelming: they preferred the computer feedback - it was much more content rich; it was more specific; there was more of it, and it also told them where to go next.
One of the things that we should be doing is letting kids write essays straight into the essay-scoring machine: they get the results back instantaneously, and they improve their essays. But teachers are very resistant to that, because they believe that they are the ones who should be given the judgement [role].
The reason that computer games are so successful with kids is that they make the criteria for success very clear before they start: you know when you’re staring a game what you have to do, and you also know your prior achievement. One of those things we’re not particularly good at in the classroom: we have had this view that if you wait until the end we’ll test you and we’ll tell you how well you’ve done.
That’s not very exciting and challenging for a lot of kids. There’s a lot of ??? schools where the answer to [kids’] questions [about what happens] when they finish is that they’re going to get more, not, “How well did you go?”
It’s using assessment to make success very clear to the kids. [In the outdated system] we make kids go to the end of the year, or the end of a series of lessons, for assessment, and then they find out what they can’t do. So there’s been nought commitment to getting there.
One thing it [the new approach] does, is it shows teachers that about 60% of what they were going to teach, the kids knew already. That’s typical. Teachers have got to learn how to work out when kids learn, what they don’t learn, where they are on the continuum and from that you have to modify your behaviour.
The system I came from [NZ] is cumulative over the last three years of schooling - there’s a incredible amount of feedback for teachers, and the teachers modify what they do. The statistic I like is that in the last three years of doing this, we’ve had a 24% increase in the number of year 12 students. That’s just a phenomenal increase, and it happened because the success is very clear to the kids.
It’s like if I said to you, “Go out and watch an Australian Rules football game and then in ten minutes time you’re going to go out and play it, but I’m not going to tell you and you’re not going to see how they get points: there’d be chaos. That’s what it’s like for many kids. They don’t know what it means to get success; they don’t know when they get there. Typically what they do is wait for the test.
If you take music, art, phys ed, they’re a lot better because the standards of performance are usually very clear to kids before they start.
Music is one of my pet loves with this, because, particularly in primary school, a lot of our teachers are not very skilled in music. So you can use the assessment routines to tell teachers what the kids know and what they don’t know relative to the curriculum and what they should be doing. You’re not assessing the teachers, but you’re providing assessment to the teachers so that they can reframe what they’re doing - whether it’s about tempo, or about rhythm, whatever it is they’re trying to do. So we’ve been using assessment that way to inform the teachers, and we’ve had a success with it.
There are constraints but it’s also a mental attitude. The theme I push all the time is that your fundamental role as a teacher is to evaluate your impact and if you think of your job as that, then you can see how assessment can feed that evaluation. Whereas often teachers think that they’re fundamental role of assessment is to evaluate kids.
The role of parents: high expectations with a tolerance for error
There’s no question that parents can have a dramatic role but the biggest role of the parents is in expectations. If you have high expectations, the kid is more likely to achieve no matter what happens at the school. One of the things that bothers me is that we did a five year study of the lowest socio-economic schools in New Zealand, and we interviewed every parent when their kid came into year 1, asking them, "What do you want your kid to do when they leave school?” All of them said some kind of tertiary education. We interviewed them again when they left primary school at ages 11 and 12, and almost every single parent said, “Get a job.” We had failed them. You cannot tell me that in a sample size of around three or four thousand kids, even if they are the lowest SES, none of them can go on to tertiary. One of the problems we have in our schools is that we have very low expectations of what the kids can do. One of the things we found in the New Zealand study is that teachers dramatically underestimated what kids can do.
And, hey, parents, if you agree to that, then you’re part of the problem. There is no reason you can’t have high expectations. The job of schools is not to meed the needs of kids. The job of schools is not to help kids reach their potential. The job of schools is to help kids realise and exceed what they think is their potential. It’s the same with parents. If you continue to have high expectations for your kids no matter what environment they’re in then that’s the best effect you’re going to have.
Obviously if you have the resources and where-with-all to help them realise that - that helps, but you know that’s dispersed relative to the property values. But if you go into the low SES schools and have high expectations, they’re more likely to be realised. That has a dramatic role.
One of the things a parent should never, ever say is “Do your best,” because whatever a kid does is their best and it’s not always good enough. I say to my kids all the time, “Sometimes your best isn’t good enough.” That’s the purpose of education: to do better than your best.
Accepting low or non-challenging situations is hopeless, so as a parent you encourage the kid to take challenges, take risks. The hardest thing is that you have to have a safety net when they make errors. It'a a gold point of learning, but unfortunately what happens in so many of our schools, and particularly in Australia, is that we privilege ability and we denigrate effort. All of us are going to succeed by putting in effort, regardless of our ability. So many parents and so many schools talk about ability, ability, ability: “Oh, they don’t have the ability so they can’t do that.” It’s effort.
But that safety net’s so important. So parent, allow your kids to make errors - they’re a learning opportunity. This is not the Tiger Mum: high expectations and no toleration of error. That’s a way to drive kids out of education. And parents when you make errors that’s a learning opportunity, too.
Bringing the measurement revolution into university education
It would be great if we could do it with university staff. Typically what we do, particularly with undergraduates, is we give the students lot of content and then we give them the odd assignment, but we don’t use that information to feedback to us and then make us change what we do. Sometimes it’s because we have classes of a thousand and it’s tougher to do that - but it’s been implemented. When I was in Auckland it was implemented in engineering and those kind of subjects first. They were the best at worrying about they would then change in light of what the students knew and didn’t know, and how they would then change their instruction.
What they did in engineering and in computer science was to get the students to create their own assessments and administer them to each other, and to give reports back on the quality of the assessments they write. The teachers then monitor that. When you ask a question you reveal a hell of a lot more than when you give an answer. It’s not trivial being able to come up with the right question. In many cases you don’t know where the hell the speaker’s going to go and you have to ask some open questions to get them to open up, but not too open.
Typically in an assignment they have to ask ten questions, put it on PeerWise, other students must do 10 answers, and they get the responses back. If everyone gets 100% right, they set them too easy - they misunderstood - and vice versa: they had to understand the content. But here’s the killer: they had to understand what misinformation others are going to have. We’ve done all kinds of randomised trials and the learning in that is dramatic for the student, but the biggest beneficiary is the teacher. They can see what the students are asking about, what they’re not asking about - relative to the curriculum - and what they know and don’t know. It’s putting assessment back in the hands of what it tells the teacher.
We’ve been playing with online video stuff and putting it out for iPhone, but all that’s just the medium. This is about the purpose.
The most exciting one [assessment method] that’s happening here in Melbourne is with Patrick Griffin measuring students’ collaboration. He gets a student here in Melbourne, a student in Singapore, and a student in Finland to go onto the system and they have to collaborate to solve problems - and it’s about their degree of collaboration.
You have to think quite differently when you construct those kinds of assessments, because you have to come up with a way that will lead to rich information coming back about what each kid does relative to the others.
Comments welcome below.