tag:theconversation.com,2011:/au/topics/pisa-scores-34195/articlesPISA scores – The Conversation2023-12-13T13:31:09Ztag:theconversation.com,2011:article/2196462023-12-13T13:31:09Z2023-12-13T13:31:09ZWales’s Pisa school test results have declined – but it’s not a true reflection of an education system<p>Every three years, an early Christmas gift arrives for the global education community from the Organisation for Economic Co-operation and Development (OECD). The Programme for International Student Assessments (<a href="https://www.oecd.org/pisa/">Pisa</a>) is an international test in which 15 year olds are tested on their knowledge and skills. </p>
<p>It relegates those far below the Pisa top ten as poor performers in desperate need of improvement, which this time includes Wales.</p>
<p>The Pisa scores for participating education systems around the world are unquestionably significant. But since its inception in 2000, Pisa has sparked much debate, especially among experts and policymakers, with many viewing it as a <a href="https://ffteducationdatalab.org.uk/2019/11/is-pisa-fundamentally-flawed-because-of-the-scaling-methodology-used/">flawed</a> assessment of educational outcomes. In <a href="https://www.oecd.org/pisa/Combined_Executive_Summaries_PISA_2018.pdf">2018</a>, around 600,000 students took part in the standardised Pisa tests, which measured their performance in maths, science and reading, and also looked at wellbeing.</p>
<p>Predictably, the 2023 Pisa results captured the negative impact of COVID on learners and learning, with some downward trends in performance visible across the data set. </p>
<p>The results signalled mixed fortunes for the <a href="https://www.education.ox.ac.uk/pisa-results-reveal-students-in-the-uk-have-higher-than-average-levels-of-maths-reading-and-science/">UK</a>. The <a href="https://www.bbc.co.uk/news/uk-wales-67616536">BBC headline</a>, reporting the results starkly stated: “Wales slumps to worst school test results.” Such sweeping statements are by now an anticipated byproduct of Pisa that ignore how the tests are often highly contested and <a href="https://revisesociology.com/2020/05/15/the-pisa-global-education-tests-arguments-for-and-against/">controversial</a>. </p>
<h2>Pisa in Wales</h2>
<p>Every three years, Pisa measures the ability of 15 year olds to use their reading, mathematics and science knowledge and skills to meet real-life challenges. In Wales, 2,568 pupils from 89 schools took a two-hour computer-based exercise. To put this in context, there are <a href="https://statswales.gov.wales/Catalogue/Education-and-Skills/Schools-and-Teachers/Schools-Census/Pupil-Level-Annual-School-Census/Pupils/pupils-by-yeargroup-sex">approximately</a> 33,000 pupils in Year 11 in 178 secondary schools and 27 middle schools in Wales.</p>
<p>The subsequent OECD report <a href="https://www.gov.wales/achievement-15-year-olds-program-international-student-assessment-pisa-national-report-2022">acknowledges</a> that “the sample for Wales, and for many other countries, did not meet some of the Pisa standards”. It is important to reflect on how a test taken by a sample of 15-year-old students, every three years for two hours, can possibly be a valid and reliable measure of a system’s performance even in a relative sense. </p>
<p>Pisa’s statistics show that Wales’s average score for mathematics in 2022 was significantly lower than the average across OECD countries. Wales’s average scores for mathematics, reading and science have all declined significantly since 2018. This was also the case, on average, across OECD countries for mathematics and reading. Although for science, the difference between the OECD average in 2022 was not significantly different to that in 2018. </p>
<p>It also noted that the gap in performance between pupils from the most disadvantaged backgrounds and the least disadvantaged backgrounds was smaller in Wales than it was on average across OECD countries for all subjects. </p>
<p>The important thing to observe is that Pisa deals in averages. In the latest results, those averages are derived from the 81 countries that took part, which is a huge range. The report notes a relative fall in Welsh performance against an aggregated average of OECD countries. </p>
<p>It then highlights that this decline was also the case on average across OECD countries for mathematics and reading. In other words, this is a trend. It also suggests that Wales has been more successful in closing the achievement gap between the most disadvantaged and least disadvantaged pupils than most other OECD countries. Yet this important indicator of success has been overshadowed by the blanket headlines of abject educational failure.</p>
<p>If all countries participating in Pisa now recover fully after COVID and improve their educational performance across the board, it is highly likely that Wales will “underperform” on Pisa yet again, whatever it does. If all countries in Pisa continue a steady trajectory of improvement, the country differentials will remain largely the same. Some countries may move up or down, but that movement will be marginal. </p>
<h2>A game of relatives</h2>
<p>There were no real surprises in the latest results. Countries like Singapore, Taiwan and Japan have retained their comparative advantage and will probably continue to do so, because Pisa is a game of relatives. The complexity and dynamic of any education system cannot (and should not) be at the mercy of a single measure of assessment however compelling or lucrative. </p>
<p>But what does that mean for Wales? First, it should encourage us to look at Pisa as one data set only and to not be obsessed by its findings. It is important to put Pisa in perspective by looking far beyond the simple headlines and delving into the detail of the report. </p>
<p>Second, it should be a reminder that Pisa is a snapshot of performance at a particular moment in time. It takes no account of the possibilities and potential of ongoing reform – deep contextual detail is not on its global radar. Hence, the danger is that every three years, Pisa fuels doubt, dissent and concern, when education systems need certainty, confidence and consensus about the reforms they are putting in place.</p>
<p>Third, no education system is perfect. Getting great Pisa scores is certainly no guarantee that the wellbeing and mental health of children and young people is not compromised or sidestepped along the way. The potential of human collateral damage in achieving high Pisa performance over two decades, unsurprisingly, does not feature in the OECD reports. </p>
<p>Wales has a choice, to either let this global compass direct its educational pathway, accepting that every three years it will derail and disrupt the reform agenda, or to hold its nerve. We cannot ignore Pisa, but we can put it in perspective and continue to focus on the learning and wellbeing of all children and young people in Wales. This is what matters most.</p><img src="https://counter.theconversation.com/content/219646/count.gif" alt="The Conversation" width="1" height="1" />
<p class="fine-print"><em><span>Alma Harris does not work for, consult, own shares in or receive funding from any company or organisation that would benefit from this article, and has disclosed no relevant affiliations beyond their academic appointment.</span></em></p>Pisa measures 15-year-olds in reading, maths and science every three years - but is that the best way to test an education system?Alma Harris, Professor of Leadership in Education, Cardiff Metropolitan UniversityLicensed as Creative Commons – attribution, no derivatives.tag:theconversation.com,2011:article/1784262022-03-09T19:09:11Z2022-03-09T19:09:11Z‘I just go to school with no food’ – why Australia must tackle child poverty to improve educational outcomes<figure><img src="https://images.theconversation.com/files/450596/original/file-20220308-44826-114p4ii.jpg?ixlib=rb-1.1.0&rect=0%2C0%2C5760%2C3828&q=45&auto=format&w=496&fit=clip" /><figcaption><span class="caption">
</span> <span class="attribution"><span class="source">Shutterstock</span></span></figcaption></figure><p>About <a href="https://povertyandinequality.acoss.org.au/wp-content/uploads/2020/02/Poverty-in-Australia-2020_Part-1_Overview.pdf">one in six children</a> in Australia live in poverty. These children generally have poorer educational outcomes than more advantaged children. <a href="https://www.cambridge.org/core/journals/journal-of-social-policy/article/who-excludes-young-peoples-experience-of-social-exclusion/F817D2FE8C0742EED5C38669853D680A">Our recently published research</a> shows students who live in poverty also experience more social exclusion at school than their more advantaged peers. </p>
<p>These findings suggests disadvantage at home carries over into disadvantage at school. </p>
<p>Interventions such as anti-bullying programs and increased funding for schools in disadvantaged communities can help. However, our analysis suggests there’s a bigger structural problem. To reduce educational disadvantage, action is needed to reduce child poverty, which has remained stubbornly high since the early 2000s.</p>
<p>In 1987, Prime Minister Bob Hawke famously <a href="https://www.youtube.com/watch?v=bx0IeQQ7WjI">pledged</a> to end child poverty by 1990. As a result of his government’s actions, child poverty initially <a href="https://onlinelibrary.wiley.com/doi/abs/10.1111/j.1475-4932.2000.tb00020.x">declined</a> before increasing again. Child poverty rates now are only <a href="https://povertyandinequality.acoss.org.au/wp-content/uploads/2020/02/Poverty-in-Australia-2020_Part-1_Overview.pdf">slightly lower</a> than in 1999.</p>
<p>In that time, child poverty has been largely absent from policy agendas. Failure to act on poverty will cripple the <a href="https://theconversation.com/life-chances-policy-must-respond-to-the-real-lives-of-young-people-27425">life chances</a> and productivity of future generations. </p>
<figure>
<iframe width="440" height="260" src="https://www.youtube.com/embed/bx0IeQQ7WjI?wmode=transparent&start=0" frameborder="0" allowfullscreen=""></iframe>
<figcaption><span class="caption">As prime minister, Bob Hawke put child poverty on the agenda with his pledge that no child would live in poverty by 1990.</span></figcaption>
</figure>
<hr>
<p>
<em>
<strong>
Read more:
<a href="https://theconversation.com/by-2030-no-australian-child-will-be-living-in-poverty-why-cant-we-promise-that-64166">By 2030, 'no Australian child will be living in poverty' – why can't we promise that?</a>
</strong>
</em>
</p>
<hr>
<h2>The high costs of social exclusion at school</h2>
<p><a href="https://www.cambridge.org/core/journals/journal-of-social-policy/article/who-excludes-young-peoples-experience-of-social-exclusion/F817D2FE8C0742EED5C38669853D680A#article">Our research</a> has looked at the schooling experiences of 3,535 students aged 13 to 14 in in every state and territory. </p>
<p>Children whose families lacked items most Australian households take for granted, such as cars, computers or holidays, were identified as experiencing family poverty. Children who reported lacking items that most children see as essential were identified as experiencing child deprivation. These items included clothes that allowed them to fit in with other children, and their family having money to send them on school camp. </p>
<p>The proportions living in family poverty or child deprivation were highest among children who experienced multiple forms of disadvantage. One in five children with a disability lived in poverty, as did one in three who had a caring responsibility for a family member. Over one in four Indigenous children and children with a language background other than English also lived in poverty. By comparison, this was the case for only one in eight children who were not part of a marginalised group. </p>
<p><div data-react-class="Tweet" data-react-props="{"tweetId":"1230682250229895168"}"></div></p>
<hr>
<p>
<em>
<strong>
Read more:
<a href="https://theconversation.com/one-quarter-of-australian-11-12-year-olds-dont-have-the-literacy-and-numeracy-skills-they-need-148912">One quarter of Australian 11-12 year olds don't have the literacy and numeracy skills they need</a>
</strong>
</em>
</p>
<hr>
<p>Teachers make great <a href="https://www.tandfonline.com/doi/full/10.1080/1359866X.2019.1602863">efforts to support</a> the education of disadvantaged students. Despite these efforts, children living in poverty have <a href="https://melbourneinstitute.unimelb.edu.au/__data/assets/pdf_file/0008/3522482/Breaking-Down-Barriers-Report-1-October-2020.pdf">lower school completion rates</a> and lower scores on national tests such as <a href="https://www.dss.gov.au/sites/default/files/documents/11_2017/sprp47_low_income_web.pdf">NAPLAN</a>. And our study shows the effects of poverty still permeate school classrooms and playgrounds. </p>
<p>In our study, we asked children how much they agreed with the statement: “At my school, there is a teacher or another adult: who really cares about me; who believes that I will be a success; who listens to me when I have something to say.” The children experiencing deprivation reported less support from their teachers. They also reported higher rates of bullying than non-deprived children. </p>
<p>These experiences were in turn associated with students reporting lower levels of life satisfaction. That’s an early indicator of <a href="https://journals.sagepub.com/doi/10.1177/0002716203260078">mental health problems</a> in youth and adulthood. </p>
<figure class="align-center ">
<img alt="Upset girl being comforted by teacher in school corridor" src="https://images.theconversation.com/files/450595/original/file-20220308-126102-1ctut5w.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=754&fit=clip" srcset="https://images.theconversation.com/files/450595/original/file-20220308-126102-1ctut5w.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=600&h=400&fit=crop&dpr=1 600w, https://images.theconversation.com/files/450595/original/file-20220308-126102-1ctut5w.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=600&h=400&fit=crop&dpr=2 1200w, https://images.theconversation.com/files/450595/original/file-20220308-126102-1ctut5w.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=600&h=400&fit=crop&dpr=3 1800w, https://images.theconversation.com/files/450595/original/file-20220308-126102-1ctut5w.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=754&h=503&fit=crop&dpr=1 754w, https://images.theconversation.com/files/450595/original/file-20220308-126102-1ctut5w.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=754&h=503&fit=crop&dpr=2 1508w, https://images.theconversation.com/files/450595/original/file-20220308-126102-1ctut5w.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=754&h=503&fit=crop&dpr=3 2262w" sizes="(min-width: 1466px) 754px, (max-width: 599px) 100vw, (min-width: 600px) 600px, 237px">
<figcaption>
<span class="caption">Children living in poverty report higher rates of bullying and lower levels of life satisfaction than their more advantaged peers.</span>
<span class="attribution"><span class="source">Shutterstock</span></span>
</figcaption>
</figure>
<hr>
<p>
<em>
<strong>
Read more:
<a href="https://theconversation.com/what-do-children-think-of-economic-inequality-we-did-an-experiment-to-find-out-163262">What do children think of economic inequality? We did an experiment to find out</a>
</strong>
</em>
</p>
<hr>
<h2>Children’s potential is being stifled</h2>
<p>The <a href="https://www.oecd.org/pisa/">Programme for International Student Assessment</a> (PISA) conducts comparable academic tests of 15-year-old students in all OECD countries. Gaps in test performance between the most socioeconomically advantaged and the most disadvantaged students in Australia have <a href="https://research.acer.edu.au/ozpisa/51/">hardly changed</a> since the surveys were launched in 2000. </p>
<p>The gaps for the most recent tests in 2018 represented around three years of education for reading, maths and science literacy. When students fall that far behind, it seriously blights their life chances.</p>
<p><a href="https://www.tandfonline.com/doi/pdf/10.1080/04250494.2019.1672502">Teachers recognise</a> that children living in poverty face many challenges that impact their learning and relationships. Children also talk about the challenges of poverty. <a href="https://www.researchgate.net/profile/Gerry-Redmond/publication/308894654_Material_Deprivation_and_Capability_Deprivation_in_the_Midst_of_Affluence_The_Case_of_Young_People_in_Australia/links/5af3a3d14585157136c9218f/Material-Deprivation-and-Capability-Deprivation-in-the-Midst-of-Affluence-The-Case-of-Young-People-in-Australia.pdf">One boy explained</a>: </p>
<blockquote>
<p>“My mum would take me to the op shop because I keep on splitting my pants when I kneel down but she can’t afford to buy me new pants. I don’t get pocket money and have to make my own lunch and sometimes I don’t even do that. I just go to school with no food.”</p>
</blockquote>
<p>That such experiences should be associated with poor educational outcomes is not surprising. What is surprising is how badly Australia’s education system is failing to achieve a key objective: to support all children to <a href="https://www.dese.gov.au/download/4816/alice-springs-mparntwe-education-declaration/7180/alice-springs-mparntwe-education-declaration/pdf">reach their full educational potential</a>. </p>
<p><div data-react-class="Tweet" data-react-props="{"tweetId":"1023746071963394048"}"></div></p>
<hr>
<p>
<em>
<strong>
Read more:
<a href="https://theconversation.com/already-badly-off-single-parents-went-dramatically-backwards-during-covid-they-are-raising-our-future-adults-157767">Already badly off, single parents went dramatically backwards during COVID. They are raising our future adults</a>
</strong>
</em>
</p>
<hr>
<h2>It’s time to focus again on child poverty</h2>
<p>Child poverty and children’s educational disadvantage require different solutions, but they are closely linked. The more poverty there is in Australia, the harder education systems and individual teachers have to work to compensate for its effect on student outcomes. </p>
<p>The <a href="https://theconversation.com/gonski-2-0-there-is-evidence-inclusive-schooling-will-help-those-left-behind-95934">Gonski 2.0</a> package of school funding reforms, launched in 2018, aims to at least partially address educational disadvantage. However, it is unlikely to break the poverty-educational outcomes nexus on its own. </p>
<p>The challenge that Hawke set 35 years ago, to end child poverty in Australia, needs to be taken up again. Both the Hawke government’s actions in the years following his pledge and the current Australian government’s responses to the COVID-19 pandemic show how this can be done. </p>
<p>After 1987, family payments were significantly increased and targeted to lower-income families. This increased support helped <a href="https://www.researchgate.net/publication/333915308_MAKING_A_DIFFERENCE_THE_IMPACT_OF_GOVERNMENT_POLICY_ON_CHILD_POVERTY_IN_AUSTRALIA_1982_TO_1997-98_Paper_Prepared_for_the_26th_General_Conference_of_the_International_Association_for_Research_in_Income">reduce child poverty</a>. </p>
<p>In 2020, in response to the growing COVID-19 emergency, the Morrison government introduced the JobKeeper payment and added the Coronavirus Supplement to the Jobseeker Allowance. Poverty rates <a href="https://povertyandinequality.acoss.org.au/wp-content/uploads/2022/03/Build-back-fairer-report-3_FINAL.pdf">declined</a>, at least temporarily, while these supports were in place. </p>
<p>Money does not solve all the problems of child disadvantage. But it <a href="https://link.springer.com/article/10.1007/s12187-020-09782-0">does matter</a>. </p>
<hr>
<p>
<em>
<strong>
Read more:
<a href="https://theconversation.com/young-australians-prospects-still-come-down-to-where-they-grow-up-102640">Young Australians' prospects still come down to where they grow up</a>
</strong>
</em>
</p>
<hr>
<p>The next Australian government could follow Hawke’s example and set targets to reduce child poverty. History (in Australia and <a href="https://cpag.org.uk/recent-history-uk-child-poverty">elsewhere</a>) suggests that action will follow and child poverty will fall. </p>
<p>Reducing poverty will have positive flow-on effects for children’s well-being, development and educational outcomes. It will also represent a major step towards Australia achieving the UN <a href="https://sdgs.un.org/goals/goal1">Sustainable Development Goal</a> of halving poverty rates of all men, women and children by 2030.</p><img src="https://counter.theconversation.com/content/178426/count.gif" alt="The Conversation" width="1" height="1" />
<p class="fine-print"><em><span>Gerry Redmond receives funding from the Australian Research Council.</span></em></p>Australia has a wide gap in educational outcomes between children in poverty and their better-off peers. A new study indicates why reducing child poverty is the best way to lift our educational game.Gerry Redmond, Professor, College of Business, Government & Law, Flinders UniversityLicensed as Creative Commons – attribution, no derivatives.tag:theconversation.com,2011:article/1720502021-11-18T22:22:27Z2021-11-18T22:22:27ZYes, Australia’s PISA test results may be slipping, but new findings show most students didn’t try very hard<figure><img src="https://images.theconversation.com/files/432575/original/file-20211118-27-1jh8wfp.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=496&fit=clip" /><figcaption><span class="caption">
</span> <span class="attribution"><a class="source" href="https://www.shutterstock.com/image-photo/group-high-school-students-doing-exam-704245237">Shutterstock</a></span></figcaption></figure><p>Most Australian students who took part in the last OECD Programme for International Student Assessment (PISA) said they would have invested more effort if the test had counted towards their school marks.</p>
<p>This is a finding from a <a href="https://research.acer.edu.au/snapshots/vol16/iss16/1/">recent report</a> issued by the Australian Council for Education Research. The data came from a questionnaire students filled out at the end of the two-hour PISA test in 2018. They were asked to rate how much effort they would have invested if they knew their results would count towards their school marks.</p>
<p>Some 73% of students indicated they would have put in more effort had that been the case.</p>
<p>While 56% of Australian students claimed to put in “high effort” in the PISA tests, this would have increased to 91% if the results were included in their school results.</p>
<p>We spend a lot of time focusing on debates about curriculum (what is being taught to students) and pedagogy (how it is being taught). Data from standardised tests such as PISA and NAPLAN are often used as evidence of declining standards, falling outcomes and failing teachers.</p>
<p><div data-react-class="Tweet" data-react-props="{"tweetId":"1204164407804084226"}"></div></p>
<p>But the above results show yet again that schooling is more complex than politicians like to advocate. Methods to lift standards such as going “<a href="https://www.afr.com/policy/health-and-education/alan-tudge-s-10-year-plan-to-get-schools-back-to-basics-20210413-p57ir9">back to the basics</a>” – as the then education minister, Dan Tehan, vowed to do after the last PISA results came out – or encouraging the “<a href="https://www.smh.com.au/national/yes-minister-you-can-entice-our-best-and-brightest-into-teaching-you-will-have-to-pay-them-more-20210415-p57jib.html">best and brightest</a>” to become teachers – a goal of the current education minister, Alan Tudge – are too simplistic for the real world.</p>
<h2>What is PISA?</h2>
<p>Every three years, PISA tests how 15-year-old students in dozens of countries apply <a href="https://theconversation.com/the-pisa-world-education-test-results-are-about-to-drop-is-australia-getting-worse-127011">reading, science, maths and other skills to real-life problems</a>.</p>
<p>PISA generates <a href="https://theconversation.com/pisa-results-dont-look-good-but-before-we-panic-lets-look-at-what-we-can-learn-from-the-latest-test-69470">much attention</a> from policymakers and the media. It is often used as a proxy for making judgements about the quality of teaching and learning in Australian schools.</p>
<hr>
<p>
<em>
<strong>
Read more:
<a href="https://theconversation.com/pisa-doesnt-define-education-quality-and-knee-jerk-policy-proposals-wont-fix-whatever-is-broken-128389">PISA doesn't define education quality, and knee-jerk policy proposals won't fix whatever is broken</a>
</strong>
</em>
</p>
<hr>
<p>But there are important questions regarding what exactly the PISA tests measure and how useful the results are for informing policymaking and education debates.</p>
<h2>Is it knowledge or effort?</h2>
<p>The ACER report showed levels of effort in PISA were higher for female students, those attending metropolitan schools, non-Indigenous students and students from backgrounds of relatively high socioeconomic advantage. </p>
<p>But, when averaged out, nearly half of Australian students who sat the 2018 PISA test admitted they did not try their best. </p>
<p>These results are comparable with the <a href="https://www.oecd-ilibrary.org/sites/04fd5153-en/index.html">OECD average</a> of 68% of students claiming they tried less on the PISA tests than they would if it counted towards their school grades. In contrast, students in the <a href="https://www.oecd.org/pisa/publications/pisa-2018-results.htm">highest-performing education systems</a> of Beijing, Shanghai, Jiangsu and Zhejiang (China) reported very high levels of effort. There could be several reasons why the same theory may be less applicable to these Chinese systems, such as them having a more strongly competitive academic culture. </p>
<hr>
<p>
<em>
<strong>
Read more:
<a href="https://theconversation.com/the-pisa-world-education-test-results-are-about-to-drop-is-australia-getting-worse-127011">The PISA world education test results are about to drop. Is Australia getting worse?</a>
</strong>
</em>
</p>
<hr>
<p>Educational psychologists <a href="https://www.sciencedirect.com/science/article/pii/S014019710800122X">in Australia</a> have long studied the links between motivation, <a href="https://www.tandfonline.com/doi/abs/10.1080/00461520.1991.9653133">self-efficacy</a> (students’ beliefs they can perform at the level they need to) and <a href="https://aps.onlinelibrary.wiley.com/doi/abs/10.1080/00050060701405584">academic achievement</a>.</p>
<p>For example, <a href="https://www.sciencedirect.com/science/article/pii/S0361476X99910159">expectancy–value theory</a>, to put it simply, suggests the lower the perceived value or usefulness of a task, the less motivated one potentially is to put in much effort.</p>
<figure class="align-center zoomable">
<a href="https://images.theconversation.com/files/432572/original/file-20211118-25-1sv82up.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=1000&fit=clip"><img alt="Student running up some stairs that has a door at the top opening up to the sky." src="https://images.theconversation.com/files/432572/original/file-20211118-25-1sv82up.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=754&fit=clip" srcset="https://images.theconversation.com/files/432572/original/file-20211118-25-1sv82up.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=600&h=400&fit=crop&dpr=1 600w, https://images.theconversation.com/files/432572/original/file-20211118-25-1sv82up.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=600&h=400&fit=crop&dpr=2 1200w, https://images.theconversation.com/files/432572/original/file-20211118-25-1sv82up.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=600&h=400&fit=crop&dpr=3 1800w, https://images.theconversation.com/files/432572/original/file-20211118-25-1sv82up.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=754&h=503&fit=crop&dpr=1 754w, https://images.theconversation.com/files/432572/original/file-20211118-25-1sv82up.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=754&h=503&fit=crop&dpr=2 1508w, https://images.theconversation.com/files/432572/original/file-20211118-25-1sv82up.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=754&h=503&fit=crop&dpr=3 2262w" sizes="(min-width: 1466px) 754px, (max-width: 599px) 100vw, (min-width: 600px) 600px, 237px"></a>
<figcaption>
<span class="caption">Motivation to do the task is determined by its perceived value.</span>
<span class="attribution"><a class="source" href="https://www.shutterstock.com/image-photo/female-high-school-student-running-on-571923439">Shutterstock</a></span>
</figcaption>
</figure>
<p>Perhaps one of the unintended side effects of assuring participating students that PISA is a low-stakes task — it does not count towards their school grades — is the potential for downward pressure on performance.</p>
<h2>The year 9 slump</h2>
<p>Another potential reason for the lack of motivation in students taking the PISA test is the well-documented slump in <a href="https://journals.sagepub.com/doi/abs/10.3102/00346543064002287">engagement and motivation</a> during the <a href="https://www.journals.uchicago.edu/doi/abs/10.1086/461740">middle years of schooling</a>.</p>
<p>NAPLAN data have consistently shown a pronounced drop in performance from year 7 to year 9, when students are 14–15 years old. For example, 9.1% of year 7 students didn’t <a href="https://search.informit.org/doi/abs/10.3316/INFORMIT.357583795641713">meet the national minimum standard</a> in the 2013 NAPLAN writing task. Two years later in the NAPLAN 2015 writing task, nearly twice as many (17.7%) year 9 students didn’t meet the minimum standard. </p>
<p>At the higher end of performance, the proportion of students above the national minimal standard dropped from 72.2% in 2013 to 59% in 2015.</p>
<p>The pattern is persistent. The results from the <a href="https://theconversation.com/naplan-results-show-year-3-students-perform-better-than-year-9-in-writing-and-its-a-worrying-trend-122541">year 9 NAPLAN writing task in 2019</a> clearly demonstrate a dramatic drop in performance. The percentage of students in year 9 meeting or exceeding the national minimum standard was 82.9%, compared to 95% of the same student cohort in the 2013 year 3 writing task.</p>
<p>Research has shown the middle years of schooling is a challenging time for many students. Their bodies and minds are changing rapidly, the demands of high school and their social lives become more complex, and the level of disengagement and disaffection with school rapidly escalates. </p>
<hr>
<p>
<em>
<strong>
Read more:
<a href="https://theconversation.com/the-missing-middle-puberty-is-a-critical-time-at-school-so-why-arent-we-investing-in-it-more-150071">The missing middle: puberty is a critical time at school, so why aren’t we investing in it more?</a>
</strong>
</em>
</p>
<hr>
<h2>What does this mean for school policy?</h2>
<p>Instead of policies such as going back to basics, student motivation and <a href="https://www.tandfonline.com/doi/abs/10.1080/13603116.2021.1956605">engagement</a> must be part of the education policy landscape.</p>
<p>This means paying closer attention to the lives, knowledges, experiences, hopes, fears, challenges and opportunities facing young people.</p>
<p>Educators and policymakers must consider complex factors of social, economic and educational disadvantage and advantage to meet the <a href="https://www.dese.gov.au/alice-springs-mparntwe-education-declaration/resources/alice-springs-mparntwe-education-declaration">Mparntwe Declaration</a> goals of educational excellence and equity. This includes the interplay of socioeconomics, location, culture and community, school resourcing and access for all young people to housing, health, economic and social stability, and quality schooling.</p><img src="https://counter.theconversation.com/content/172050/count.gif" alt="The Conversation" width="1" height="1" />
<p class="fine-print"><em><span>Stewart Riddle does not work for, consult, own shares in or receive funding from any company or organisation that would benefit from this article, and has disclosed no relevant affiliations beyond their academic appointment.</span></em></p>Motivation plays a large part in educational success. Of students who sat the PISA test in 2018, 73% indicated they would have put in more effort had the scores counted towards their school marks.Stewart Riddle, Associate Professor, School of Education, University of Southern QueenslandLicensed as Creative Commons – attribution, no derivatives.tag:theconversation.com,2011:article/1686502021-10-07T22:59:14Z2021-10-07T22:59:14ZChildren live online more than ever – we need better definitions of ‘good’ and ‘bad’ screen time<figure><img src="https://images.theconversation.com/files/425165/original/file-20211007-23-3a1d1v.jpg?ixlib=rb-1.1.0&rect=0%2C0%2C5463%2C3645&q=45&auto=format&w=496&fit=clip" /><figcaption><span class="caption">
</span> <span class="attribution"><span class="source">Shutterstock</span></span></figcaption></figure><p>The pandemic has fundamentally altered every part of our lives, not least the time we spend on digital devices. For young people in particular, the blurred line between recreational and educational screen time presents new challenges we are only beginning to appreciate. </p>
<p>Even before COVID, there were concerns about screen time for children. A <a href="https://www.health.govt.nz/your-health/healthy-living/food-activity-and-sleep/physical-activity/activity-levels-new-zealand">2019-20 survey</a> found four in five children were exceeding the current Ministry of Health recommendation of two hours’ recreational screen time a day. This was on top of screen time linked to learning. </p>
<p>With lockdowns and social restrictions now a new normal, it is increasingly difficult to disengage from screens. Children are growing up in a digital society, surrounded by a multitude of devices used for everything from social connection to learning and entertainment. </p>
<p>The boundaries between recreation, communication and learning are becoming less distinct. Screen time that may seem on the surface to be purely recreational can in reality be important for learning, supporting <a href="https://www.commonsensemedia.org/research/coping-with-covid19-how-young-people-use-digital-media-to-manage-their-mental-health">mental health</a> and driving awareness of important issues.</p>
<p>YouTube, for example, can be both entertaining and educational. It is increasingly used in classes to supplement teaching. But it is also used in other ways, including to drive social change, as German star Rezo demonstrated with a viral climate change video that prompted <a href="https://www.cogitatiopress.com/mediaandcommunication/article/view/2862">sweeping public reforms</a>. </p>
<p>Likewise the popular online game Minecraft has been shown to provide rich educational and social benefits. Even games like Roblox or Fortnite, where those benefits may be less apparent, still <a href="https://www.sciencedirect.com/science/article/abs/pii/S0747563211002706">provide opportunities</a> for rich social engagement and spaces for problem solving and experiential learning. </p>
<figure class="align-center ">
<img alt="moble phone screen showing Fortnite game" src="https://images.theconversation.com/files/425166/original/file-20211007-26-bvv8wb.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=754&fit=clip" srcset="https://images.theconversation.com/files/425166/original/file-20211007-26-bvv8wb.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=600&h=400&fit=crop&dpr=1 600w, https://images.theconversation.com/files/425166/original/file-20211007-26-bvv8wb.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=600&h=400&fit=crop&dpr=2 1200w, https://images.theconversation.com/files/425166/original/file-20211007-26-bvv8wb.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=600&h=400&fit=crop&dpr=3 1800w, https://images.theconversation.com/files/425166/original/file-20211007-26-bvv8wb.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=754&h=503&fit=crop&dpr=1 754w, https://images.theconversation.com/files/425166/original/file-20211007-26-bvv8wb.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=754&h=503&fit=crop&dpr=2 1508w, https://images.theconversation.com/files/425166/original/file-20211007-26-bvv8wb.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=754&h=503&fit=crop&dpr=3 2262w" sizes="(min-width: 1466px) 754px, (max-width: 599px) 100vw, (min-width: 600px) 600px, 237px">
<figcaption>
<span class="caption">Play or education? Online games like Fortnite can be both.</span>
<span class="attribution"><span class="source">Shutterstock</span></span>
</figcaption>
</figure>
<h2>Are official guidelines outdated?</h2>
<p>This all presents an interesting dilemma: can we really fit screen time into discrete categories, and should we apply limits to some but not others? </p>
<p>This blurring of boundaries has led researchers from the University of Auckland’s Centre for Informed Futures – <a href="https://informedfutures.org/#">Koi Tū</a> – to call for clearer and more detailed official screen time recommendations. </p>
<hr>
<p>
<em>
<strong>
Read more:
<a href="https://theconversation.com/minecraft-can-increase-problem-solving-collaboration-and-learning-yes-at-school-113335">Minecraft can increase problem solving, collaboration and learning - yes, at school</a>
</strong>
</em>
</p>
<hr>
<p>Specifically, they felt the current recommended limits failed to represent the variety of screen time students experience. This was supported by a <a href="https://informedfutures.org/wp-content/uploads/Screen-time-The-effects-on-childrens-emotional-social-cognitive-development.pdf">review of the academic literature</a> covering the impacts of screen time. </p>
<p>While research indicates a broad association between excessive screen time and a range of behavioural, learning and other problems, the results are far from conclusive and can generally be attributed to other factors. </p>
<p>The review also found the type of screen time is important: in many cases, negative effects were driven by passive screen use, whereas interactive use didn’t have the same impacts. In fact, the latter can have positive influences, such as better learning achievement and enhanced cognitive skills.</p>
<p><div data-react-class="Tweet" data-react-props="{"tweetId":"1433883952566005761"}"></div></p>
<h2>Getting the balance right</h2>
<p>This suggests we need to reorient our views of screen time away from a blunt measure of time spent on screens and towards better understanding what children are really doing on those screens. </p>
<p>While balancing passive and interactive screen time is <a href="https://clalliance.org/blog/screen-time-kids-getting-balance-right/">clearly important</a>, so is finding ways to encourage and prioritise more socially and educationally productive online behaviour.</p>
<hr>
<p>
<em>
<strong>
Read more:
<a href="https://theconversation.com/stop-worrying-about-screen-time-its-your-childs-screen-experience-that-matters-118610">Stop worrying about screen 'time'. It's your child’s screen experience that matters</a>
</strong>
</em>
</p>
<hr>
<p>This should also guide the adoption of technology in schools. Rather than wholesale integration within every aspect of learning, devices should clearly add value or improve teaching and learning, not simply replace traditional practices. </p>
<p>The role of screen devices in classrooms is particularly relevant in light of New Zealand’s 2018 PISA results, which indicated children using devices in subjects like mathematics and science achieved lower scores than those who didn’t. </p>
<p>In August this year, the Ministry of Education <a href="https://www.educationcounts.govt.nz/publications/schooling/pisa-2018-digital-devices-and-student-outcomes">responded</a> by saying: </p>
<blockquote>
<p>Digital devices have the potential to enhance learning, but there are few situations where this happens currently and many in which learning may be hindered. </p>
</blockquote>
<p><div data-react-class="Tweet" data-react-props="{"tweetId":"1352685261512187905"}"></div></p>
<h2>Active versus passive time</h2>
<p>It’s true there is considerable scepticism about the validity of the PISA tests, and <a href="https://educationendowmentfoundation.org.uk/education-evidence/evidence-reviews/digital-technology-2019">wider research</a> into the influence of screens in classrooms has shown mixed results.</p>
<p>Generally, however, we cannot claim a causal, linear relationship between use of devices and academic outcomes. Rather than assuming the PISA results indicate screen time is detrimental to learning, we need to consider <em>how</em> screens are actually being used in classes.</p>
<p>We need to focus on integrating technology that makes a difference and enhances learning. Students learn best when they are <a href="https://clalliance.org/about-connected-learning/">actively engaged</a> and create and drive their own learning.</p>
<hr>
<p>
<em>
<strong>
Read more:
<a href="https://theconversation.com/problems-with-pisa-why-canadians-should-be-skeptical-of-the-global-test-118096">Problems with PISA: Why Canadians should be skeptical of the global test</a>
</strong>
</em>
</p>
<hr>
<p>The same principles can apply to the use of digital devices – limiting passive consumption in favour of students being actively creative. This will open up new <a href="https://gazette.education.govt.nz/articles/teachers-supported-to-embrace-new-technologies/">learning opportunities</a> and provide students with authentic experiences.</p>
<p>For example, rather than students simply watching a YouTube clip to learn about the solar system, they might create their own augmented reality simulation, requiring them to apply their knowledge to correctly place, size and animate digital objects.</p>
<p>Rebalancing screen time in this way will help avoid the more negative consequences of these ubiquitous devices and highlight some of their unique advantages. </p>
<p>But this will require deeper and more critical thinking about what might be gained or lost in a world where engaging with digital technology is increasingly unavoidable.</p><img src="https://counter.theconversation.com/content/168650/count.gif" alt="The Conversation" width="1" height="1" />
<p class="fine-print"><em><span>Cheryl Brown receives funding from the Ministry of Business, Innovation and Employment </span></em></p><p class="fine-print"><em><span>Kathryn MacCallum does not work for, consult, own shares in or receive funding from any company or organisation that would benefit from this article, and has disclosed no relevant affiliations beyond their academic appointment.</span></em></p>Lockdowns and learning from home have further embedded digital technology in young people’s lives. Educational theory and practice need to catch up fast.Kathryn MacCallum, Associate Professor of Digital Education Futures, University of CanterburyCheryl Brown, Associate Professor of e-Learning, University of CanterburyLicensed as Creative Commons – attribution, no derivatives.tag:theconversation.com,2011:article/1601012021-05-03T20:07:58Z2021-05-03T20:07:58ZYes, quality teaching improves student outcomes. But that means all teachers need support – not just those in training<figure><img src="https://images.theconversation.com/files/398279/original/file-20210503-21-182uquo.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=496&fit=clip" /><figcaption><span class="caption">
</span> <span class="attribution"><a class="source" href="https://www.shutterstock.com/image-photo/teacher-helping-two-small-kids-classroom-310986524">Shutterstock</a></span></figcaption></figure><p>In his speech to <a href="https://theageschools.com.au/">The Age Schools Summit</a> in Melbourne last week, federal Education Minister Alan Tudge talked about his <a href="https://ministers.dese.gov.au/tudge/initial-teacher-education-review-launched">recently launched review</a> of initial teacher education. He said quality teaching was the most important in-school factor for determining student outcomes, and the review was a step towards this goal. </p>
<p><a href="https://www.dese.gov.au/uncategorised/resources/school-quality-australia-exploring-drivers-student-outcomes-and-links-practice-and-schooling-quality">Some research</a> backs the minister’s claim — teaching has a significant impact on student outcomes. But the focus on initial teacher education is insufficient. </p>
<p>First, <a href="https://epaa.asu.edu/ojs/article/view/1977">research also shows</a> a school’s level of advantage or disadvantage has a significant role to play in student outcomes, in some cases more so than the “quality” of its teachers.</p>
<p>And second, <a href="https://www.aitsl.edu.au/research/ite-data-report-2019">15,000 teachers</a> are graduating from Australian universities each year. This is a fraction of the <a href="https://www.abs.gov.au/statistics/people/education/schools/latest-release#staff">300,000 teachers</a> in the workforce, all having and continuing to have an impact on students.</p>
<p>This means reviewing initial teacher education does little to help the <a href="https://www.abs.gov.au/statistics/people/education/schools/latest-release#:%7E:text=there%20were%204%2C006%2C974%20students%20enrolled,13.5%20students%20to%20one%20teacher.">more than 4 million students</a> enrolled in Australian schools.</p>
<p>Helping all teachers improve their teaching is a better and faster way to improve the performance of Australian students. Our research shows how we can do this.</p>
<h2>Quality teaching and equality</h2>
<p>In 2019, Deloitte Access Economics issued a report, commissioned by the federal Education Department, called “<a href="https://www.dese.gov.au/uncategorised/resources/school-quality-australia-exploring-drivers-student-outcomes-and-links-practice-and-schooling-quality">School quality in Australia: Exploring the drivers of student outcomes and the links to practice and schooling quality</a>”. The report found the most important in-school factor driving student outcomes was teaching practice.</p>
<p>According to the report, the effect of teaching practice on student outcomes is twice as great as the next most significant driver — the classroom environment.</p>
<p>However, other studies, both in <a href="https://link.springer.com/chapter/10.1007%2F978-3-319-08759-7_7">Australia</a> and <a href="https://epaa.asu.edu/ojs/article/view/1977">internationally</a>, point to socio-economic inequalities having concentrated and considerable effects on student engagement and achievement.</p>
<p>For instance, a 2014 <a href="https://link.springer.com/chapter/10.1007%2F978-3-319-08759-7_7">Australian study</a> noted leaders have tended to cherry-pick evidence. The study’s author’s wrote:</p>
<blockquote>
<p>[…] State and Commonwealth education ministers have tended to focus quite selectively on research findings that speak to the positive outcomes associated with quality teaching, while neglecting the complexity of this field […] The phenomenon of “residualisation” in particular, whereby disadvantage is concentrated in certain public schools as a result of “school choice”, has quite powerful effects on the engagement and achievement of low SES [socioeconomic] students.</p>
</blockquote>
<p>The education minister’s current approach emphasises in-school factors while minimising the impact of out-of-school factors on student achievement. Both are important if we are to improve our students’ results.</p>
<hr>
<p>
<em>
<strong>
Read more:
<a href="https://theconversation.com/australian-schools-are-becoming-more-segregated-this-threatens-student-outcomes-155455">Australian schools are becoming more segregated. This threatens student outcomes</a>
</strong>
</em>
</p>
<hr>
<h2>How do we improve teaching quality?</h2>
<p>Worldwide, <a href="https://doi.org/10.1080/03054985.2020.1842182">four broad strategies</a> are used to improve teaching: </p>
<ol>
<li><p>recruiting and training “better” teachers</p></li>
<li><p>improving initial teacher education</p></li>
<li><p>measuring and evaluating the quality of teaching</p></li>
<li><p>providing professional development to build the capacity of practising teachers. </p></li>
</ol>
<p>Recruiting strong candidates into teaching and improving teacher education have merit, but they are long-term strategies. Evaluating the quality of teaching might be helpful in identifying needed reforms but does not, in itself, guarantee improvement.</p>
<p>However, building teaching capacity in all teachers will deliver results. This is especially true when seeking quick outcomes, such as <a href="https://ministers.dese.gov.au/tudge/being-our-best-returning-australia-top-group-education-nations">Alan Tudge’s goal</a> for Australian schools to be back “among the world’s top nations” in reading, maths and science by 2030. </p>
<h2>So, how do we build capacity?</h2>
<p>As a profession, we struggle to agree on what makes a quality teacher. We developed an approach focused on what teachers do in the classroom rather than who they are. In other words, quality teaching rather than quality teachers.</p>
<p>At the core of our approach is a framework called the <a href="https://qtacademy.edu.au/what-is-the-quality-teaching-model/">quality teaching model</a>, which focuses on three key concepts:</p>
<ul>
<li><p>the need for intellectual quality, rigour or challenge in every learning experience</p></li>
<li><p>the need to create classroom environments that support not only students but also their learning</p></li>
<li><p>the need to increase the significance of student learning so they can see its connection to the world beyond the classroom.</p></li>
</ul>
<p>Using this model, we devised a professional development process called “<a href="https://qtacademy.edu.au/what-is-qtr/">quality teaching rounds</a>”. It is applicable to every grade, subject and teacher career stage.</p>
<p>These rounds involve teachers collaborating in professional learning communities of four or more. They observe and analyse each other’s teaching using the quality teaching model. Over a period of weeks, each teacher takes a turn to host a lesson observed by their peers. </p>
<hr>
<p>
<em>
<strong>
Read more:
<a href="https://theconversation.com/making-better-use-of-australias-top-teachers-will-improve-student-outcomes-heres-how-to-do-it-131297">Making better use of Australia's top teachers will improve student outcomes: here's how to do it</a>
</strong>
</em>
</p>
<hr>
<p>All the teachers (including the host) assess the lesson using the elements of quality in our model. Next, they have discussions about each teacher’s justification of their assessment, drawing on evidence gathered during the lesson. </p>
<p>The goal is to reach consensus on what is working. This process generates lively interaction, critical insights and goes well beyond providing feedback to the host teacher. Importantly, the assessments remain confidential to the participants, creating a safe space for their analysis. </p>
<h2>Does it work?</h2>
<p>This approach has been shown to improve the quality of teaching, teacher morale and, most importantly in the current context, student performance.</p>
<p>We conducted a trial involving 192 teachers randomly assigned to two groups: the first group did quality teaching rounds and the comparison group did professional development as usual. The researchers were blinded to group allocation.</p>
<p>Our <a href="https://protect-au.mimecast.com/s/YvxLC0YKMxS4plyMuwVJjH?domain=sciencedirect.com">findings</a> show the quality of teaching (measured by our quality teaching model) improved significantly in the group that participated in rounds.</p>
<hr>
<p>
<em>
<strong>
Read more:
<a href="https://theconversation.com/the-government-claims-teaching-is-a-national-priority-but-cheaper-degrees-wont-improve-the-profession-141524">The government claims teaching is a national priority, but cheaper degrees won't improve the profession</a>
</strong>
</em>
</p>
<hr>
<p>This year, we <a href="https://protect-au.mimecast.com/s/Ua_WCgZ0y3cNDnBmfoeNN9?domain=sciencedirect.com">published findings</a> of a more recent trial involving 234 year 3 and 4 primary teachers and more than 5,000 students from 133 New South Wales government schools. The participants were randomly assigned to one of three groups: a group involved in quality teaching rounds; a less structured form of peer observation; or professional development as usual (control). </p>
<p>Compared to the control group, student outcomes in mathematics improved by 25% in the group where teachers participated in quality teaching rounds. This was equal to two months additional improvement over an eight month period. The results also improved by less than one month in the peer observation group but were not statistically significant.</p>
<h2>Resources matter too</h2>
<p>If we are to meet the education minister’s objectives for Australia to again be among the world’s leading nations in student performance, we must support all teachers with professional development shown to work.</p>
<p>Yet it would be remiss not to acknowledge the enormous contribution of out-of-school factors in determining student outcomes.</p>
<p>Inadequate resources and disadvantage in low socioeconomic schools play a significant role in students’ poorer educational outcomes.</p>
<p>Teachers, teaching and teacher education cannot alone make the improvements sought without considerable commitment to, and investment in, rectifying longstanding inequalities.</p><img src="https://counter.theconversation.com/content/160101/count.gif" alt="The Conversation" width="1" height="1" />
<p class="fine-print"><em><span>Jenny Gore receives funding from the Australian Research Council, Paul Ramsay Foundation and NSW Department of Education.</span></em></p>Improving initial teacher education is a long-term strategy. It won’t achieve the education minister’s goal of getting Australia to the top-performing nations in maths and literacy by 2030.Jenny Gore, Laureate Professor of Education, Director Teachers and Teaching Research Centre, University of NewcastleLicensed as Creative Commons – attribution, no derivatives.tag:theconversation.com,2011:article/1283882019-12-06T04:50:35Z2019-12-06T04:50:35ZChinese students top the PISA rankings, but some Shanghai parents are turning away from the school system<figure><img src="https://images.theconversation.com/files/305530/original/file-20191206-39009-b51h84.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=496&fit=clip" /><figcaption><span class="caption">China is fast becoming a middle-class nation.</span> <span class="attribution"><a class="source" href="https://unsplash.com/photos/kwypdNYwebY">Ewan Yap/Unsplash</a></span></figcaption></figure><p>Australian 15 year olds were around <a href="https://www.abc.net.au/news/2019-12-03/australia-education-results-maths-reading-science-getting-worse/11760880">three and a half years behind</a> their counterparts in China in maths, according to the OECD’s latest results for education systems around the world.</p>
<p>The four cities of China (Beijing, Shanghai, Jiangsu and Zhejiang) that participated in the Programme for International Student Assessment (PISA) <a href="https://research.acer.edu.au/ozpisa/34/">topped the league tables</a> in maths, science and reading. </p>
<p>These four cities don’t represent China as a whole, but their combined size is comparable to a typical OECD country. However, it’s important to mention China’s PISA results don’t reflect the huge number of students living outside the big cities. </p>
<p>Many Westerners believe Chinese students are superseding them because of their <a href="https://www.oecd-ilibrary.org/docserver/9789264190672-6-en.pdf?expires=1575593027&id=id&accname=guest&checksum=96371FFF58012A748179911A21B7E540">Confucian values</a> which see students as hardworking, disciplined and therefore teachable. </p>
<p>But my research into middle-class parents in Shanghai shows they find traditional schools too rigid. They want their children to be globally-minded and adopt values those schools don’t focus on as much – like self-discovery and creativity.</p>
<h2>The hybrid education model</h2>
<p>My two-year research involved interviews with a group of 46 middle-class Chinese parents in Shanghai.</p>
<p>China’s rapid social and economic transformation means it’s fast becoming a middle-class nation with significant spending power. Only 4% of China was made up of <a href="https://www.chinausfocus.com/2022/wp-content/uploads/Part+02-Chapter+07.pdf">middle-class urban households</a> in 2000, but this is estimated to soar to 45% by 2022. </p>
<p>I chose to interview these parents because they represent the recent shift in the educational landscape in Chinese cities. The parents I interviewed are <a href="https://www.relocatemagazine.com/articles/education-schools-int-guide-apac-chinas-insatiable-appetite-for-a-western-education">demanding a world-class education</a> for their children. While academic achievement, such as that evidenced by PISA scores, is important to them, they all said the local Chinese school system was too rigid and exam-oriented. </p>
<p>These parents are more accepting of a <a href="https://www.lek.com/insights/sr/ultimate-consumer-good-education-trends-china">Western-style education</a> because they believe it will give their children an “edge” and grant them access to international opportunities. </p>
<p>They have enrolled their kids in what are called “hybrid” or “East-meets-West” schools. Here, Chinese students learn subjects such as maths, Chinese language and values education from local teachers. But they also study for the International Baccalaureate or the General Certificate of Education at Advanced-Level (the higher school certificate in the UK). </p>
<p>Subjects such as English, arts, music, physical education and philosophy within these curricula are taught by Western expatriate teachers and Chinese bilingual teachers.</p>
<p>The parents who send their children to such schools also value extra-curricular activities and overseas holiday study tours for their children. These extra curriculum activities aren’t just “cram school”. They include drama, dance, singing, sports, and learning one or two musical instruments. </p>
<h2>What parents said</h2>
<p>China’s one-child policy (which ended in 2015) means parents feel more pressure for their child’s success. The modern Chinese family has an <a href="https://www.sixthtone.com/news/1000899/chinas-modern-families-double-income-and-an-invisible-kid">inverted family structure</a>, known as the “4-2-1” (four grandparents, two parents and one child).</p>
<p>The Shanghai middle-class parents in my study were mostly single children themselves. This partly explains why education, and the desire to ensure their child has good job prospects, is so vital for them, as they support their elderly parents alone, and their children are expected to also.</p>
<p>Still, the parents I interviewed placed more emphasis on developing their children’s global mindsets and intercultural competence than just having them attain good academic results. </p>
<p>Mrs Xuan told me her ten-year-old son’s school “exposed (him) to various Western teachers’ way of speaking and communicating […] They have taught him how to communicate better and think differently”.</p>
<p>She said:</p>
<blockquote>
<p>I want to give my child different options that will allow him to see multiple perspectives and know how to find his own direction.</p>
</blockquote>
<p>Another parent, Ms Ju, who owns a private business believes a “good” education involves giving her 11-year-old daughter an opportunity to experience being “under-pressure, but not just at school. </p>
<p>She told me:</p>
<blockquote>
<p>I don’t want my child to just focus on doing well academically which is why […] she is playing piano competitively outside school so that she can learn how to manage her time and improve herself.</p>
</blockquote>
<p>Professor Zheng, an engineering academic, talked about his experience of teaching an increasingly diverse university student cohort in his elite Chinese university. </p>
<p>He believes his role is to prepare his son (who was eight years of old when I interviewed him) for a world that is becoming a "global village”:</p>
<blockquote>
<p>My son and his generation will see the world differently from my parents and me because of the technological and economic advancements that he is now experiencing. […] Through travelling, he understands what cultural diversity is and gets to learn how to live in a globalised world.</p>
</blockquote>
<p>Other <a href="https://www.tandfonline.com/doi/full/10.1080/03050068.2017.1360566">studies on Chinese parents</a> who send their children to international schools have suggested they offer a haven for domestic students from the competitive and discriminatory features of the Chinese educational system.</p>
<p>Education is more complex than what we see on the PISA chart. Shanghai parents’ aspirations for their children show they are leaning towards the kind of educational system countries like Australia are known for.</p><img src="https://counter.theconversation.com/content/128388/count.gif" alt="The Conversation" width="1" height="1" />
<p class="fine-print"><em><span>Hannah Soong receives funding from Mulit-Year New Colombo Mobility Program for International Placements in Shanghai (2017-2019)</span></em></p>Parents in Shanghai are aiming for their children to adopt Western style values – like self-discovery.Hannah Soong, Senior Lecturer in Teacher Education Practice, School of Education, University of South AustraliaLicensed as Creative Commons – attribution, no derivatives.tag:theconversation.com,2011:article/1283012019-12-05T18:33:38Z2019-12-05T18:33:38ZVital Signs: Australia’s slipping student scores will lead to greater income inequality<figure><img src="https://images.theconversation.com/files/305291/original/file-20191205-70174-1kjpymn.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=496&fit=clip" /><figcaption><span class="caption">While no test is perfect but the Programme for International Student Assessment rankings are pretty useful for understanding the skills young people are being equipped with.</span> <span class="attribution"><span class="source">www.shutterstock.com</span></span></figcaption></figure><p>The latest Programme for International Student Assessment (PISA) <a href="https://www.oecd.org/pisa/publications/PISA2018_CN_AUS.pdf">results</a> are bad news for Australian students, parents, employers and policymakers. Pretty much everyone.</p>
<p>Australian 15-year-olds are now below the OECD average in mathematics, and our results in reading and science have fallen badly.</p>
<hr>
<figure class="align-center zoomable">
<a href="https://images.theconversation.com/files/305284/original/file-20191204-70101-ocuq0h.png?ixlib=rb-1.1.0&q=45&auto=format&w=1000&fit=clip"><img alt="" src="https://images.theconversation.com/files/305284/original/file-20191204-70101-ocuq0h.png?ixlib=rb-1.1.0&q=45&auto=format&w=754&fit=clip" srcset="https://images.theconversation.com/files/305284/original/file-20191204-70101-ocuq0h.png?ixlib=rb-1.1.0&q=45&auto=format&w=600&h=266&fit=crop&dpr=1 600w, https://images.theconversation.com/files/305284/original/file-20191204-70101-ocuq0h.png?ixlib=rb-1.1.0&q=30&auto=format&w=600&h=266&fit=crop&dpr=2 1200w, https://images.theconversation.com/files/305284/original/file-20191204-70101-ocuq0h.png?ixlib=rb-1.1.0&q=15&auto=format&w=600&h=266&fit=crop&dpr=3 1800w, https://images.theconversation.com/files/305284/original/file-20191204-70101-ocuq0h.png?ixlib=rb-1.1.0&q=45&auto=format&w=754&h=334&fit=crop&dpr=1 754w, https://images.theconversation.com/files/305284/original/file-20191204-70101-ocuq0h.png?ixlib=rb-1.1.0&q=30&auto=format&w=754&h=334&fit=crop&dpr=2 1508w, https://images.theconversation.com/files/305284/original/file-20191204-70101-ocuq0h.png?ixlib=rb-1.1.0&q=15&auto=format&w=754&h=334&fit=crop&dpr=3 2262w" sizes="(min-width: 1466px) 754px, (max-width: 599px) 100vw, (min-width: 600px) 600px, 237px"></a>
<figcaption>
<span class="caption">Trends in performance in reading, mathematics and science: The blue line indicates the average mean performance across OECD countries with valid data in all PISA assessments. The red dotted line indicates mean performance in Australia. The black line represents a trend line for Australia. * indicates mean-performance estimates that are statistically significantly above or below PISA 2018 estimates for Australia.</span>
<span class="attribution"><a class="source" href="https://www.oecd.org/pisa/publications/PISA2018_CN_AUS.pdf">OECD, PISA 2018 Database</a></span>
</figcaption>
</figure>
<hr>
<p>Reading has gone backwards by three-quarters of a school year since 2000, mathematics by more than one school year since 2003, and science by a school year since 2006.</p>
<hr>
<p>
<em>
<strong>
Read more:
<a href="https://theconversation.com/aussie-students-are-a-year-behind-students-10-years-ago-in-science-maths-and-reading-127013">Aussie students are a year behind students 10 years ago in science, maths and reading</a>
</strong>
</em>
</p>
<hr>
<p>The results have generated a firestorm of press coverage, and it’s true they are certainly alarming. </p>
<p>They underscore the need for a serious discussion about education reform in this country.</p>
<p>They also help explain our poor productivity growth in recent years, and are full of gloomy news for future employment prospects and income inequality.</p>
<h2>Skills and productivity</h2>
<p>While no test is perfect, PISA is pretty useful for understanding the skills young people are equipped with as they enter the workforce or embark on post-secondary education.</p>
<p>This is because, as <a href="https://www.acer.org/au/">Australian Council for Educational Research</a> chief Geoff Masters <a href="https://www.theage.com.au/education/focus-on-basics-leaves-schoolkids-short-in-essential-deep-thinking-20191203-p53gd6.html">has pointed out</a>:</p>
<blockquote>
<p>Unlike many tests and examinations, PISA does not assess students’ abilities to recall facts or basic literacy and numeracy skills. Instead, it assesses the ability to transfer and apply learning to new situations and unseen problems. This requires an understanding of fundamental concepts and principles, as well as the ability to think.</p>
</blockquote>
<p>This week the Australian Bureau of Statistics <a href="https://www.abs.gov.au/AUSSTATS/abs@.nsf/Lookup/5260.0.55.002Main+Features12018-19?OpenDocument">published figures</a> revealing a 0.2% fall in labour productivity for 2019, with a shocking 0.8% fall adjusting for the quality of work performed.</p>
<p>Many factors contribute to labour productivity, but what economist call “human capital” (essentially “skills”) are a key part. With the skills of Australia’s workforce and future workforce falling, is it any wonder that productivity is lagging?</p>
<p>That’s a big problem for wages growth. In the long run, one basically expects real living standards for workers to track productivity. There can be deviations for certain time periods, but as Nobel laureaute Paul Krugman <a href="https://mitpress.mit.edu/books/age-diminished-expectations-third-edition">famously put it in 1990</a>:</p>
<blockquote>
<p>Productivity isn’t everything, but in the long run it is almost everything. A country’s ability to improve its standard of living over time depends almost entirely on its ability to raise its output per worker.</p>
</blockquote>
<h2>Widening the gap</h2>
<p>The PISA results also tell us we have also dropped relative to other countries. In a world where routine tasks are increasingly being automated and there is fairly strong international mobility of labour, that’s doubly concerning. </p>
<p>Our ability to compete in a globalised world of competitive international trade depends not only only our absolute level of skills but our relative level of skills. </p>
<p>Take mathematics. Our PISA scores are now comparable to those of Portugal, and behind Latvia and the Czech Republic. Estonia – with a <a href="https://tradingeconomics.com/estonia/gdp-per-capita">GDP per capita that’s half ours</a> – is miles ahead of us. We’re even getting beaten by New Zealand.</p>
<hr>
<p><iframe id="tc-infographic-450" class="tc-infographic" height="400px" src="https://cdn.theconversation.com/infographics/450/1dfde5e7ef453dc463a581efdf72dc01acca6ab5/site/index.html" width="100%" style="border: none" frameborder="0"></iframe></p>
<hr>
<p>A final piece of depressing news is that we have fewer top performers and many more poor performers. In NSW, for instance, our number of poor performers has doubled since 2003 <a href="https://www.smh.com.au/education/pisa-tests-put-fewer-nsw-students-among-the-world-s-brightest-20191204-p53gxa.html">(from 10% to 20%)</a>. Our number of top performers has roughly halved over the same period.</p>
<p>As University of Chicago economics professor <a href="https://www.theatlantic.com/science/archive/2019/02/green-new-deal-economic-principles/582943/">Michael Greenstone has emphasised</a>, the fundamental fact of labour economics is that people get paid for their skills.</p>
<p>If the spread of skills between the best and worst students in Australia is getting wider, then we should expect income inequality to worsen. The tax and transfer system in Australia does a lot to mitigate that inequality, but there are limits.</p>
<h2>What to do?</h2>
<p>So how to improve Australia’s education system?</p>
<p>As <a href="http://research.economics.unsw.edu.au/richardholden/assets/the-australian%2c-friday%2c-june-17%2c-2016%2c-pages-from-12-to-12.pdf">I have pointed out previously</a>, the international evidence on “what works” in education involves some things that cost real money and others that are basically free.</p>
<p><a href="https://pubs.aeaweb.org/doi/pdfplus/10.1257/app.5.4.28">An excellent 2013 paper</a> by economists Will Dobbie and Roland Fryer used data from New York City charter schools to calculate that five policies explained about 45% of the variation in school effectiveness. Those policies were:</p>
<blockquote>
<p>frequent teacher feedback, the use of data to guide instruction, high-dosage tutoring, increased instructional time and high expectations.</p>
</blockquote>
<p>Of course, people can argue about the “external validity” of causal effects in specific geographies or educational contexts. That’s why we need good evidence from randomised controlled trials in education in Australia.</p>
<hr>
<p>
<em>
<strong>
Read more:
<a href="https://theconversation.com/the-top-ranking-education-systems-in-the-world-arent-there-by-accident-heres-how-australia-can-climb-up-128225">The top ranking education systems in the world aren't there by accident. Here's how Australia can climb up</a>
</strong>
</em>
</p>
<hr>
<p>But make no mistake: the cost of failing to improve the skills of our children and future workforce is huge.</p>
<p>It has dramatic implications for productivity, living standards and income inequality.</p>
<p>And it’s too important for ideological commitments on the left or the right to prevent real reform. The only thing that matters is “what works”.</p><img src="https://counter.theconversation.com/content/128301/count.gif" alt="The Conversation" width="1" height="1" />
<p class="fine-print"><em><span>Richard Holden does not work for, consult, own shares in or receive funding from any company or organisation that would benefit from this article, and has disclosed no relevant affiliations beyond their academic appointment.</span></em></p>The skills children learn at school have dramatic implications for their own future and the nation’s productivity, living standards and income inequality.Richard Holden, Professor of Economics, UNSW SydneyLicensed as Creative Commons – attribution, no derivatives.tag:theconversation.com,2011:article/1270132019-12-03T08:47:35Z2019-12-03T08:47:35ZAussie students are a year behind students 10 years ago in science, maths and reading<figure><img src="https://images.theconversation.com/files/304631/original/file-20191202-66982-1gwv3yy.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=496&fit=clip" /><figcaption><span class="caption">The latest OECD PISA results show Australia is doing worse than last time in science and maths.</span> <span class="attribution"><span class="source">from shutterstock.com</span></span></figcaption></figure><p>Australian 15-year-old reading scores are way below those of their peers in ten countries – including Singapore, Estonia, Canada, Finland, Ireland, Korea and Poland.</p>
<p>And around 41% of Australian 15 year olds have failed to meet the minimum national standards in reading – up from 31% in 2000.</p>
<p>These are some of the <a href="http://www.acer.edu.au/ozpisa/reports/">results</a> from the OECD’s 2018 Program for International Student Assessment (PISA) released today, which tested the performance of education systems across 79 countries and economies.</p>
<p>Since PISA first assessed reading literacy in 2000, Australia’s mean score has declined by the equivalent of around three-quarters of a year of schooling. </p>
<p>In maths, Australia trailed 23 countries including Singapore, Japan, Korea, Estonia, the Netherlands, Poland, Canada, the United Kingdom and Ireland. And in science, we were behind 12 countries including Singapore, Estonia, Japan, Finland, Canada, Poland and New Zealand. </p>
<p>The latest results put Australia in 11th place in reading – on par with China, Denmark, Germany, Norway, Sweden, the United Kingdom and the United States. </p>
<p>This placement doesn’t sound bad, but compared to the highest performing OECD countries, Australia is about seven months in reading and 12 months in science behind Estonia, and 15 months behind Japan in maths.</p>
<p>In science, maths and reading, Australia’s students today are almost a full year behind those of more than a decade ago.</p>
<h2>Australia’s performance on a downward trajectory</h2>
<p>PISA is a two-hour test to see how well 15-year-old students in (randomly selected) secondary schools across all 36 OECD countries, and 43 other countries or economies, can apply reading, maths and science to real-life situations.</p>
<p>The three assessment domains are rotated every three years, so one domain is the major focus (the major domain). A larger amount of the assessment time is devoted to this domain compared to the other two (the minor domains). This year, reading was the major domain.</p>
<hr>
<p>
<em>
<strong>
Read more:
<a href="https://theconversation.com/pisa-results-dont-look-good-but-before-we-panic-lets-look-at-what-we-can-learn-from-the-latest-test-69470">PISA results don’t look good, but before we panic let’s look at what we can learn from the latest test</a>
</strong>
</em>
</p>
<hr>
<p>Australian students did achieve an average of 503 points in reading. The OECD average – the benchmark against which each country’s performance in PISA 2018 can be compared – was 487 points in reading.</p>
<p>The four provinces of China which participated (Beijing, Shanghai, Jiangsu and Zhejiang) had the highest mean reading score of 555 points. Singapore was the highest performing country with an average of 549 points.</p>
<hr>
<p><iframe id="tc-infographic-450" class="tc-infographic" height="400px" src="https://cdn.theconversation.com/infographics/450/1dfde5e7ef453dc463a581efdf72dc01acca6ab5/site/index.html" width="100%" style="border: none" frameborder="0"></iframe></p>
<hr>
<p>The results also show 20% of Australian students did not meet the international level of reading proficiency on the PISA performance scale. This is the level the OECD determines a student needs to actively participate in life. In 2000, 12% of Australian students didn’t attain this level in reading.</p>
<p>Australia’s 2018 performance was above the OECD average in science, but it fell to be at the OECD average in maths.</p>
<p>The PISA results show 46% of Australian 15 year olds failed to meet the minimum national standards in mathematics and 42% fell short in science.</p>
<hr>
<p>
<em>
<strong>
Read more:
<a href="https://theconversation.com/the-pisa-world-education-test-results-are-about-to-drop-is-australia-getting-worse-127011">The PISA world education test results are about to drop. Is Australia getting worse?</a>
</strong>
</em>
</p>
<hr>
<p>Over the 15 years of measuring maths literacy, Australia’s performance has declined by the equivalent of about one and one-quarter years of schooling. And over the 12 years of measuring science literacy, Australia’s performance has declined by almost one full year of schooling. </p>
<p>Female students across all countries and economies participating in PISA 2018 outperformed male students in reading. In Australia, girls were around the equivalent of one year of schooling ahead of boys.</p>
<h2>How we compare across the nation</h2>
<p>Based on the latest scores, the OECD has labelled Australia as having a “high quality - high equity” education system because scores on both were above the OECD average.</p>
<p>But there was a difference of about three years of schooling in each subject area between students in the highest socioeconomic quarter (advantaged students) and those in the lowest socioeconomic quarter (disadvantaged students).</p>
<p>The OECD labels students who have not reached the <a href="https://www.oecd.org/australia/PISA-2012-low-performers-Australia-ENG.pdf">baseline level of proficiency</a> required to participate fully in modern society as “low performers”.</p>
<p>In reading, one in three disadvantaged students was classed as a “low performer”, compared to just one in ten advantaged students. In maths, the numbers are even more sobering: 37% of disadvantaged students were low performers, compared to 11% of advantaged students. </p>
<hr>
<p><iframe id="4nRvA" class="tc-infographic-datawrapper" src="https://datawrapper.dwcdn.net/4nRvA/4/" height="400px" width="100%" style="border: none" frameborder="0"></iframe></p>
<hr>
<p>Indigenous students were between two and three years behind their non-Indigenous peers across all areas – with 43% (compared to 18%) classed as low performers in reading, 48% (compared to 21%) in maths and 44% (compared to 18%) in science.</p>
<h2>How we compare internationally</h2>
<p>If excelling by international standards means performing to a standard similar to the Asian powerhouses, we have a great deal of work ahead. </p>
<p>The combined four provinces of China that participated – Beijing, Shanghai, Jiangsu and Zhejiang – while by no means representing China as a whole, represent more than 180 million people, and have an average income well below the OECD average. </p>
<p>Their scores are about one and a half years of schooling higher than Australia in reading, three and a half years higher in maths, and three years higher in science. </p>
<p>Participation in international studies such as PISA enable us to stop and look at how Australia’s education system stacks up against those of other countries – including our trading partners. </p>
<p>These findings show, again, that achievement in reading, maths, and science has been in steady decline for many years. We need to push the pause button and take stock of our curriculum, teaching and assessment methods.</p><img src="https://counter.theconversation.com/content/127013/count.gif" alt="The Conversation" width="1" height="1" />
<p class="fine-print"><em><span>Sue Thomson does not work for, consult, own shares in or receive funding from any company or organisation that would benefit from this article, and has disclosed no relevant affiliations beyond their academic appointment.</span></em></p>Australian education is still going backwards from where we started in 2000 – new PISA results show.Sue Thomson, Deputy CEO (Research), Australian Council for Educational ResearchLicensed as Creative Commons – attribution, no derivatives.tag:theconversation.com,2011:article/1190692019-06-26T19:36:38Z2019-06-26T19:36:38ZWhy Jason Kenney’s ‘common sense’ education platform gets it wrong<figure><img src="https://images.theconversation.com/files/281033/original/file-20190624-97777-1kg1tzl.jpg?ixlib=rb-1.1.0&rect=0%2C20%2C3383%2C1519&q=45&auto=format&w=496&fit=clip" /><figcaption><span class="caption">More testing won't improve math achievement. Here, Alberta premier Jason Kenney with Adriana LaGrange, Minister of Education, after being sworn into office in Edmonton on April 30, 2019.</span> <span class="attribution"><span class="source"> THE CANADIAN PRESS/Jason Franson</span></span></figcaption></figure><p>Alberta Premier Jason Kenney wants to “bring common sense to <a href="https://www.cbc.ca/news/canada/edmonton/alberta-ucp-education-plan-ata-teachers-1.5076141">education</a>.” He has insinuated teachers are <a href="https://jumpradio.ca/news/5099538/alberta-teachers-ucp-kenney-education-reforms/">not already accountable</a>
to their students and suggested that <a href="https://www.cbc.ca/news/canada/calgary/kenney-education-platform-funding-class-size-testing-standardized-1.5070660">“failed teaching fads” including “inquiry learning”</a> are responsible for the decline in <a href="https://edmontonjournal.com/opinion/columnists/david-staples-jason-kenney-would-reverse-course-on-education-reform-photo-radar">student scores measured by the Program for International Student Assessment</a>, better known as <a href="https://www.cmec.ca/251/Programme_for_International_Student_Assessment_(PISA).html">PISA</a>. </p>
<p>Both Kenney’s diagnosis and his prescriptions are misguided, even harmful: the main problems plaguing Alberta students’ performance isn’t the way math is taught but inequality and huge classes. </p>
<p>I am an associate professor of education at St. Mary’s University specializing in curriculum and instruction, and a Calgarian with kids of my own. I help educate future elementary teachers, and I’ve taught in California when the use of standardized testing was accelerated. I also research <a href="http://www.socstrpr.org/?page_id=2372">economic inequality</a> and its implications for <a href="https://www.routledge.com/Teaching-for-Democracy-in-an-Age-of-Economic-Disparity-1st-Edition/Wright-Maley-Davis/p/book/9781315678603">democracy and education</a>. </p>
<h2>Source of decline?</h2>
<p>Kenney told Albertans that the province has seen a <a href="https://www.facebook.com/kenneyjasont/videos/802445530110451/">“devastating reduction” and “disturbing decline in math proficiency</a>,” which jeopardizes the province’s future. Currently, Alberta’s PISA ranking falls <a href="https://education.alberta.ca/media/3402015/pisa-2015-math.pdf">just above average in Canada, behind Québec and B.C.; internationally, Alberta’s ranking is above average</a>.</p>
<p>Scholars point out that it’s problematic to take PISA results at face value because there are issues having to do with <a href="https://www.tandfonline.com/doi/pdf/10.1080/00313831.2016.1258726?needAccess=true">translation and reporting of error rates</a>, with <a href="https://www.uv.es/RELIEVE/v22n1/RELIEVEv22n1_M15eng.pdf">methods, sampling and response rates</a> and with unequal application of standards in all <a href="https://www.forbes.com/sites/realspin/2017/01/04/are-the-pisa-education-results-rigged/#5b14b0d81561">countries</a>.</p>
<hr>
<p>
<em>
<strong>
Read more:
<a href="https://theconversation.com/problems-with-pisa-why-canadians-should-be-skeptical-of-the-global-test-118096">Problems with PISA: Why Canadians should be skeptical of the global test</a>
</strong>
</em>
</p>
<hr>
<p>Kenney has criticized the way math is taught. But he might be surprised to learn that the biggest drop in Alberta’s PISA math achievement occurred <a href="https://www.cmec.ca/docs/pisa2006/Pisa2006.en.pdf">between 2003 and 2006</a>. The rollout of the 2007 math curriculum, which he has deemed responsible, <a href="https://edmontonjournal.com/news/local-news/hitting-the-books-how-alberta-education-is-rewriting-curriculum-for-the-next-generation-of-students">took four years to implement</a>. If test scores were falling before and after the curriculum changes, then the math curriculum is not the likeliest culprit. </p>
<h2>More testing not the answer</h2>
<p>Kenney has <a href="https://www.huffingtonpost.ca/2019/03/27/jason-kenneys-alberta-education_a_23701184/">suggested</a> that Alberta not only return to testing Grade 3, but doing so for Grades 1 and 2 as well. The idea that more testing will improve achievement is, however, contradicted by evidence. </p>
<figure class="align-center ">
<img alt="" src="https://images.theconversation.com/files/281043/original/file-20190624-97794-1tqjtza.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=754&fit=clip" srcset="https://images.theconversation.com/files/281043/original/file-20190624-97794-1tqjtza.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=600&h=400&fit=crop&dpr=1 600w, https://images.theconversation.com/files/281043/original/file-20190624-97794-1tqjtza.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=600&h=400&fit=crop&dpr=2 1200w, https://images.theconversation.com/files/281043/original/file-20190624-97794-1tqjtza.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=600&h=400&fit=crop&dpr=3 1800w, https://images.theconversation.com/files/281043/original/file-20190624-97794-1tqjtza.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=754&h=503&fit=crop&dpr=1 754w, https://images.theconversation.com/files/281043/original/file-20190624-97794-1tqjtza.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=754&h=503&fit=crop&dpr=2 1508w, https://images.theconversation.com/files/281043/original/file-20190624-97794-1tqjtza.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=754&h=503&fit=crop&dpr=3 2262w" sizes="(min-width: 1466px) 754px, (max-width: 599px) 100vw, (min-width: 600px) 600px, 237px">
<figcaption>
<span class="caption">Kenney has suggested bringing back standardized testing for Grade 3, and adding it to grades 1 and 2 as well.</span>
<span class="attribution"><span class="source">(Shutterstock)</span></span>
</figcaption>
</figure>
<p>In the United States, the vast increase in testing seen since 2002’s federal <a href="https://www.npr.org/sections/ed/2015/10/27/443110755/no-child-left-behind-what-worked-what-didnt">No Child Left Behind Act</a> increased scores on state tests (though not on <a href="https://www.brookings.edu/blog/brown-center-chalkboard/2017/04/07/what-international-test-scores-reveal-about-american-education/">PISA</a>), but at the cost of <a href="https://www.tcpress.com/what-works-may-hurt%E2%80%94side-effects-in-education-9780807759059">diminishing the quality of education</a> as a whole. Since 2000, the global standing of the U.S. in education <a href="https://nces.ed.gov/pubs2002/2002116.pdf">plummeted from 18th</a> to <a href="https://www.washingtonpost.com/local/education/on-the-world-stage-us-students-fall-behind/2016/12/05/610e1e10-b740-11e6-a677-b608fbb3aaf6_story.html?noredirect=on&utm_term=.8360e264a22f">to 38th</a>. </p>
<p>More high-stakes testing doesn’t improve the quality of education; it often <a href="https://journals.sagepub.com/doi/abs/10.3102/0013189x07306523">narrows the curriculum</a> and inhibits the development of essential <a href="https://www.tandfonline.com/doi/abs/10.1080/0305764X.2011.607151">thinking skills</a> required of today’s graduates. Put simply, you don’t fatten a pig by weighing it.</p>
<hr>
<p>
<em>
<strong>
Read more:
<a href="https://theconversation.com/reduce-childrens-test-anxiety-with-these-tips-and-a-re-think-of-what-testing-means-111730">Reduce children’s test anxiety with these tips — and a re-think of what testing means</a>
</strong>
</em>
</p>
<hr>
<h2>Growing economic inequality</h2>
<p>Epidemiologists Richard G. Wilkinson and Kate Pickett synthesized global education data from the OECD, UNICEF, the World Health Organization and the World Bank. They found that for children aged 15 years and younger, math achievement scores are <a href="https://www.bmj.com/content/335/7629/1080">inversely correlated</a> to economic inequality: as inequality increases, test scores decline. Scholars have studied this phenomenon since <a href="https://books.google.ca/books?hl=en&lr=&id=gwfcAgAAQBAJ&oi=fnd&pg=PR9&ots=JjcB_V0vF4&sig=1pEtxo8AZp1rkDsgnreBRR4NMMM#v=onepage&q&f=false">at least the 1960s</a>.</p>
<p>Between the years 2000-2015, <a href="https://irpp.org/research-studies/framing-the-new-inequality/">provincial tax policy</a> helped Alberta become the <a href="https://www.huffingtonpost.ca/2014/11/19/alberta-income-inequality_n_6186066.html">most unequal</a> <a href="https://www.parklandinstitute.ca/from_gap_to_chasm">province in Canada</a> — a situation that likely <a href="https://www.theguardian.com/inequality/2018/sep/18/kate-pickett-richard-wilkinson-mental-wellbeing-inequality-the-spirit-level">impacts the health and wellness</a> of Albertans.</p>
<p>Not surprisingly, <a href="https://www.oecd-ilibrary.org/docserver/9789264102873-en.pdf?expires=1554927840&id=id&accname=guest&checksum=DA94E5E0027B254CE204688EC595BED8">economic inequality within a population is a significant indicator of student performance</a>. The performance gap related to inequality <a href="http://files-eu.clickdimensions.com/ei-ieorg-a4ucu/files/oecd2018equityineducation-embargoedcopy.pdf?1539877267533&_cldee=aGVsZW5hLnNjaHVsekBlaS1pZS5vcmc%3D&recipientid=contact-cd8fdaf713b0e3118e4d005056ad0002-7ecf26c888e24500b2d9ecda6e8fc31f&esid=d6ac42ea-15df-4e5d-ad99-b2a835d93d34">widens in adolescence</a>, and Canadian performance deterioration occurred in the context of <a href="https://www.oecd-ilibrary.org/docserver/9789264266490-10-en.pdf?expires=1554930139&id=id&accname=guest&checksum=CC4ED450B62FEFB9791C4DC6B4E262A0">increasing socioeconomic inequality</a>.</p>
<h2>Teacher-student ratios burden achievement</h2>
<p>Meanwhile, Alberta class sizes have soared. Since <a href="https://education.alberta.ca/media/1626474/commissionreport.pdf">Alberta’s Commission on Learning in 2003</a> called for class-size caps, the number of core classes (math, English language arts, social studies and science) that have more than 40 kids crammed into them has <a href="https://edmontonjournal.com/news/local-news/alberta-school-class-size-crowded-rooms-surged-as-spending-remained-untracked">grown by nearly 600 per cent</a>. </p>
<p>The OECD’s own analysis reveals that when teacher-student ratios increase above 25 “there is a continuous decline in school performance in all <a href="http://www.oecd.org/education/school/programmeforinternationalstudentassessmentpisa/33691596.pdf">PISA subject domains</a>.” </p>
<p>This year, some Alberta schools <a href="https://calgaryherald.com/news/local-news/public-school-principals-prepare-for-fewer-teachers-larger-classrooms-more-anxiety-next-fall">plan to reduce teaching staff</a>, which will further increase class sizes. The province already needs as many as <a href="https://edmontonjournal.com/news/local-news/large-class-sizes-becoming-unmanageable-says-alberta-teachers-association-president">3,000 teachers</a> across the province.</p>
<p>According to a 2014 report, Alberta’s <a href="https://www.cdhowe.org/sites/default/files/attachments/research_papers/mixed/Commentary_488.pdf">poorest students outperform their peers worldwide</a>. Canadian education has among the lowest <a href="https://www.oecd.org/publications/equity-in-education-9789264073234-en.htm">socioeconomic gaps in the world</a>, which suggests that teachers and the existing system are acting as life preservers to buoy educational achievement in an inequitable context.</p>
<p>While Kenney has highighted declining math scores in Alberta, their problem-solving success is conveniently ignored. Between (<a href="http://www.oecd.org/education/school/programmeforinternationalstudentassessmentpisa/34009000.pdf">2003</a> and <a href="http://cmec.ca/Publications/Lists/Publications/Attachments/324/AMatters_No6_EN_Web.pdf">2012</a>), Alberta students’ performance in <a href="https://www.pisa.tum.de/en/domains/problem-solving/">problem solving as measured on PISA</a> has steadily improved: Alberta now ranks behind only <a href="https://www.cmec.ca/Publications/Lists/Publications/Attachments/389/PISA2015_CPS_EN.pdf">Singapore, British Columbia and Japan</a> in this PISA testing domain. Problem solving is the first stated <a href="http://www.learnalberta.ca/ProgramOfStudy.aspx?ProgramId=26061#">goal of the Alberta K-9 math curriculum</a>. </p>
<h2>Savings and investment</h2>
<p>The Kenney government is investing heavily in corporate tax cuts. There is ample evidence to suggest that these cuts <a href="https://ips-dc.org/wp-content/uploads/2017/08/EE17-final-embargoed-for-August-30.pdf">do not lead to job growth</a> and <a href="https://ideas.repec.org/p/eca/wpaper/2013-264399.html">may even lead to job losses</a>. Across Canada, tax cuts have cost provinces <a href="https://www.policyalternatives.ca/publications/reports/corporate-income-taxes-profit-and-employment-performance-canadas-largest-compa">$12 billion</a> annually since 2000, and have rapidly <a href="https://ips-dc.org/report-corporate-tax-cuts-boost-ceo-pay-not-jobs/">accelerated inequality</a>. </p>
<p>It is important to highlight that inequality destroys <a href="https://www.wiley.com/en-ca/Rethinking+Capitalism:+Economics+and+Policy+for+Sustainable+and+Inclusive+Growth-p-9781119120957">economic growth</a> in the <a href="https://www.oecd.org/els/soc/OECD2015-In-It-Together-Chapter1-Overview-Inequality.pdf">long term</a>. Improvements in educational achievement, however, are <a href="https://www.tandfonline.com/doi/abs/10.1080/09645292.2015.1059801">strongly tied to long-term economic growth</a>, as is investment in teachers and schools. </p>
<p>The Conference Board of Canada reports that the long-term return for every <a href="https://www.conferenceboard.ca/press/newsrelease/2017/10/26/expanding-early-childhood-education-and-care-in-canada-yields-significant-economic-and-societal-benefits">dollar spent on early childhood education (ages two to four) is $6 in terms of reduced costs on social spending</a> (for example, on prisons, social services and health care, etc.) and improved tax revenues. </p>
<p>Likewise, dollars spent on education from kindergarten to Grade 12 translate to roughly <a href="https://www.ipr.northwestern.edu/publications/policy-briefs/school-spending-policy-research-brief-Jackson.pdf">$2 per dollar invested</a>. The Conference Board of Canada concludes “every dollar spent on post-secondary education creates $1.36 <a href="https://www.conferenceboard.ca/hcp/Details/education/tertiary.aspx">for the Canadian economy</a>.” Education spending is a substantial <a href="https://openknowledge.worldbank.org/handle/10986/29672">net-growth investment vehicle</a>, one that is being driven off the road by careless and ignorant policy.</p>
<p>There’s no quick or cheap fix to the mess created by decades of <a href="https://thenarwhal.ca/norway-s-oil-savings-just-hit-1-trillion-alberta-has-17-billion-what-s-gives/">Conservative neglect of investment in future needs</a>.</p>
<p>Alberta can improve achievement by cultivating conditions for growth: reduce class sizes to allow more attention from teachers, provide teachers with time and opportunities for professional development and collaboration and implement a progressive system of taxation that helps create a society in which children have adequate resources to thrive.</p><img src="https://counter.theconversation.com/content/119069/count.gif" alt="The Conversation" width="1" height="1" />
<p class="fine-print"><em><span>Cory Wright-Maley receives funding from Alberta Education for work on implementing newly mandated Teacher Quality Standards. His views are his own and do not necessarily represent those of his employer.</span></em></p>The main problem plaguing Alberta students’ math performance isn’t the current math curriculum or teacher accountability, but inequality and ballooning class sizes.Cory Wright-Maley, Associate Professor, St. Mary’s University (Calgary)Licensed as Creative Commons – attribution, no derivatives.tag:theconversation.com,2011:article/1180962019-06-04T22:19:55Z2019-06-04T22:19:55ZProblems with PISA: Why Canadians should be skeptical of the global test<figure><img src="https://images.theconversation.com/files/277415/original/file-20190531-69091-1c83c55.jpg?ixlib=rb-1.1.0&rect=46%2C0%2C5184%2C3453&q=45&auto=format&w=496&fit=clip" /><figcaption><span class="caption">Focusing on narrow PISA measures may increase skill levels but cause students to miss out on the kinds of learning that generates higher-order thinking.</span> <span class="attribution"><span class="source">(Shutterstock)</span></span></figcaption></figure><p>The <a href="http://www.oecd.org/pisa/">Program for International Student Assessment (PISA)</a> — the Organization for Economic
Co-operation and Development’s (OECD) global standardized test of student achievement — is frequently used by commentators to compare and rank national or <a href="https://theconversation.com/canadian-schools-spend-more-as-enrolment-and-test-scores-fall-116945">provincial education systems</a>. </p>
<p>PISA, which has now spread into 80 countries as a <a href="https://simonbreakspear.com/wp-content/uploads/2015/09/Breakspear-PISA-Paper.pdf">best education practice</a>, presents itself as a tool to <a href="https://www.oecd.org/pisa/aboutpisa/">help countries make their systems more inclusive leading to equitable outcomes</a>.
But PISA is far more ambiguous and <a href="https://www.theguardian.com/education/2014/may/06/oecd-pisa-tests-damaging-education-academics">controversial</a>. </p>
<p>Many academics and educators critique PISA as an economic measurement, <a href="https://www.tandfonline.com/doi/abs/10.1080/03050068.2016.1143278">not an educational one.</a> The media generally use PISA results to <a href="https://www.tandfonline.com/doi/abs/10.1080/03050068.2018.1425243">blame and shame</a> school systems. And the way that some politicians, policy-makers and researchers have used PISA is more closely aligned to a <a href="https://onlinelibrary.wiley.com/doi/abs/10.1002/berj.3120">political process</a> than an educational one.</p>
<p>There are many reasons to be skeptical about PISA rankings and how they’re used to compare student achievement or to identify best practices or solutions for educational problems. </p>
<h2><em>1. A narrow measurement</em></h2>
<p>PISA numbers are limited in what they can explain and the conclusions they can support.</p>
<p>PISA measures math, science and reading skills, not more <a href="https://unesdoc.unesco.org/ark:/48223/pf0000247444">holistic educational goals</a> or <a href="https://en.unesco.org/themes/literacy">understanding of literacy</a> as defined by the United Nations Educational, Scientific and Cultural Organization <a href="https://en.unesco.org/">(UNESCO)</a>. PISA is a narrow measure of educational achievement.</p>
<hr>
<p>
<em>
<strong>
Read more:
<a href="https://theconversation.com/testing-literacy-today-requires-more-than-a-pencil-and-paper-114154">Testing literacy today requires more than a pencil and paper</a>
</strong>
</em>
</p>
<hr>
<p>PISA typically samples 5,000 to 10,000 15-year-old students from about 500 randomly selected schools in each country every three years. Up to 53 students are randomly selected in each of these schools. </p>
<p>In small countries with fewer than 5,000 students, all 15-year-old students are sampled. The two-hour test relies heavily on multiple choice and rating scale questions. </p>
<p>PISA ignores the importance of engagement and <a href="https://www.tandfonline.com/doi/full/10.1080/14767724.2018.1531234">positive attitudes to learning</a> for future success. </p>
<figure class="align-center ">
<img alt="" src="https://images.theconversation.com/files/277413/original/file-20190531-69051-1s2yzrc.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=754&fit=clip" srcset="https://images.theconversation.com/files/277413/original/file-20190531-69051-1s2yzrc.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=600&h=400&fit=crop&dpr=1 600w, https://images.theconversation.com/files/277413/original/file-20190531-69051-1s2yzrc.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=600&h=400&fit=crop&dpr=2 1200w, https://images.theconversation.com/files/277413/original/file-20190531-69051-1s2yzrc.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=600&h=400&fit=crop&dpr=3 1800w, https://images.theconversation.com/files/277413/original/file-20190531-69051-1s2yzrc.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=754&h=503&fit=crop&dpr=1 754w, https://images.theconversation.com/files/277413/original/file-20190531-69051-1s2yzrc.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=754&h=503&fit=crop&dpr=2 1508w, https://images.theconversation.com/files/277413/original/file-20190531-69051-1s2yzrc.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=754&h=503&fit=crop&dpr=3 2262w" sizes="(min-width: 1466px) 754px, (max-width: 599px) 100vw, (min-width: 600px) 600px, 237px">
<figcaption>
<span class="caption">The capacity for teamwork resists measurement but is significant for student and lifelong success.</span>
<span class="attribution"><span class="source">(Shutterstock)</span></span>
</figcaption>
</figure>
<p>Educational outcomes that are important for future citizens or a well-rounded education are not captured: for example, democratic participation, understanding of history and politics, teamwork, well-being, moral values, creativity, aesthetic skills, athletic talent or communication skills. </p>
<h2><em>2. Relation to Indigenous schools, special needs students</em></h2>
<p>In Canada, <a href="https://www.cdhowe.org/media-release/measuring-indigenous-education-outcomes-key-progress">federally funded Indigenous schools are not considered in PISA data</a> and would likely impact provincial scores. The absence of Indigenous schools may help mask disparities in educational equity in Canada and marginalize the significance of <a href="https://www.cbc.ca/cbcdocspov/features/first-nations-schools-are-chronically-underfunded">chronic under-funding of Indigenous-controlled education</a> and <a href="https://policyoptions.irpp.org/magazines/july-2018/gaping-holes-ottawas-indigenous-fiscal-policy/">basic infrastructure</a> necessary to Indigenous child well-being.</p>
<p>Differing levels of how <a href="https://www.tandfonline.com/doi/abs/10.1080/0969594X.2017.1421523?journalCode=caie20">special needs students are integrated in distinct jurisdictions</a> also confound PISA results.</p>
<h2><em>3. Statistical uncertainty</em></h2>
<p>PISA tests a sample of students and the results are then adjusted to reflect a whole population of 15-year-old students. The scores therefore include a measure of statistical uncertainty and PISA <a href="http://www.oecd.org/pisa/pisafaq/">can only report a range of positions</a> (upper rank and lower rank) where a country can be placed. </p>
<p>There is a <a href="https://eric.ed.gov/?id=EJ1022498">co-relation between poverty and lower test results</a>: the OECD notes that up to <a href="https://www.oecd.org/pisa/keyfindings/pisa-2012-results-volume-I.pdf">46 per cent of the differences in PISA mathematics scores among OECD countries</a> can be explained by socio-economic disadvantage.</p>
<p><a href="https://www.merriam-webster.com/dictionary/psychometrics">Psychometric</a> experts, who examine the fitness and effects of particular methodological choices and the validity and reliability of modelling and calculations, <a href="https://scholarworks.iu.edu/journals/index.php/ried/article/view/616">criticize PISA for downplaying the problematic nature of its calculations</a>, and its <a href="https://eric.ed.gov/?id=EJ1056717">lack of transparency in reporting uncertainties</a>.</p>
<p>The majority of nations fall in the middle PISA rankings. However, small differences in mean scores can shift rankings by 10 to 20 places.</p>
<p>Relying on a small number of questions also means scores are highly affected by completion rates. In some jurisdictions, higher scores may result from greater significance being placed on <a href="https://www.tandfonline.com/doi/abs/10.1080/01596306.2014.930020">PISA completion</a> by parents and school authorities.</p>
<h2><em>4. Pressure to narrow curricula</em></h2>
<p>As an instrument of international comparison, PISA has <a href="https://eric.ed.gov/?id=EJ1033739">created pressure for states to narrow curricula</a>, relegating subjects such as the arts and social studies to second-class status, and to introduce <a href="https://journals.sagepub.com/doi/pdf/10.1177/1478210316652009">testing cultures to monitor performance and achievement</a>. </p>
<figure class="align-center ">
<img alt="" src="https://images.theconversation.com/files/277412/original/file-20190531-69067-g3yofy.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=754&fit=clip" srcset="https://images.theconversation.com/files/277412/original/file-20190531-69067-g3yofy.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=600&h=400&fit=crop&dpr=1 600w, https://images.theconversation.com/files/277412/original/file-20190531-69067-g3yofy.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=600&h=400&fit=crop&dpr=2 1200w, https://images.theconversation.com/files/277412/original/file-20190531-69067-g3yofy.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=600&h=400&fit=crop&dpr=3 1800w, https://images.theconversation.com/files/277412/original/file-20190531-69067-g3yofy.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=754&h=503&fit=crop&dpr=1 754w, https://images.theconversation.com/files/277412/original/file-20190531-69067-g3yofy.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=754&h=503&fit=crop&dpr=2 1508w, https://images.theconversation.com/files/277412/original/file-20190531-69067-g3yofy.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=754&h=503&fit=crop&dpr=3 2262w" sizes="(min-width: 1466px) 754px, (max-width: 599px) 100vw, (min-width: 600px) 600px, 237px">
<figcaption>
<span class="caption">Testing culture has been linked to a dislike of schools and learning.</span>
<span class="attribution"><span class="source">(Shutterstock)</span></span>
</figcaption>
</figure>
<p>Testing culture and curriculum narrowing have been linked to students dropping out, students and teachers cheating, students undergoing <a href="https://www.straitstimes.com/singapore/education/nurture-joy-of-learning">stress and anxiety disorders</a>, teachers leaving the profession, a fear of failing and a <a href="https://www.tandfonline.com/doi/abs/10.1080/01596306.2014.930020">dislike of schools and learning</a>. </p>
<h2><em>5. Overlooking inquiry-based learning</em></h2>
<p>As school systems narrow curricula to focus on testable concepts, students may reach high levels of proficiency in a few subjects but lose out on programs of study based on active, inquiry-based processes and content.</p>
<p>Countries with the highest PISA scores appear to have the lowest levels of inquiry-based learning. High levels of inquiry-based science appear to have a negative association with <a href="https://www.europhysicsnews.org/articles/epn/pdf/2017/04/epn2017484p17.pdf">PISA science scores</a>. Focusing on PISA may <a href="https://search.informit.com.au/documentSummary;dn=337192064494253;res=IELAPA">increase skill levels but cause students to miss out on learning</a> that generates higher-order thinking. </p>
<h2><em>6. Cultural specificity</em></h2>
<p>Even though PISA use is spreading globally, and is translated into national languages, it is still framed by Western understandings and <a href="https://journals.sagepub.com/doi/abs/10.1177/1474904116658299">may distort</a> results from students with <a href="https://eric.ed.gov/?id=EJ752980">diverse social and cultural histories</a>. This will become more of a challenge as <a href="https://theconversation.com/new-global-testing-standards-will-force-countries-to-revisit-academic-rankings-115199">PISA’s ambition is to move beyond testing skills to assess attitudes that promote student success</a>.</p>
<h2><em>7. Corporate partnership in the age of digital surveillance</em></h2>
<p>Finally, <a href="https://newint.org/features/2016/04/01/edu-businesses-impact">journalists</a> and <a href="https://bctf.ca/uploadedFiles/Public/Issues/Privatization/PearsonGutsteinReport.pdf">researchers have expressed concerns</a> about <a href="https://www.pearson.com/corporate/news/media/news-announcements/2014/12/pearson-to-develop-pisa-2018-student-assessment-21st-century-fra.html">PISA’s partnership with Pearson</a>, a global educational business enterprise that boasts it operates <a href="https://www.pearsonglobalschools.com/index.cfm?locator=PSZqAo">more than 70 countries worldwide, reaching 100 million people</a>. </p>
<p>The partnership is a worrisome conflict of interest. PISA assesses and ranks school systems and Pearson is a <a href="https://worldsofeducation.org/en/woe_homepage/woe_detail/4858/unmaking-the-market-maker-pearson-in-the-global-south">provider of global</a> and <a href="https://www.reuters.com/article/us-pearson-outlook/virtual-schools-keep-pearson-on-course-for-return-to-profit-growth-idUSKCN1MR0S0">online</a>
<a href="https://www.connectionsacademy.com/news/pearson-acquisition">charter schools</a>, tests and <a href="https://www.pearson.com/us/higher-education/why-choose-pearson/thought-leadership/online-blended-learning/online-learning-services.html">education consulting</a>. Additionally, in an age of rising concern about <a href="https://www.alrc.gov.au/publications/9.%20Overview%3A%20Impact%20of%20Developing%20Technology%20on%20Privacy/surveillance-technologies">digital surveillance and data privacy</a>, it is reasonable to ask how data on students and teachers that’s collected globally may be used, and to what end. </p>
<h2>What needs to change</h2>
<p>Other democratically governed organizations such as UNESCO should play a larger role in the collection of international educational data. This would ensure students’ and teachers’ rights are protected and children have the vibrant and democratic educational structures, processes and relationships they are entitled to. </p>
<p>PISA has shifted education discussion globally in alarming ways. The OECD needs to listen to critical voices and rethink its PISA strategy and framework.</p><img src="https://counter.theconversation.com/content/118096/count.gif" alt="The Conversation" width="1" height="1" />
<p class="fine-print"><em><span>Sarfaroz Niyozov works for the University of Toronto.
He received funding from SSHRC in 2009.</span></em></p><p class="fine-print"><em><span>Wendy Hughes does not work for, consult, own shares in or receive funding from any company or organisation that would benefit from this article, and has disclosed no relevant affiliations beyond their academic appointment.</span></em></p>There are many reasons to be skeptical about PISA rankings, and their use to compare student achievement or to identify best practices or solutions for educational problems.Sarfaroz Niyozov, Associate Professor, OISE, University of TorontoWendy Hughes, EdD student, OISE, University of TorontoLicensed as Creative Commons – attribution, no derivatives.tag:theconversation.com,2011:article/1169452019-05-21T22:57:54Z2019-05-21T22:57:54ZCanadian schools spend more as enrolment and test scores fall<figure><img src="https://images.theconversation.com/files/275211/original/file-20190517-69182-gos14q.jpg?ixlib=rb-1.1.0&rect=51%2C8%2C5760%2C3819&q=45&auto=format&w=496&fit=clip" /><figcaption><span class="caption">The big spending provinces in Canada did not necessarily get the best Program for International Student Assessment (PISA) standardized test results. </span> <span class="attribution"><span class="source">(Shutterstock)</span></span></figcaption></figure><p>The question of value for money is central to any public policy consideration. Given its scale, coupled with its critical social and economic impact, education ranks as one of the most important and challenging policies for analysis.</p>
<p>Canada’s school boards spent a total of <a href="https://www150.statcan.gc.ca/t1/tbl1/en/tv.action?pid=3710006501">$53.2 billion — about $11,300 per student — educating students in kindergarten to Grade 12 public systems in 2015</a> and that amount has been growing by more than $1 billion every year for several years.</p>
<p>Economists <a href="https://www.tandfonline.com/doi/abs/10.1080/09645292.2010.511840">have recently analyzed the efficiency</a> of <a href="https://www.tandfonline.com/doi/abs/10.1080/09645292.2011.595580">education sectors</a> in various countries. But despite the importance of schooling and school policy, policy-makers and economists in Canada have done little to analyze how well the public school system funded by provincial governments is working in terms of students’ academic achievement. </p>
<p>In our recent study, we examined how much money each province spends and how this stands up compared to how students achieve in the <a href="http://www.oecd.org/pisa/">Program for Student Assessment (PISA)</a> — a global standardized test that assesses reading, math and science. We learned that the <a href="https://www.tandfonline.com/doi/full/10.1080/00036846.2019.1584380">provinces varied widely in their ability to produce academic results for the money they spent</a>. </p>
<h2>Converting money into performance</h2>
<p>We looked at data from three different sources to try to analyze outcomes: first, Statistics Canada data on <a href="https://www150.statcan.gc.ca/t1/tbl1/en/tv.action?pid=3710006501">spending levels by school boards across Canada on a province-by-province basis</a>; second, Statistics Canada data on the <a href="https://www150.statcan.gc.ca/t1/tbl1/en/cv.action?pid=3710010901">number of students in the public K-12 system in each of the provinces in Canada</a>; third, data from the <a href="http://www.oecd.org/">Organization for Economic Cooperation and Development (OECD)</a>, which collects data in the <a href="http://www.oecd.org/pisa/">PISA</a> program which it administers. </p>
<figure class="align-center ">
<img alt="" src="https://images.theconversation.com/files/275213/original/file-20190517-69186-1t1c9y3.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=754&fit=clip" srcset="https://images.theconversation.com/files/275213/original/file-20190517-69186-1t1c9y3.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=600&h=400&fit=crop&dpr=1 600w, https://images.theconversation.com/files/275213/original/file-20190517-69186-1t1c9y3.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=600&h=400&fit=crop&dpr=2 1200w, https://images.theconversation.com/files/275213/original/file-20190517-69186-1t1c9y3.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=600&h=400&fit=crop&dpr=3 1800w, https://images.theconversation.com/files/275213/original/file-20190517-69186-1t1c9y3.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=754&h=503&fit=crop&dpr=1 754w, https://images.theconversation.com/files/275213/original/file-20190517-69186-1t1c9y3.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=754&h=503&fit=crop&dpr=2 1508w, https://images.theconversation.com/files/275213/original/file-20190517-69186-1t1c9y3.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=754&h=503&fit=crop&dpr=3 2262w" sizes="(min-width: 1466px) 754px, (max-width: 599px) 100vw, (min-width: 600px) 600px, 237px">
<figcaption>
<span class="caption">The Program for Student Assessment surveys how students perform in reading, math and science.</span>
<span class="attribution"><span class="source">(Shutterstock)</span></span>
</figcaption>
</figure>
<p>These PISA tests are conducted every three years, so we had data on Canadian student performance, by province, on the tests from six different surveys between 2000 and 2015 (the 2018 PISA result was not available when we conducted the research).</p>
<p>These three sets of data allowed us to construct an index of school outputs comprised of two factors: the number of students served by the public schools in each province and their performance on PISA tests.</p>
<p>Then we compared those output indices with the amount that school boards spent in each jurisdiction to see how the boards were doing at converting money into student performance, and how they compared among provinces.</p>
<p>Research suggests <a href="https://www.jstor.org/stable/2725865?seq=1#page_scan_tab_contents">socio-economic status</a> and other factors may be important determinants of what economists would term “outputs” — in this case, the number of students served and test results. Such factors considered environmental are not identical across the provinces and are not controlled by school boards. </p>
<p>For this reason, we wanted to allow for cost-efficiency comparisons based on a more level playing field. So, after examining provincial costs against PISA scores, we conducted a second-stage analysis of the data and we corrected for the effects of the cross-province differences in socio-economic conditions.</p>
<h2>Student expenditures growing</h2>
<p>If you look at PISA performance, you will see that there is <a href="https://www.tandfonline.com/doi/full/10.1080/00036846.2019.1584380">substantial variation among provinces</a>. </p>
<p>Alberta students scored an average of 542 on science tests in 2015, while students in Saskatchewan and Manitoba on average scored only 496. Alberta students also did very well on reading tests (scoring 536), while Saskatchewan and Manitoba lagged far behind at 496 and 495, respectively. Québec led the country in math, with an average score of 531, while Saskatchewan, Manitoba and Newfoundland and Labrador lagged far behind with scores in the 480s.</p>
<p>The provinces also vary widely in their spending on public education. Québec spent more than $12,400 per student in 2015, while British Columbia spent 26 per cent less at $9,200 per student.</p>
<p>Our index revealed some startling things: from 2000 to 2015, <a href="https://www.tandfonline.com/doi/full/10.1080/00036846.2019.1584380">per student expenditures (adjusted for inflation) grew by 41 per cent in the 10 provinces</a>. During the same time, enrolments fell by seven per cent across the provinces. But student performance on PISA tests also fell during this period by 11 points in reading (about two per cent), 23 points in math (about 4.5 per cent), and four points in science (about one per cent).</p>
<figure class="align-center ">
<img alt="" src="https://images.theconversation.com/files/275214/original/file-20190517-69178-186x636.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=754&fit=clip" srcset="https://images.theconversation.com/files/275214/original/file-20190517-69178-186x636.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=600&h=400&fit=crop&dpr=1 600w, https://images.theconversation.com/files/275214/original/file-20190517-69178-186x636.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=600&h=400&fit=crop&dpr=2 1200w, https://images.theconversation.com/files/275214/original/file-20190517-69178-186x636.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=600&h=400&fit=crop&dpr=3 1800w, https://images.theconversation.com/files/275214/original/file-20190517-69178-186x636.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=754&h=503&fit=crop&dpr=1 754w, https://images.theconversation.com/files/275214/original/file-20190517-69178-186x636.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=754&h=503&fit=crop&dpr=2 1508w, https://images.theconversation.com/files/275214/original/file-20190517-69178-186x636.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=754&h=503&fit=crop&dpr=3 2262w" sizes="(min-width: 1466px) 754px, (max-width: 599px) 100vw, (min-width: 600px) 600px, 237px">
<figcaption>
<span class="caption">Student performance on PISA tests in Canada fell from 2000 to 2015.</span>
<span class="attribution"><span class="source">(Shutterstock)</span></span>
</figcaption>
</figure>
<p>Because they served fewer students and achieved worse PISA outcomes, even though they spent more (in total and per student), the school boards became, on average, 20 per cent less efficient in turning their budgets into measurable student outcomes in the 15 years under review.</p>
<h2>P.E.I. most cost-effective</h2>
<p>The provinces also varied widely in their ability to produce results for money they spent. The big spending provinces did not necessarily get the best PISA results. And the provinces that spent the least did not necessarily get the worst. </p>
<p>Overall, Prince Edward Island was the most efficient province (set at 100 per cent on the scale of cost-efficiency scores), while Saskatchewan and Manitoba were the least efficient, at 33.9 per cent and 38.5 per cent, respectively, below P.E.I. </p>
<p>In between, the other provinces also fared badly. Big-spending Québec, for example, was 31.8 per cent less efficient than P.E.I., while low-spending B.C. was only 7.2 per cent behind P.E.I.</p>
<h2>Equity measures and efficiency</h2>
<p>To gauge the extent to which performance measures, such as PISA scores, factored into budgeting decisions, we also interviewed 28 budget managers in 10 Canadian provinces and two territories (12 from the department of finance or treasury board of governments, and 16 from the budget offices or equivalents of departments of education). We did this to probe what criteria budget officers use to allocate education resources. </p>
<p>We found that the preponderant pattern of budgeting used by these managers is an “increments-based-on-formula” approach — meaning that new expenditures are based on the previous expenditures, and the adjustment is mainly according to changes in student numbers and/or salary costs.</p>
<p>We argue that this formula often takes account of equity imperatives (such as allocating adequate resources to schools with more students with special needs), but is not particularly responsive to efficiency concerns. </p>
<p>It is unclear how the fact that more money is being spent may relate to particular, new or emerging needs in schools. What is known is that increased spending does not seem to be raising student achievement on PISA tests.</p>
<p>The Fraser Institute finds that compensation <a href="https://www.fraserinstitute.org/studies/education-spending-in-public-schools-in-canada-2019">(salaries, wages, pensions and fringe benefits) “accounts for most of the increase in spending” in education in Canada</a>. But their research does not parse what exactly this compensation covers: for example, whether it relates to additional positions or roles to meet student needs.</p>
<p>The results of our study leave several unanswered questions.<br>
There may be many factors impacting some provinces’ cost-to-achievement performance such as
<a href="https://theconversation.com/every-child-matters-what-principals-need-to-effectively-lead-inclusive-schools-114249">challenges related to inclusive classrooms</a>, geography and socio-economic conditions, but more money has not led to better measurable academic results. </p>
<p>It may be time to find out why.</p><img src="https://counter.theconversation.com/content/116945/count.gif" alt="The Conversation" width="1" height="1" />
<p class="fine-print"><em><span>JIm Marshall received funding from Social Sciences and Humanities research Council to complete this study. </span></em></p><p class="fine-print"><em><span>Haizhen Mou receives funding from the Social Sciences and Humanities Research Council (SSHRC) of Canada, grant #430-2014-00466. </span></em></p><p class="fine-print"><em><span>Michael M. Atkinson receives funding from the Social Sciences and Humanities Research Council to complete this study. </span></em></p>Research shows that the provinces vary widely in their ability to produce academic results for money they spend, and PEI shows the most efficient results.Jim Marshall, Lecturer, Johnson Shoyama Graduate School of Public Policy, University of ReginaHaizhen Mou, Associate Professor, Johnson Shoyama Graduate School of Public Policy, University of SaskatchewanMichael M. Atkinson, Public Policy Professor Emeritus, University of SaskatchewanLicensed as Creative Commons – attribution, no derivatives.tag:theconversation.com,2011:article/1151992019-04-16T22:46:02Z2019-04-16T22:46:02ZNew global testing standards will force countries to revisit academic rankings<figure><img src="https://images.theconversation.com/files/269405/original/file-20190415-147508-1d0brgm.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=496&fit=clip" /><figcaption><span class="caption">Policy-makers must remember that the social consequences of a test are just as important as the test's content.</span> <span class="attribution"><span class="source">(Shutterstock)</span></span></figcaption></figure><p>Since 2000 when the <a href="https://www.oecd.org/about/">Organization for Economic Co-operation and Development (OECD)</a> launched a global academic benchmark for measuring student outcomes by testing 15-year-olds, many global education systems have been impacted by what sometimes looks and feels like a race to rank high. </p>
<p>When the OECD launched <a href="http://www.oecd.org/pisa/">the Programme for International Student Assessment — PISA</a> — the idea was to enable countries to make cross-national comparisons of student achievement using a common/standard metric to increase human capital. In other words, higher academic achievement should corelate with earnings in the future and a country’s standard of living. As PISA states, it publishes the results of the test a year after the students are tested to help governments shape their education policies.</p>
<p>As PISA has developed, through seven global testing rounds <a href="http://www.oecd.org/pisa/aboutpisa/">every three years</a>, with the first in 2000 and the most recent in 2018, for some <a href="https://www.theguardian.com/education/2008/dec/11/primary-maths-science-politics">it has gained a reputation as the “Olympics of education”</a> given the widespread <a href="https://www.businessinsider.com/pisa-worldwide-ranking-of-math-science-reading-skills-2016-12">attention</a> <a href="https://www.cbc.ca/news/technology/canada-s-15-year-old-students-among-best-global-performers-in-science-math-1.3883341">that country rankings receive</a> following <a href="https://www.bbc.com/news/av/uk-wales-25203445/pisa-ranks-wales-education-the-worst-in-the-uk">the release of results</a>.</p>
<p>Recent cross-cultural research suggests <a href="https://www.routledge.com/The-PISA-Effect-on-Global-Educational-Governance-1st-Edition/Volante/p/book/9781138217416">the influence of PISA is growing around the world</a>. Indeed, in countries such as Germany and Canada assessment systems have been developed that mirror the PISA test. Further, governments look to PISA results twinned with other <a href="http://www.oecd.org/pisa/Equity-in-Education-country-note-Canada.pdf">social outcome measures such as equity in education and social mobility</a> or <a href="https://cmec.ca/Publications/Lists/Publications/Attachments/343/AMatters_No9_EN.pdf">immigrant success</a>.</p>
<p>Now, partly in the face of criticisms, PISA is looking at expanding how and what it tests. Collectively, changes to PISA will likely spur a shift in priorities by national governments — particularly since <a href="https://timesofindia.indiatimes.com/home/education/news/kvs-jnvs-and-chandigarh-schools-to-participate-in-pisa-2021/articleshow/68887781.cms">countries are keen to achieve good outcomes</a> and to rank high.</p>
<p>As this process unfolds, policy-makers must remember that the social consequences of a test are just as important as the test’s content. Putting a new face on PISA will undoubtedly present various opportunities and challenges. </p>
<h2>What PISA now tests</h2>
<p>To date, PISA has been restricted to what is generally called the “cognitive” side of learning, focusing on reading, mathematics and scientific literacy. In addition to test questions, students and school principals fill out questionnaires to provide contextual information on student and school environment characteristics that can be associated with more or less favourable performance.</p>
<figure class="align-center ">
<img alt="" src="https://images.theconversation.com/files/269606/original/file-20190416-147483-1z0ar8g.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=754&fit=clip" srcset="https://images.theconversation.com/files/269606/original/file-20190416-147483-1z0ar8g.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=600&h=400&fit=crop&dpr=1 600w, https://images.theconversation.com/files/269606/original/file-20190416-147483-1z0ar8g.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=600&h=400&fit=crop&dpr=2 1200w, https://images.theconversation.com/files/269606/original/file-20190416-147483-1z0ar8g.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=600&h=400&fit=crop&dpr=3 1800w, https://images.theconversation.com/files/269606/original/file-20190416-147483-1z0ar8g.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=754&h=503&fit=crop&dpr=1 754w, https://images.theconversation.com/files/269606/original/file-20190416-147483-1z0ar8g.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=754&h=503&fit=crop&dpr=2 1508w, https://images.theconversation.com/files/269606/original/file-20190416-147483-1z0ar8g.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=754&h=503&fit=crop&dpr=3 2262w" sizes="(min-width: 1466px) 754px, (max-width: 599px) 100vw, (min-width: 600px) 600px, 237px">
<figcaption>
<span class="caption">Keeping up with rankings can drive an emphasis on a narrow range of measurable aspects of education.</span>
<span class="attribution"><span class="source">(Shutterstock)</span></span>
</figcaption>
</figure>
<p>Countries that excel in PISA tests, such as Finland, a country with less than six million people, have become <a href="https://www.routledge.com/The-PISA-Effect-on-Global-Educational-Governance-1st-Edition/Volante/p/book/9781138217416">regarded by policy-makers as a “global reference society”</a> — an ideal to aspire to — due to their high performance in PISA rankings. </p>
<p>Asian countries or jurisdictions like Singapore, Hong Kong (China) and Japan tend to consistently achieve exceptional PISA performances and hence <a href="https://www.tandfonline.com/doi/abs/10.1080/03054985.2015.1028525">get a lot of attention from other countries wishing to emulate their success via borrowing policy.</a>
For example, England <a href="https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/773320/MTE_main_report.pdf">flew teachers out to China to study mathematics teaching</a>.</p>
<p>Yet even as countries strive to keep pace with preparing students for PISA, criticism — which largely follows any large-scale test that has important implications — has emerged. Some policy-makers, educators and researchers decry that keeping up with <a href="https://journals.sagepub.com/doi/abs/10.1177/1478210316652009">rankings drives an emphasis on a narrow, rather than broad, range of measurable aspects of education</a>.
Crude interpretations of country rankings and policy lessons can easily be used to promote rigid reforms. </p>
<p>Perhaps the most notable resistance to PISA was expressed in an open letter in <em>The Guardian</em> headlined “<a href="https://www.theguardian.com/education/2014/may/06/oecd-pisa-tests-damaging-education-academics">OECD and PISA tests are damaging education worldwide — academics</a>,” signed by a group of more than 80 high-profile global academics. </p>
<p>Clearly, the OECD has been aware of criticisms of PISA. In 2015 and in 2018, the test covered new areas respectively of collaborative problem solving and financial literacy and then global competence. In 2015, the OECD also changed the model on which <a href="https://ec.europa.eu/jrc/en/publication/insights-survey-errors-large-scale-educational-achievement-surveys">previous PISA scores were based with the aim of reducing measurement error</a>.</p>
<p>More recent statements from the director of PISA, Andreas Schleicher, suggest significant changes are on the horizon because the modern world <a href="https://www.smh.com.au/education/international-education-testing-program-set-to-change-20190324-p516zq.html">doesn’t reward us for what we know but for “how we apply knowledge</a>.”</p>
<p>Schleicher recently acknowledged “<a href="https://qz.com/1540222/how-changing-the-pisa-test-could-change-how-kids-learn/">PISA is, at the moment, a partial picture of what is important</a>.” But future testing innovations seem likely to fall flat if they are taken up by governments in a prescriptive manner. </p>
<h2>Not ‘back to basics’</h2>
<p>In the next administration in 2021, PISA will tackle creative thinking, trying to find ways to assess, and have <a href="https://qz.com/1540222/how-changing-the-pisa-test-could-change-how-kids-learn/">students assess, flexibility in thinking and habits of creativity such as being inquisitive and persistent</a>. The PISA team is also developing a way of <a href="https://www.devex.com/news/pisa-founder-andreas-schleicher-on-the-future-of-the-education-ranking-94561">testing students’ digital learning, which should be ready in time for the 2024 assessment</a>.</p>
<p>Recently, and somewhat ironically, the highest-ranking global country on PISA tests, Singapore, made waves when its education minister announced that it would be scaling back testing in the country to try to find balance with the “<a href="https://www.moe.gov.sg/news/speeches/opening-address-by-mr-ong-ye-kung--minister-for-education--at-the-schools-work-plan-seminar">joy of learning</a>.” </p>
<p><div data-react-class="Tweet" data-react-props="{"tweetId":"1058539355181064192"}"></div></p>
<p>Collectively, these statements run somewhat <a href="https://www.springer.com/gp/book/9789400740945">against a dominant standards-based reform agenda</a> that has emphasized <a href="https://www.springer.com/gp/book/9789400740945">“back to basics” in math and language since the late 1980s and early 1990s in the United Kingdom and then the United States</a>.</p>
<h2>Policy learning and borrowing?</h2>
<p>The OECD has stressed <a href="https://www.oecd.org/about/secretary-general/betterpoliciesforbetterlives.htm">“better policies for better lives.”</a> For the most part, policy-makers have largely agreed with Schleicher’s position that the PISA can help <a href="https://eric.ed.gov/?id=EJ865075">facilitate public education system improvements through the lessons learned from top performing nations or economies</a>.</p>
<p>However, it should be remembered that education policies from high achieving nations don’t migrate across international boundaries without consideration given to national and cultural contexts.
Rather, innovations and changes in education require teachers to have the time and opportunity to <a href="https://theconversation.com/culturally-responsive-teaching-in-a-globalized-world-109881">re-educate themselves in relation to more recent insights in what it means to get the best out of children</a>.</p>
<p>The OECD will need to respond to <a href="https://ec.europa.eu/jrc/en/publication/insights-survey-errors-large-scale-educational-achievement-surveys">previous critiques and provide greater transparency around newer test instruments and the choices made to arrive at rankings</a>. The latter is no small challenge since the future focus of PISA is based on topics which seem more difficult to evaluate than math, science or reading skills.</p><img src="https://counter.theconversation.com/content/115199/count.gif" alt="The Conversation" width="1" height="1" />
<p class="fine-print"><em><span>Louis Volante receives funding from the Social Sciences and Humanities Research Council of Canada (SSHRC). </span></em></p><p class="fine-print"><em><span>Jo Ritzen, John Jerrim, and Sylke Schnepf do not work for, consult, own shares in or receive funding from any company or organisation that would benefit from this article, and have disclosed no relevant affiliations beyond their academic appointment.</span></em></p>The stakes could be highest for students around the world as education systems decide how to respond to the changing shape of global standardized testing.Louis Volante, Professor, Brock UniversityJohn Jerrim, Lecturer in Economics and Social Statistics, UCLJo Ritzen, Professor of International Economics of Education, Science and Technology, Maastricht UniversitySylke Schnepf, Senior researcher, European Commission's Joint Research CentreLicensed as Creative Commons – attribution, no derivatives.tag:theconversation.com,2011:article/1101072019-01-28T14:50:26Z2019-01-28T14:50:26ZMaths: should English schools look to Switzerland rather than Shanghai for inspiration?<figure><img src="https://images.theconversation.com/files/254992/original/file-20190122-100295-1yfl9lx.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=496&fit=clip" /><figcaption><span class="caption">
</span> <span class="attribution"><span class="source">Pexels</span></span></figcaption></figure><p>When it comes to education, how countries rank against each other in league tables has become a big influence on education policy. And one of the biggest rankings is <a href="https://theconversation.com/pisa-results-four-reasons-why-east-asia-continues-to-top-the-leaderboard-69951">the Programme for International Student Assessment</a> – also known as PISA. This is a regular comparison of the performance of 15-year-olds in different countries, including in maths. </p>
<p>Increasingly politicians want to see their countries rise up the league tables, which has led to attempts to import ways of teaching from overseas. In England, many schools have adopted East Asian methods in maths teaching to try <a href="https://theconversation.com/maths-challenge-england-has-one-of-the-biggest-gaps-between-high-and-low-performing-pupils-in-the-developed-world-88678">to climb the league tables</a>. This has come be to be called the “<a href="https://theconversation.com/explainer-what-is-the-mastery-model-of-teaching-maths-25636">mastery maths</a>” model.</p>
<p><a href="https://schoolsweek.co.uk/what-is-south-asian-mastery-maths/">Singapore and Shanghai</a> both do very well in international PISA tests compared to England, but the way both places approach mastery is slightly different. The mastery method adopted in English schools has mostly been modelled on the Shanghai system – and has involved teachers from England going to Shanghai and having Shanghai teachers go to England.</p>
<p>We have spent the past four years researching whether the Shanghai exchange would lead to changes in teaching in England and whether this <a href="https://www.gov.uk/government/publications/evaluation-of-the-maths-teacher-exchange-china-and-england">would lead to better test scores</a>. We found that in many of the schools that were involved in the first exchange in 2014-2015 there have been a lot of changes – and many adopted Shanghai-style maths teaching. These changes have involved slowing down the curriculum, using more learning by heart, more interaction between teacher and pupils, and using different ways of representing maths ideas. Schools have also adopted ways to give all children access to challenging maths.</p>
<p>But when we compared the test scores to similar schools who have not adopted the mastery method, we found there had been no change in the test results of 11-year-olds. For seven-year-olds we found a small change but nothing that would suggest a massive improvement in scores from the new mastery method.</p>
<h2>Importing education</h2>
<p>Perhaps this isn’t surprising though, because teaching methods are not the <a href="https://theconversation.com/pisa-results-four-reasons-why-east-asia-continues-to-top-the-leaderboard-69951">only reason for East Asian success</a>. And in some ways, Shanghai maths hasn’t yet had a fair test in England. UK teachers have been expected to adopt Shanghai approaches, but have not had the same amount of time for professional development or to plan lessons.</p>
<p>There’s also the fact that in Shanghai primary schools, children are generally taught by the same teacher for two to three years – so the teacher gets to know the pupils very well. Policies in China that have led to many parents only having one child also mean that young children get a lot of adult attention from parents and grandparents – which could also help their development.</p>
<hr>
<p>
<em>
<strong>
Read more:
<a href="https://theconversation.com/maths-challenge-england-has-one-of-the-biggest-gaps-between-high-and-low-performing-pupils-in-the-developed-world-88678">Maths challenge: England has one of the biggest gaps between high and low performing pupils in the developed world</a>
</strong>
</em>
</p>
<hr>
<p>Although Shanghai maths imported to England has not had the effect on results that was hoped for, it does not meant that with more time it cannot lead to improvements. And there are particular ideas in the Shanghai maths approach – such as high quality mathematical talk and <a href="https://www.cambridgemaths.org/espresso/view/mastery-in-mathematics/">careful choice of how to represent maths ideas</a> – that do have <a href="https://www.mdpi.com/2227-7102/8/4/202">evidence for success in England as well as other countries</a>.</p>
<h2>Further afield</h2>
<p>If East Asian methods do not import easily because of other factors, then perhaps politicians should broaden where they look for ideas in maths. Switzerland, for example, was the top <a href="https://www.oecd.org/pisa/pisa-2015-results-in-focus.pdf">European performer in maths in PISA in 2015</a> – not far behind East Asian countries. </p>
<p>Canada also did significantly better than England. Though recently, Canadians have also started <a href="https://theconversation.com/why-canada-fails-to-be-an-education-superpower-82558">worrying about how their performance</a>.</p>
<figure class="align-center ">
<img alt="" src="https://images.theconversation.com/files/254994/original/file-20190122-100270-1lo642k.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=754&fit=clip" srcset="https://images.theconversation.com/files/254994/original/file-20190122-100270-1lo642k.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=600&h=400&fit=crop&dpr=1 600w, https://images.theconversation.com/files/254994/original/file-20190122-100270-1lo642k.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=600&h=400&fit=crop&dpr=2 1200w, https://images.theconversation.com/files/254994/original/file-20190122-100270-1lo642k.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=600&h=400&fit=crop&dpr=3 1800w, https://images.theconversation.com/files/254994/original/file-20190122-100270-1lo642k.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=754&h=503&fit=crop&dpr=1 754w, https://images.theconversation.com/files/254994/original/file-20190122-100270-1lo642k.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=754&h=503&fit=crop&dpr=2 1508w, https://images.theconversation.com/files/254994/original/file-20190122-100270-1lo642k.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=754&h=503&fit=crop&dpr=3 2262w" sizes="(min-width: 1466px) 754px, (max-width: 599px) 100vw, (min-width: 600px) 600px, 237px">
<figcaption>
<span class="caption">Children as young as two are grouped by ability in English nurseries.</span>
<span class="attribution"><span class="source">Shutterstock</span></span>
</figcaption>
</figure>
<p>There’s also more that can be learnt from PISA and other international tests. It seems, for example, that all high-performing education systems have some common features. These include: supporting teachers with good pay, conditions and status. Teachers that are given the freedom to decide what and how to teach with support from researchers but without political interference, also seems to be another contributing factor. As does having an emphasis on good outcomes for everyone – not just the highest performing children – most high performing countries do not group children by ability from a young age, as often happens in England.</p>
<p>In any case, different tests tell different stories. Another international test – <a href="https://nces.ed.gov/timss/">Trends in Maths and Science study</a> – shows England’s results in maths have been <a href="https://dera.ioe.ac.uk/28040/1/TIMSS_2015_England_Report_FINAL_for_govuk_-_reformatted.pdf">steadily improving for the past 20 years</a>. Perhaps then, the most important lesson from England’s mastery experiment is to stop worrying so much about <a href="https://theconversation.com/international-pisa-tests-show-how-evidence-based-policy-can-go-wrong-77847">international league table positions</a> – and for schools to focus on encouraging all pupils to be the best they can be.</p>
<hr>
<p>
<em>
<strong>
Read more:
<a href="https://theconversation.com/why-global-education-rankings-dont-reveal-the-whole-picture-72134">Why global education rankings don't reveal the whole picture</a>
</strong>
</em>
</p>
<hr>
<img src="https://counter.theconversation.com/content/110107/count.gif" alt="The Conversation" width="1" height="1" />
<p class="fine-print"><em><span>Mark Boylan receives funding from Department for Education, England and the Education Endowment Foundation for research in areas related to this article.</span></em></p>The Shanghai maths method is considered to be one the best in the world for teaching students mathematics, but it doesn’t necessarily translate well into English schools.Mark Boylan, Professor of Education, Sheffield Hallam UniversityLicensed as Creative Commons – attribution, no derivatives.tag:theconversation.com,2011:article/1066892018-12-11T23:22:35Z2018-12-11T23:22:35ZWhy won’t Canada collect data on race and student success?<figure><img src="https://images.theconversation.com/files/249828/original/file-20181210-76959-1jj7wni.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=496&fit=clip" /><figcaption><span class="caption">Canadians seem not to want to talk about race and racism, deferring instead to 'income' and immigration status when it comes to measuring education success.</span> <span class="attribution"><span class="source">LeonardoBurgos /Unsplash</span></span></figcaption></figure><p>Although the impact of <a href="https://www.journals.uchicago.edu/doi/abs/10.1086/692713">income inequality</a> and <a href="https://eric.ed.gov/?id=ED585396">gender</a> on education outcomes is much discussed in Canadian government-level policy debates, factors of race and racism are seldom measured or addressed.</p>
<p>However, as an education researcher <a href="https://socialsciences.mcmaster.ca/the-gateway-cities-project">comparing student outcomes in Toronto, Vancouver, New York, Chicago and London</a>, I can see Canada’s policy-makers have a big knowledge gap because they don’t deal with or have access to information regarding race. </p>
<p>Students are impacted by factors of income, gender and <em>also race</em>. The combinations of these identities <a href="https://onlinelibrary.wiley.com/doi/abs/10.1111/cars.12044">undoubtedly shape how students experience access to education</a>, work and other types of social mobility.</p>
<p>Research shows that <a href="https://onlinelibrary.wiley.com/doi/abs/10.1111/j.1541-0064.2006.00142.x">low-income can be highly racialized, yet in Canadian cities, the patterns are not completely divided along racial lines.</a> Therefore, examining income alone overlooks the many important ways that inequalities in education are not simply an issue of economics.</p>
<p>In the <a href="https://nces.ed.gov/pubs2016/2016007.pdf">United States</a> and the <a href="https://www.theguardian.com/uk-news/2017/oct/09/audit-lays-bare-racial-disparities-in-uk-schools-courts-and-workplaces">United Kingdom</a>, research reports regularly provide summaries of student outcomes by various characteristics, including race. </p>
<p>In Canada, we have a tendency to focus exclusively on whether or not a student comes from immigrant parents. I believe this focus is problematic. </p>
<h2>Is Canada leading the way in equity?</h2>
<p>Canada has been deemed an education <a href="https://www.bbc.com/news/business-40708421">superpower</a> because comparisons between the <a href="http://www.oecd.org/pisa/test/">standardized test scores (PISA)</a> of Canadian children with those in other OECD nations find Canada near the top. As well, the <a href="https://www12.statcan.gc.ca/census-recensement/2016/dp-pd/index-eng.cfm">2016 federal census</a> revealed that Canada has the highest proportion of post-secondary graduates in all 36 member countries of the Organization for Economic Development (OECD). This mean that more than half of adult citizens in Canada between the ages of 25 and 64 have a college or university credential.</p>
<p>In particular, <a href="https://www.bbc.com/news/business-40708421">the success of immigrant children is used to argue that Canada is leading the way in equity</a>.</p>
<p>In a country grappling with the heritage of colonialism, the success of immigrant children comes as good news to share and promote. This success can be interpreted as a sign that multiculturalism has been successful, that racism is not a barrier to education attainment and that immigrants are treated equally and have the same opportunities as children born in Canada.
This story continues in other arenas. Education researcher Trevor Gulliver analyzed citizenship guides for new Canadians and found that group identities in these education texts creates an idealized version of Canada as “<a href="https://www.tandfonline.com/doi/abs/10.1080/17405904.2017.1360192">Canada the redeemer.”</a></p>
<p>Such celebratory concepts of Canada need to be carefully considered. </p>
<p>There is a common misconception that racism is something that <a href="https://theconversation.com/dear-white-people-wake-up-canada-is-racist-83124">occurs in the U.S. and not in Canada</a>. </p>
<hr>
<p>
<em>
<strong>
Read more:
<a href="https://theconversation.com/quiet-canadian-ugly-american-does-racism-differ-north-of-the-border-81388">Quiet Canadian, ugly American: Does racism differ north of the border?</a>
</strong>
</em>
</p>
<hr>
<p>One of the reasons that children of immigrants do so well in Canada is because of our immigration system which favours certain assets. The “points system” of immigration awards points to applicants who speak one of Canada’s official languages. Points are also awarded for job skills and level of education. </p>
<p>This is not to discount the work that teachers and schools do to integrate, educate and welcome students of immigrants; my point is that there are some reasons that such children who already speak an official language may be doing better than immigrants who arrive as refugees in another immigrant receiving country, like Sweden.</p>
<p>Focusing on the success of immigrants detracts from the problem of how systemic racism contributes to inequality in educational experiences and outcomes. Another common misconception is that race and immigrant status are equated; of course they are not the same thing. </p>
<h2>Data on race is needed</h2>
<p>The <a href="https://www.bbc.com/news/business-40708421">celebrated study about how well Canada did with global assessment scores</a> only carries information on immigrant status, not race. With the exception of the Toronto District School Board, boards of education across the country do not record race data. </p>
<p>This lack of data has led to a dearth of studies examining the relationship between race and educational outcomes in Canada. Researchers simply do not have the data to analyze.</p>
<p>In response to research demonstrating a gap between <a href="https://theconversation.com/why-there-are-so-few-indigenous-graduates-at-convocation-96782">Indigenous and non-Indigenous student outcomes in Canada</a>, a Canadian think tank reports that some provinces have pledged to begin asking <a href="https://www.cdhowe.org/sites/default/files/attachments/research_papers/mixed/C.D.%20Howe%20E-Brief%20272%20-%20Measuring%20Student%20Outcomes%20The%20Case%20for%20Identifying%20Indigenous%20Students%20in%20Canada%E2%80%99s%20PISA%20Sample.pdf">Indigenous students attending mainstream schools to self-identify when provinces collect their PISA data</a>. No known similar move is afoot, however, with regards to collecting data about racialized students or their income levels.</p>
<p>The Ministry of Education in B.C. has opted to ask students about the language students speak at home rather than their self-identified race.</p>
<p>Not only do we not have data on race, but it seems Canadians are also reluctant to talk about race. Even Statistics Canada defers to its <a href="https://www.cbc.ca/news/canada/term-visible-minorities-may-be-discriminatory-un-body-warns-canada-1.690247">old and outdated notion of</a> “<a href="https://www12.statcan.gc.ca/census-recensement/2016/ref/dict/pop127-eng.cfm">visible minority</a>” when attempting to measure and discuss issues around race. </p>
<h2>Lack of data does not mean lack of problem</h2>
<p><a href="https://link.springer.com/article/10.1007/s12134-005-1020-7">Lumping all non-whites</a> together masks the huge <a href="https://gatewaycities.mcmaster.ca/publications/publication-documents/robson-brown-ansifef-2015.pdf">differences we see in the educational outcomes of racialized students in Toronto</a>. </p>
<p>Basically, this means comparisons are made between white and non-white people. This comparison happens even in areas like Toronto where “visible minorites” make up more than half of the population, making <a href="https://www.cbc.ca/news/canada/toronto/census-visible-minorities-1.4371018">whites in fact, a minority</a>. Further, the data tells us nothing about <a href="http://www.urbancentre.utoronto.ca/pdfs/gtuo/PovertybyPostalCodeFinal.pdf">poverty by postal code</a>.</p>
<p>In Toronto, where we do have data, the figures show that <a href="http://journals.sfu.ca/cjhe/index.php/cjhe/article/view/187972">Asian and South Asian students trend towards having high marks and are more likely to go on to university. Black and Latino students trend towards lower grades and are more likely to be placed in the “applied” stream of high school courses (which are not eligible for university)</a>. </p>
<p>Considerable research in <a href="https://www.cbc.ca/news/canada/toronto/study-black-students-toronto-york-university-1.4082463">Toronto</a> has identified Black males as having the lowest post-secondary opportunities due to their disproportionate placement in the “applied” stream of study. </p>
<p>These problems are <a href="https://www.oecd.org/education/school/50293148.pdf">not unique to Toronto</a>; they are only measured in Toronto. </p>
<p>Lack of data does not mean lack of a problem. By not collecting data on race and other important sociodemographic factors of students, we fail to correct systemic barriers to success in our educational system. </p>
<p>By conflating immigrant success with a blanket commitment to equality, we blindly assume we are doing OK as we do not have any evidence to the contrary — because we haven’t taken the time to collect it.</p><img src="https://counter.theconversation.com/content/106689/count.gif" alt="The Conversation" width="1" height="1" />
<p class="fine-print"><em><span>Karen Robson receives research funding from the Social Sciences and Humanities Research Council, the Higher Education Quality Council of Ontario, and the Ontario Council on Articulation and Transfer. </span></em></p>News of Canada’s successful immigrant students glosses over important stories of racism, for example the ‘streaming’ of Black males. But without more data beyond Toronto, the story is hard to share.Karen Robson, Ontario Research Chair in Academic Achievement and At-Risk Youth, McMaster UniversityLicensed as Creative Commons – attribution, no derivatives.tag:theconversation.com,2011:article/978662018-06-06T10:04:11Z2018-06-06T10:04:11ZMany Australian school students feel they ‘don’t belong’ in school: new research<figure><img src="https://images.theconversation.com/files/222088/original/file-20180607-137298-zcan09.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=496&fit=clip" /><figcaption><span class="caption">Students' sense of belonging at school is linked to how well they do at school.</span> <span class="attribution"><span class="source">Shutterstock</span></span></figcaption></figure><p>A report released today by the <a href="https://www.acer.org/">Australian Council for Educational Research</a> (ACER), the managers of the <a href="http://www.oecd.org/pisa/">Program for International Student Assessment</a> (PISA) in Australia, explores Australian students’ sense of belonging. This has been <a href="https://www.tandfonline.com/doi/abs/10.1080/00220670309596617">shown</a> to play a big part in academic success at school. </p>
<p>Australian students, on average, reported a poorer sense of belonging at school compared to students across the OECD. A lower proportion of Australian students than the OECD average said they “feel like they belong at school”.</p>
<h2>Why does this matter?</h2>
<p>For some students, a sense of belonging is <a href="http://www.oecd.org/education/school/programmeforinternationalstudentassessmentpisa/34002216.pdf">indicative</a> of educational success and long-term health and wellbeing. It has also been <a href="https://www.tandfonline.com/doi/abs/10.1080/00220670309596617">found</a> to promote positive attitudes towards students’ learning. </p>
<p>What’s more, students who feel part of, and accepted by, their school community are not only <a href="http://www.oecd.org/education/pisa-2015-results-volume-iii-9789264273856-en.htm">more likely</a> to participate in school activities, both academic and non-academic, but will be actively engaged in these activities.</p>
<h2>What were students asked?</h2>
<p>PISA 2015 asked students to rate their reaction to these six statements on how they feel about school:</p>
<ul>
<li>I feel like an outsider (or left out of things) at school<br></li>
<li>I make friends easily at school </li>
<li>I feel like I belong at school<br></li>
<li>I feel awkward and out of place in my school<br></li>
<li>other students seem to like me<br></li>
<li>I feel lonely at school.</li>
</ul>
<p>Student responses (strongly agree, agree, disagree and strongly disagree) were combined to construct the sense of belonging index. This allowed us to compare Australian students with their OECD counterparts and with their peers in different states and territories, socioeconomic groups, and between genders. </p>
<hr>
<figure class="align-center ">
<img alt="" src="https://images.theconversation.com/files/221945/original/file-20180606-137291-1bnsz45.png?ixlib=rb-1.1.0&q=45&auto=format&w=754&fit=clip" srcset="https://images.theconversation.com/files/221945/original/file-20180606-137291-1bnsz45.png?ixlib=rb-1.1.0&q=45&auto=format&w=600&h=545&fit=crop&dpr=1 600w, https://images.theconversation.com/files/221945/original/file-20180606-137291-1bnsz45.png?ixlib=rb-1.1.0&q=30&auto=format&w=600&h=545&fit=crop&dpr=2 1200w, https://images.theconversation.com/files/221945/original/file-20180606-137291-1bnsz45.png?ixlib=rb-1.1.0&q=15&auto=format&w=600&h=545&fit=crop&dpr=3 1800w, https://images.theconversation.com/files/221945/original/file-20180606-137291-1bnsz45.png?ixlib=rb-1.1.0&q=45&auto=format&w=754&h=686&fit=crop&dpr=1 754w, https://images.theconversation.com/files/221945/original/file-20180606-137291-1bnsz45.png?ixlib=rb-1.1.0&q=30&auto=format&w=754&h=686&fit=crop&dpr=2 1508w, https://images.theconversation.com/files/221945/original/file-20180606-137291-1bnsz45.png?ixlib=rb-1.1.0&q=15&auto=format&w=754&h=686&fit=crop&dpr=3 2262w" sizes="(min-width: 1466px) 754px, (max-width: 599px) 100vw, (min-width: 600px) 600px, 237px">
<figcaption>
<span class="caption"></span>
<span class="attribution"><span class="source">ACER/The Conversation</span>, <a class="license" href="http://creativecommons.org/licenses/by-nd/4.0/">CC BY-ND</a></span>
</figcaption>
</figure>
<hr>
<h2>How does Australia compare?</h2>
<p>Across the full spectrum of PISA participants, students in Spain had the highest levels of sense of belonging. This was followed by students in Austria and Albania. Students in Turkey had the lowest sense of belonging, followed by students in Macao (China) and the Dominican Republic. </p>
<p>Ten countries were selected for further comparison with Australia. These included seven high-performing countries – Canada, Estonia, Finland, Hong Kong (China), Japan, Macao (China) and Singapore – who performed significantly higher in scientific, reading and mathematical literacy than Australia, and three culturally similar English-speaking OECD countries – New Zealand, the UK and the US.</p>
<hr>
<p>
<em>
<strong>
Read more:
<a href="https://theconversation.com/ideas-for-australia-why-is-australia-falling-behind-in-maths-science-and-literacy-and-what-can-be-done-about-it-56267">Ideas for Australia: Why is Australia falling behind in maths, science and literacy – and what can be done about it?</a>
</strong>
</em>
</p>
<hr>
<p>Students in the high-performing countries did not necessarily report a greater sense of belonging than Australian students. Macao (China), Hong Kong (China), Singapore, Canada, Estonia and Japan came in below, and Finland above, the OECD average. Students in the other English-speaking countries also reported a sense of belonging below the OECD average. </p>
<hr>
<p><iframe id="xGHvR" class="tc-infographic-datawrapper" src="https://datawrapper.dwcdn.net/xGHvR/1/" height="400px" width="100%" style="border: none" frameborder="0"></iframe></p>
<hr>
<p>Australia performed close to the OECD average on most questions except “other students seem to like me” (88% compared with the OECD average of 82%) and “I feel like an outsider (or left out of things) at school” (77% against the OECD average of 83%). </p>
<p>Fewer Australian students disagreed with the remaining negative statements than the average. This indicates more Australian students feel awkward, out of place, and lonely in school than their OECD peers.</p>
<h2>Sense of belonging in different demographic groups</h2>
<p>We also examined sense of belonging among a number of different groups within Australia.</p>
<p>Male students in Australia reported a greater sense of belonging than female students. For males, the sense of belonging was similar to the average across OECD countries. But for female students, it was substantially lower. </p>
<p>Boys were more likely to respond positively to all of the statements. The most substantial of these were seen on the negative statements. Some 7% more female than male students reported feeling like an outsider at school, and 7% more female than male students said they felt lonely at school. </p>
<p>Indigenous students reported a much lower sense of belonging than their non-Indigenous peers. The largest difference was on the statement “I feel like I belong in school”, which 8% fewer Indigenous students agreed with.</p>
<p>Students in metropolitan schools responded more positively on all of the statements than students in either provincial or remote schools. In particular, a much lower proportion of students in provincial and remote schools felt they belonged in school (11% difference between students in metropolitan schools and those in remote schools). </p>
<p>There were stark differences in the results for different levels of socioeconomic background. In PISA, the socioeconomic index for students is broken into quarters and compared. </p>
<p>Students in the highest quarter of socioeconomic background scored at the OECD average on sense of belonging. These students’ experience of schooling is very different from that of students in the lowest quarter of socioeconomic background. </p>
<p>The differences are large on every item. The largest, again, was on the statement “I feel like I belong in school”, for which there was a 13% gap between the two groups. Just 65% of students in the lowest socioeconomic quartile agreed with this statement, compared to 78% of students in the highest socioeconomic quartile. </p>
<p>Some 73% of low socioeconomic students said they felt awkward and out of place in their school, compared to 82% of high socioeconomic students. There was a similar difference in the proportion of each group of students who said they felt like an outsider at school.</p>
<p>Finally, foreign-born and first-generation students reported a greater sense of belonging than Australian-born students.</p>
<p>Even when taken individually, some of these differences are disturbing. While the majority of Australian students feel a sense of belonging at school, there is a solid core of students who do not feel this way - roughly one in five, or five students in the average classroom.</p>
<p>But we know the issues in individual schools can be much worse than these figures describe. Other than gender, these characteristics are not randomly distributed across the population. </p>
<hr>
<p>
<em>
<strong>
Read more:
<a href="https://theconversation.com/to-reduce-inequality-in-australian-schools-make-them-less-socially-segregated-95034">To reduce inequality in Australian schools, make them less socially segregated</a>
</strong>
</em>
</p>
<hr>
<p>For example, there are schools that enrol a large proportion of low socioeconomic background students. The number of students who feel like an outsider, lonely, or awkward will be much higher in these schools than in schools in which there are few such students. </p>
<p>For provincial and remote schools, the proportions could be further inflated, with more students from Indigenous backgrounds and more students at lower levels on the socioeconomic index.</p>
<h2>Trends over time</h2>
<p>Sense of belonging at school in Australia has declined overall between PISA 2003 and PISA 2015. It has declined across all demographic groups. </p>
<p>The largest decline was on the statement “I feel like I belong at school” - with which 88% of students agreed in 2003 but just 72% agreed in 2015. The difference in the proportion who said they felt like an outsider declined by 15%, while the proportion who said they felt awkward or out of place declined by 13%.</p>
<hr>
<p><em>Read the full report <a href="https://research.acer.edu.au/ozpisa/30/">here</a>.</em></p>
<hr><img src="https://counter.theconversation.com/content/97866/count.gif" alt="The Conversation" width="1" height="1" />
<p class="fine-print"><em><span>Sue Thomson does not work for, consult, own shares in or receive funding from any company or organisation that would benefit from this article, and has disclosed no relevant affiliations beyond their academic appointment.</span></em></p>Australian students, on average, reported a poorer sense of belonging at school than the OECD average. But issues with sense of belonging aren’t distributed evenly across the population.Sue Thomson, Deputy CEO (Research), Australian Council for Educational ResearchLicensed as Creative Commons – attribution, no derivatives.tag:theconversation.com,2011:article/857552017-11-23T10:44:38Z2017-11-23T10:44:38ZCan passion make better teachers and cure Indonesia’s poor learning level?<figure><img src="https://images.theconversation.com/files/195279/original/file-20171118-11482-o0zpqt.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=496&fit=clip" /><figcaption><span class="caption">A still from a popular movie, Rainbow Troops, depicts teacher Muslimah, a character based on a real teacher, who inspires her students. Indonesia requires teachers to have passion.</span> <span class="attribution"><span class="source">Miles Films & Mizan Productions</span></span></figcaption></figure><p>After decades, <a href="https://data.worldbank.org/indicator/SE.PRM.NENR?locations=ID">89%</a> of Indonesian children are in schools. But only a few are actually learning well as shown in the results of <a href="http://penilaian.kemdikbud.go.id/perpustakaan_penilaian/uploaded/pdf/e8e92d111ccaec76c1e515dd735382ce.pdf">Indonesian National Assessment Program</a>. Indonesia was also still ranked in <a href="https://www.oecd.org/pisa/PISA-2015-Indonesia.pdf">the lowest ten</a> in the <a href="http://www.oecd.org/pisa/aboutpisa/">Programme for International Student Assessment</a> (PISA). Thus, following a recent global trend in education, Indonesia should shift its education goal from enrolment to include learning. How?</p>
<iframe src="https://data.worldbank.org/share/widget?indicators=SE.PRM.NENR&locations=ID" width="100%" height="300" frameborder="0" scrolling="no"></iframe>
<p><a href="http://documents.worldbank.org/curated/en/432391468089383429/Great-teachers-how-to-raise-student-learning-in-Latin-America-and-the-Caribbean">Research</a> shows the teacher is a key to learning improvement. <a href="http://research.acer.edu.au/cgi/viewcontent.cgi?article=1003&context=research_conference_2003">What teachers know, do and care about</a> accounts for 30% of success in students’ learning. According to an interview with an officer with a national agency, Indonesia has allocated 52% of the <a href="http://repositori.perpustakaan.kemdikbud.go.id/4070/1/JENDELA%20EDISI%20IX%202017.pdf">2017 education budget</a> for teachers. </p>
<p>Despite the huge percentage of the education budget being spent on teachers, including to raise their salaries, their performance <a href="http://www.nber.org/papers/w21806">has not yet improved</a>. Nor has student learning. What was missing?</p>
<p>We have done a diagnostic study, a preliminary part of <a href="http://www.riseprogramme.org/content/rise-indonesia">Research on Improving Systems of Education (RISE)</a>. We analysed education policy documents and interviewed education stakeholders such as policy makers, teachers and experts. We found passion is believed to be the “must have” characteristic for good teachers. This quality has commonly been assumed to improve teaching and increase learning. </p>
<p>We looked into the concept to find whether there really is a connection between passion and quality of teaching, and whether the Indonesian government has a clear guideline on how to measure passion.</p>
<h2>Teachers’ passion and its impact on learning</h2>
<p><a href="http://research.acer.edu.au/cgi/viewcontent.cgi?article=1003&context=research_conference_2003">Research</a> into teacher quality finds having passion differentiates excellent or expert teachers from mediocre teachers. Expert teachers will pay great attention to how students attain knowledge and how they interact with students. <a href="https://eric.ed.gov/?id=ED298122">Their feelings and emotions are attached to their sense of responsibility</a> about achieving success in their work. </p>
<p>Passion could be <a href="https://books.google.co.id/books/about/The_Passionate_Teacher.html?id=27KIxjwavEwC&redir_esc=y">the love of the knowledge itself or to achieving students’ potentials</a>. As a concept, the understanding of passion in the Indonesian context is similar to that of education scholars. </p>
<p>According to our interviewees (education authorities, policymakers, experts, principals), passion is the internal drive that keeps teachers improving and developing their skills and methods. Continuous improvement of teaching skills, according to <a href="https://eric.ed.gov/?id=EJ613919">Patrick, Hisley and Kempler (2000)</a>, will not only be good for teachers themselves but also for their pupils. </p>
<p>If not managed well, though, passion might have a negative impact on a teacher, such as <a href="http://psycnet.apa.org/record/2008-16034-017">burnout</a>. In addition, several studies have tried to find a correlation between teacher passion and student learning, but <a href="http://psycnet.apa.org/record/2008-16034-017">empirical investigation of teacher passion is still lacking</a>. Scholars have yet to achieve a consensus on a “theory of passion”. </p>
<p>If passion is essential to high-quality teaching, but no clear-cut theory of passion is available, how then do we identify and measure passion?</p>
<h2>How to identify passion</h2>
<p>Some see passion in teaching as enthusiasm. Others define it as commitment to the profession and dedication to student learning. In Indonesia, as mentioned in Teacher Law No. 14/2005, passion in teaching is labelled as <a href="http://luk.staff.ugm.ac.id/atur/UU14-2005GuruDosen.pdf">“calling of the soul”</a>, or <em>panggilan jiwa</em>.</p>
<p>As <a href="http://jurnaldikbud.kemdikbud.go.id/index.php/jpnk/article/view/139">the popularity of the teaching profession in Indonesia is increasing</a>, how can the Indonesian government ensure the new batches of teachers have passion as stipulated in the 2005 Teacher Law?</p>
<p>Unfortunately, we have not found clear guidance from the government on how passion should be embedded in Indonesia’s teacher education, recruitment or professional development. </p>
<h2>How to nurture passion? Or is it nature?</h2>
<p><a href="http://jurnaldikbud.kemdikbud.go.id/index.php/jpnk/article/view/139">Previous studies</a> have shown that interest in the teaching profession has increased since the implementation of professional allowances. Thus, motivation to enter teaching colleges may not necessarily be driven purely by interest in becoming a teacher, or <em>panggilan jiwa</em>, but due to the (relatively) attractive salary. </p>
<p>As our interviewees suggested, a teacher who lacks interest may be harmful to students’ learning. Such a teacher will only ensure that they follow authorities’ instructions without considering students’ needs. </p>
<p>Therefore, it is critical for Indonesia’s teacher colleges and other relevant stakeholders to develop a system to identify the passionate one from the pool of applicants. Then, any amount of passion they have at the beginning of teacher education should “be fostered” through their teaching career. </p>
<p>Echoing an education scholar, <a href="http://research.acer.edu.au/cgi/viewcontent.cgi?article=1003&context=research_conference_2003">John Hattie</a>, our government should “focus on … identifying, esteeming and encouraging excellent teachers, wherever they may be”. Attracting and selecting passionate teacher college applicants is as important as nurturing and maintaining passions among teachers. Hence, the effort should be coherent and sustainable.</p>
<p>Interestingly, one of our informants, a school principal, argued that the previous teacher education programme, <em><a href="http://eprints.uny.ac.id/30481/">Sekolah Pendidikan Guru</a></em> (SPG), was “producing” a better quality of teacher. Compared to the current teacher education, a four-year college education, our interviewee argued that SPG provided more classroom experience in which mentors fostered enthusiasm for teaching and students. </p>
<p>Various sources support the idea that passion is essential in improving teacher quality. It is important to note, however, that passion is not a silver bullet in improving the quality of education. Passionate teachers still need supportive systems, such as <a href="http://journals.sagepub.com/doi/pdf/10.1177/0013161X86022003003">proper incentives</a>, <a href="http://www.sciencedirect.com/science/article/pii/S0742051X05001228">adequate preparation to deal with the demanding nature of the profession</a>, <a href="https://eric.ed.gov/?id=EJ997144">access to teacher networks and professional development</a>, as well as passionate colleagues and mentors. </p>
<p>Supportive systems will not only attract the best and most passionate people into teaching but also improve teachers’ qualities. At the end of the day, supportive systems will benefit both our teachers and students’ learning progress.</p><img src="https://counter.theconversation.com/content/85755/count.gif" alt="The Conversation" width="1" height="1" />
<p class="fine-print"><em><span>Heni Kurniasih is part of the research quoted in this article, which has been made possible through funding from Research on Improving Systems of Education (RISE). RISE is a multi-country research programme that is funded by the UK and Australian governments. The views expressed in this document are those of the authors and do not necessarily represent the views of the RISE Programme or its funders.</span></em></p><p class="fine-print"><em><span>Mirza Annisa Izati is part of the research quoted in this article, which has been made possible through funding from Research on Improving Systems of Education (RISE). RISE is a multi-country research programme that is funded by the UK and Australian governments. The views expressed in this document are those of the authors and do not necessarily represent the views of the RISE Programme or its funders.</span></em></p>Indonesia has allocated a huge percentage of education funding to improve the quality of teachers through various reforms. Yet their performance has not improved. What was missing?Heni Kurniasih, Research Associate, SMERU Research InstituteMirza Annisa Izati, Junior Researcher, SMERU Research InstituteLicensed as Creative Commons – attribution, no derivatives.tag:theconversation.com,2011:article/825582017-09-11T23:21:11Z2017-09-11T23:21:11ZWhy Canada fails to be an education superpower<figure><img src="https://images.theconversation.com/files/185531/original/file-20170911-8010-iu15w8.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=496&fit=clip" /><figcaption><span class="caption">Almost 10 per cent of Canadian 15-year-olds do not have the science proficiency level required to participate fully in society. </span> <span class="attribution"><span class="source">(Shutterstock)</span></span></figcaption></figure><p>Canada’s <a href="http://www.oecd.org/pisa/">recent results</a> in the Programme for International Student Assessment (<a href="https://www.cmec.ca/251/Programs-and-Initiatives/Assessment/Programme-for-International-Student-Assessment-(PISA)/Overview/index.html">PISA</a>) have been celebrated widely. In August 2017, the BBC marked Canada as an “<a href="http://www.bbc.com/news/business-40708421">education superpower</a>” because we were one of the top-ranking countries.</p>
<p>In fact, in-depth analysis indicates that our high school students’ performance has remained stagnant over the past decade. Even more troubling, almost 10 per cent of Canadian 15-year-olds do not have the science proficiency level required to participate fully in society. </p>
<p>And last week’s announcement of the <a href="https://www.timeshighereducation.com/world-university-rankings/2018/world-ranking#!/page/0/length/25/sort_by/rank/sort_order/asc/cols/stats">2018 Times Higher Education World University Rankings</a> places Canadian universities in a similar situation — <a href="https://beta.theglobeandmail.com/news/national/education/canadian-universities-hold-steady-in-rankings-but-must-boost-research-investment-to-rise/article36163082/?ref=http://www.theglobeandmail.com&">just clinging to their positions</a>. </p>
<p>As an assistant professor in the Werklund School of Education at the University of Calgary, I have been <a href="http://www.tandfonline.com/doi/abs/10.1080/15305058.2013.841702?needAccess=true&">working with PISA data</a> for many years. My research with PISA stems from an interest in educational assessment which began when I was a high school science teacher. One of my research streams focuses on investigating the <a href="https://journals.library.ualberta.ca/jcie/index.php/JCIE/article/download/21392/20114">different types of assessments</a> used across Canada.</p>
<h2>No improvements</h2>
<p>PISA is an international assessment of the skills and knowledge of 15-year-old students, a collaborative effort among member countries of the Organisation for Economic Co-operation and Development (<a href="http://www.oecd.org/">OECD</a>). In the last round of the assessment, 72 countries participated. </p>
<p>After the 2015 results were released, Canada was celebrated because only a handful of countries outperformed us in science, math and reading. When these results were compared to our past performance, it suggested Canadian students’ achievement improved over the years; particularly in the past three years.</p>
<p><br></p>
<p><iframe id="h4fMY" class="tc-infographic-datawrapper" src="https://datawrapper.dwcdn.net/h4fMY/3/" height="400px" width="100%" style="border: none" frameborder="0"></iframe></p>
<p>However, analysis of Canada’s PISA results indicates that our 15-year-olds’ results have not improved at all. When we look at Canadian PISA scores over the past few years, we see (as shown in Figure 2) that there are only minimal changes in each subject. Although students’ performance has fluctuated throughout the years, when we compare the <a href="https://www.cmec.ca/Publications/Lists/Publications/Attachments/365/PISA2015-CdnReport-EN.pdf">2015 results</a> to <a href="https://www.cmec.ca/docs/pisa2006/Pisa2006.en.pdf">those from 2006</a>, the results indicate that students’ performance has either decreased or stayed the same. </p>
<p>The PISA covers reading, mathematics and science. Each assessment includes questions from all three areas, but each year the focus area shifts. We compare 2015 with 2006 here because both are years in which science was the focus. </p>
<p><br></p>
<p><iframe id="8fhvH" class="tc-infographic-datawrapper" src="https://datawrapper.dwcdn.net/8fhvH/11/" height="400px" width="100%" style="border: none" frameborder="0"></iframe></p>
<p>Statistically, these differences are not significant which means these fluctuations may have occurred by chance and not due to actual decreases in student performance. When we consider the statistical insignificance and past trends, we can safely conclude that Canadian students have shown no improvements in any of the subjects over the past decade.</p>
<h2>Celebrating a decline</h2>
<p>If students’ scores indicate there are no improvement in students’ performance, then why do our rankings (shown in Figure 1) show that we have improved? The reason is because the rankings are depending on the performance of other countries in comparison to Canada. For example, students from Finland outperformed Canadian students in science during <a href="http://cmec.ca/Publications/Lists/Publications/Attachments/318/PISA2012_CanadianReport_EN_Web.pdf">the 2012 PISA</a>. But by 2015 our students performed at the same levels as their counterparts in Finland. </p>
<p>Canada’s rankings improved because students’ performance from Finland decreased over time. Hence, as we celebrate the improvements of Canada’s rankings, we are also celebrating a decline in the performance of students from other top-performing countries. </p>
<p>Instead of focusing on the country rankings, it is more appropriate to highlight students’ performance based on the scores achieved in each subject.</p>
<h2>Failing initiatives?</h2>
<p>The stability of Canadian students’ performance on PISA warrants concerns regarding the state of our education programs. Over the past decade, there have been many initiatives and programs implemented across Canada in an effort to improve students’ achievement. These include the <a href="http://www.edu.gov.on.ca/eng/literacynumeracy/">Literacy and Numeracy Secretariat</a> in Ontario and the <a href="http://www.uleth.ca/education/resources/research/research-centers/project-support">Alberta Initiative for School Improvement</a>.</p>
<p>However, the 2015 PISA results continue to indicate that almost one in 10 Canadian students do not meet the minimum level of science proficiency required to participate in society. Instead of celebrating Canada’s PISA results, now is the time to reflect upon them — to identify areas of weaknesses in our education programs so that we may enhance Canada’s education.</p>
<p>We need to ask ourselves: Why are our students failing to improve, especially in science?</p><img src="https://counter.theconversation.com/content/82558/count.gif" alt="The Conversation" width="1" height="1" />
<p class="fine-print"><em><span>Man-Wai Chu does not work for, consult, own shares in or receive funding from any company or organisation that would benefit from this article, and has disclosed no relevant affiliations beyond their academic appointment.</span></em></p>Canada’s educational performance internationally has remained stagnant over the past decade. Students’ science and math proficiency is especially worrying.Man-Wai Chu, Assistant Professor of Education, University of CalgaryLicensed as Creative Commons – attribution, no derivatives.tag:theconversation.com,2011:article/778472017-06-12T05:55:56Z2017-06-12T05:55:56ZInternational PISA tests show how evidence-based policy can go wrong<figure><img src="https://images.theconversation.com/files/172581/original/file-20170606-3668-2sx7rv.jpg?ixlib=rb-1.1.0&rect=0%2C0%2C2048%2C1434&q=45&auto=format&w=496&fit=clip" /><figcaption><span class="caption">Chemistry class at the Dong Tien Secondary School, Thai Nguyen Province, Vietnam</span> <span class="attribution"><a class="source" href="https://www.flickr.com/photos/asiandevelopmentbank/8425369255/in/photolist-dQxj92-dQuw8R-dQzMjo-ffBWwP-dQwnJp-dQwdKa-dQwvgF-ffSbkb-dQxjd4-dQA7gN-dQwHb6-ffSbJS-eLHRXh-dQxj5H-Jj6Vhj-A9VoHR-GStjue-BmEwzk-HN9h86-pjGSzi-ybAKG5-HN9jVF-GSko2m-yqUjcd-t5iFWu-t5iCvm-ytdExp-J3u6qA-U3bMN7-U3bKQ9-SZLejQ-U3bBt9-T3yQv6-TtEvao-dr3MKf-AXEAuY-TivaTG-TxikhK-Txik7p-TtEupq-TtEuhS-SXhKfL-TtEubE-TxijSX-SXhKe3-SiqpPM-Tiv9md-Txijp2-SzLvau-SKTBAA">Asian Development Bank/flickr</a>, <a class="license" href="http://creativecommons.org/licenses/by-sa/4.0/">CC BY-SA</a></span></figcaption></figure><p><a href="http://www.oecd.org/pisa/aboutpisa/">The Programme for International Student Assessment</a> (PISA) implemented by the OECD has been controversial since the publication of its first results in 2000.</p>
<p>Measuring the mathematics, science and reading skills of 15-year-old students every three years, PISA relies on broad international participation. In the 2015 test, as many as <a href="https://nces.ed.gov/surveys/pisa/countries.asp">72</a> countries joined the exercise, including those outside the OECD. </p>
<p>It’s common to find articles where PISA is presented as a measure of a country’s <a href="https://www.opendemocracy.net/neweconomics/why-the-uk-must-invest-in-innovation-a-view-from-japan/">innovation and growth potential</a>. But it’s also not rare to find others where the metrics used are <a href="https://www.opendemocracy.net/ourkingdom/jeremy-fox/are-pisa-scores-really-that-important">contested</a> as irrelevant and potentially counter-productive.</p>
<figure class="align-center ">
<img alt="" src="https://images.theconversation.com/files/172549/original/file-20170606-3710-1cnxc54.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=754&fit=clip" srcset="https://images.theconversation.com/files/172549/original/file-20170606-3710-1cnxc54.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=600&h=988&fit=crop&dpr=1 600w, https://images.theconversation.com/files/172549/original/file-20170606-3710-1cnxc54.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=600&h=988&fit=crop&dpr=2 1200w, https://images.theconversation.com/files/172549/original/file-20170606-3710-1cnxc54.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=600&h=988&fit=crop&dpr=3 1800w, https://images.theconversation.com/files/172549/original/file-20170606-3710-1cnxc54.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=754&h=1241&fit=crop&dpr=1 754w, https://images.theconversation.com/files/172549/original/file-20170606-3710-1cnxc54.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=754&h=1241&fit=crop&dpr=2 1508w, https://images.theconversation.com/files/172549/original/file-20170606-3710-1cnxc54.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=754&h=1241&fit=crop&dpr=3 2262w" sizes="(min-width: 1466px) 754px, (max-width: 599px) 100vw, (min-width: 600px) 600px, 237px">
<figcaption>
<span class="caption">PISA results for 2015.</span>
<span class="attribution"><a class="source" href="https://pbs.twimg.com/media/C6fII-KU0AEe20v.jpg">OECD</a></span>
</figcaption>
</figure>
<p>For advocates of “evidence-based” or “informed” policy, PISA incarnates the dispassionate, objective facts that nourish the formulation of good approaches – in this case, in the field of education. It allows for country comparisons and can help identify good practices that are worth emulating.</p>
<p><a href="https://www.amazon.com/PISA-According-Schulpadagogik-Padagogische-Psychologie/dp/3825809463">Opponents</a> of the program reject the choice made by the OECD to link education and economic growth. While this choice was explained at the beginning as a way of showing <a href="https://www.oecd.org/pisa/44417824.pdf">the high cost of a low educational performance</a>, for some it also embodies <a href="https://www.theguardian.com/education/2014/may/06/oecd-pisa-tests-damaging-education-academics">a neoliberal framing of education policies</a>, which <a href="http://onlinelibrary.wiley.com/wol1/doi/10.1002/berj.3120/full">forces the sector into the context of globalisation</a>.</p>
<h2>Post-normal science</h2>
<p>In a new <a href="http://www.emeraldinsight.com/doi/full/10.1108/IJCED-12-2016-0023">review study</a> in the <a href="http://www.emeraldgrouppublishing.com/products/journals/journals.htm?id=ijced">International Journal of Comparative Education and Development</a> that I co-authored, we consider that facts such as those produced by PISA can be viewed through the lense of <a href="http://www.sciencedirect.com/science/article/pii/001632879390022L">post-normal science</a>. This approach is particularly apt for assessing scientific evidence when it feeds the policy process. </p>
<p>Post-normal science is a problem-solving strategy for issues where “facts are uncertain, values in dispute, stakes high and decisions urgent”. The concept was <a href="https://www.uu.nl/wetfilos/wetfil10/sprekers/Funtowicz_Ravetz_Futures_1993.pdf">created</a> in the 1990s by <a href="https://theconversation.com/profiles/silvio-oscar-funtowicz-312680/articles">Silvio Funtowicz</a> and Jerome R. Ravetz. </p>
<p>A key idea of the schema is “extended participation”, which suggests opening analyses to experts from different disciplines and forms of scholarship (one of the demands of <a href="https://www.theguardian.com/education/2014/may/06/oecd-pisa-tests-damaging-education-academics">PISA’s critics</a>). It also points toward the active participation of relevant and legitimate stakeholders.</p>
<p>According to our review, a post-normal science reading of the PISA survey and its implications identifies a rich mix of methodological and ideological issues – in keeping with its tenet that the distinction between facts and values becomes problematic when the stakes are high. </p>
<p>PISA gives scores to participating countries so they can be ranked from best to worst for the skills measured, as well as measuring how they stand globally over all skills. Too much importance is being given to these scores and rankings, given the many non-transparent assumptions made by the OECD in their construction. </p>
<p>We don’t know, for example, how choices are made to include or exclude questions. There is also an issue about how many and which students participate in the test. The latter issue generates the so-called “non-response bias” and noticeably affects results. </p>
<p>Our review discusses the case of PISA non-response for England, where the bias turned out to be <a href="https://eprints.soton.ac.uk/79403/">twice the size</a> of the OECD declared standard error in 2003. This case illustrates how the results are much more uncertain and dependent on non-educational variables than it appears on a superficial reading. </p>
<p>In other words, the margin of error on the scores provided by the OECD is underestimated, and the ranking of countries from best to worst is more open to interpretation than one would understand from OECD analyses. To resolve this issue, the OECD should provide PISA users with a structured <a href="http://www.ophi.org.uk/wp-content/uploads/Saisana-Saltelli-Tarantola-2005.pdf">sensitivity analysis</a> that takes all the variables in the ranking into account. </p>
<p>Ideally this analysis could be made by the users themselves, but this would only be possible if the OECD made all data available, which is not the case at present.<br>
The worldview selected for the PISA analysis is also contentious. A main issue with PISA’s ambition to measure life skills needed to function in knowledge societies, for example, is that these skill are assumed to be the same across countries and cultures. Nor is it clear that all societies can safely be assumed to be destined to become “knowledge-oriented”. </p>
<p>Other fundamental questions emerged from our reading, too. Is it acceptable to see education as an input into growth? Does PISA “flatten” curricula – <a href="https://www.theguardian.com/education/2014/may/06/oecd-pisa-tests-damaging-education-academics">narrowing our collective imagination regarding what education is and ought to be about</a> – and encourage focusing on a subset of educational topics at the expenses of others? </p>
<p>As noted by <a href="http://www.emeraldinsight.com/doi/full/10.1108/IJCED-12-2016-0023">our study</a>, country comparison is achieved by “ignoring the great diversity of curricula across the participating countries – diversity which might in fact be a source of country-specific creativity and well-being”.</p>
<p><div data-react-class="Tweet" data-react-props="{"tweetId":"501674933618237440"}"></div></p>
<h2>Urgency and caution</h2>
<p>The PISA controversy is a helpful reminder that citizens in democracies must be critical of the facts that feed into public discourse. This <a href="https://theconversation.com/to-tackle-the-post-truth-world-science-must-reform-itself-70455">predates</a> any alarm about the emergence of the purported post-truth society, though that has made these discussions are <a href="https://thebreakthrough.org/index.php/journal/past-issues/issue-3/post-truth-pluralism">more urgent</a>. </p>
<p>Facts must be taken with caution. For example, PISA scores have supported such inferences as <a href="http://www.eenee.de/dms/EENEE/Analytical_Reports/EENEE_AR20.pdf">this paragraph in a 2014 study</a> prepared for the European Commission:</p>
<blockquote>
<p>If every EU Member State achieved an improvement of 25 points in its PISA score (which is what for example Germany and Poland achieved over the last decade), the GDP of the whole EU would increase by between 4% and 6% by 2090; such an 6% increase would correspond to 35 trillion Euro.</p>
</blockquote>
<p>The authoritative tone and use of crisp numbers here suggest causality – from education to growth – and an air of accuracy in a claim that is more like an act of faith than the result of scientific processes.</p>
<p><div data-react-class="Tweet" data-react-props="{"tweetId":"582921662632488960"}"></div></p>
<p>Our review of the PISA controversy also highlighted a problem of power in the use of evidence. With PISA, the OECD – an international organisation composed of unelected officers and scholars – has constructed a neoliberal framing of education policy and used its authority to dominate the global conversation, potentially at the expense of national or regional authorities and institutions. </p>
<p>This “global super-ministry of education”, in the words of an educator quoted in our study, effectively marginalises alternative visions of education that would normally hold weight. Thus the idea of education as personal development and fulfilment, what Germans call <em>Bildung</em>, becomes invisible, because it cannot be used as an internationally comparable metric.</p>
<h2>Different skills</h2>
<p>A full discussion of all points of controversy would take more space than this contribution allows, and should touch on the tension in using metrics to appreciate cognitive skills, as well as the need for other skills, such as <a href="http://www.missionhighbook.com/">critical thinking, intrinsic motivation, resilience, self-management, resourcefulness, and relationship-building</a>.</p>
<p>The OECD is unlikely to <a href="https://www.theguardian.com/education/2014/may/06/oecd-pisa-tests-damaging-education-academics">suspend</a> or abolish the PISA study, in part because it serves a function. Before PISA, a country’s educational development was approximated by the average number of years of schooling there. </p>
<p>PISA raised awareness of other factors beyond classroom hours, such as literacy, that affect students’ educational outcomes. For those who study education, standardized tests like PISA also offer a useful instrument for comparing within and among countries.</p>
<p><div data-react-class="InstagramEmbed" data-react-props="{"url":"https://www.instagram.com/p/BTERE-HFFA5","accessToken":"127105130696839|b4b75090c9688d81dfd245afe6052f20"}"></div></p>
<p>Still, our study reinforces that democratic societies view “evidence-based policy” with a critical eye, querying who produced the evidence and whose interests are served by it. PISA is a strong example of the <a href="http://www.sciencedirect.com/science/article/pii/S0016328717300472">power asymmetries</a> inherent in producing facts to inform policy. </p>
<p>In this case the OECD, possibly the most muscular player in the arena of international education policy, can frame evidence around its preferred norms and impose them on public discourse.</p><img src="https://counter.theconversation.com/content/77847/count.gif" alt="The Conversation" width="1" height="1" />
<p class="fine-print"><em><span>Andrea Saltelli does not work for, consult, own shares in or receive funding from any company or organisation that would benefit from this article, and has disclosed no relevant affiliations beyond their academic appointment.</span></em></p>How can we improve the PISA standardised tests?Andrea Saltelli, Adjunct Professor Centre for the Study of the Sciences and the Humanities, University of Bergen, University of BergenLicensed as Creative Commons – attribution, no derivatives.tag:theconversation.com,2011:article/702912016-12-14T09:12:10Z2016-12-14T09:12:10ZPISA global education rankings are the road to ruin – here’s why<p>No doubt twitchy politicians and nervous bureaucrats are still digesting the triennial <a href="https://www.oecd.org/pisa/pisa-2015-results-in-focus.pdf">OECD test scores</a> on international educational achievement. From <a href="http://www.bbc.co.uk/news/education-38157811">all</a> the <a href="http://www.abc.net.au/news/2016-12-06/australian-school-performance-in-absolute-decline-globally/8098028">media coverage</a>, one thing is abundantly clear: this arithmetic continues its rise as vehicle of choice for social, economic and cultural improvement worldwide. </p>
<p>Often considered disinterested and objective, the new scores in the Programme for International Student Assessment (PISA) showed the UK and US making no progress from stodgy mid-table; Australia, Sweden and Scotland among those in decline; Russia and Poland improving; and east Asian systems like Singapore, Hong Kong and Shanghai remaining streets ahead of almost everywhere else. </p>
<p>The PISA gives two-hour multiple choice tests to 15-year-olds in 64 countries or economic regions, measuring their aptitude in maths, reading and science. First introduced in 2000, the scores are used to cajole and influence political and public sentiment and manipulate policy. </p>
<p>And what can possibly be wrong with knowing as much as we can about the educational performance of high school students? If you can count it, it must be real and if it’s real, surely it reflects some reality. </p>
<p>Andreas Schleicher, the OECD’s director for education and skills and architect of the PISA report, once <a href="http://www.theatlantic.com/magazine/archive/2011/07/the-worlds-schoolmaster/308532/">opined that</a> without data “you are just another person with an opinion”. This is palpable nonsense and matters are rarely that simple. Rather like quantum observations, attempting to observe one property of an object often misses another property that can be equally or more important. </p>
<h2>Numbers and reality</h2>
<p>If Schleicher meant “data” in a comprehensive sense the discussion might be different but he really means numbers – specific ones which value data recall over other forms of knowing. Of course, such a non-verifiable proposition as the PISA as a proxy for an educational system and participants’ skills reinforces the status of someone whose raison d’être is promoting the arithmetic as a solution to the global(ised) challenges of education.</p>
<p>Existing in the actual world of lived experience rather than the virtual world of OECD bureaucrats, however, might make us cautious. We have reached a stage where governments govern according to ideological obsessions and predispositions, and populations behave in defiance of numbers. </p>
<p>Take <a href="http://www.bbc.co.uk/news/uk-politics-32810887">Brexit</a> and <a href="http://www.bbc.co.uk/news/election/us2016">Donald Trump</a>. The numbers suggest it is inadvisable at best to leave the EU; and that Trump is unlikely to be able or willing to make the changes to trade and taxation necessary to benefit dispossessed American workers. People voted for them anyway – against their apparent interests. </p>
<figure class="align-center zoomable">
<a href="https://images.theconversation.com/files/149664/original/image-20161212-26056-6a8lq0.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=1000&fit=clip"><img alt="" src="https://images.theconversation.com/files/149664/original/image-20161212-26056-6a8lq0.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=754&fit=clip" srcset="https://images.theconversation.com/files/149664/original/image-20161212-26056-6a8lq0.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=600&h=400&fit=crop&dpr=1 600w, https://images.theconversation.com/files/149664/original/image-20161212-26056-6a8lq0.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=600&h=400&fit=crop&dpr=2 1200w, https://images.theconversation.com/files/149664/original/image-20161212-26056-6a8lq0.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=600&h=400&fit=crop&dpr=3 1800w, https://images.theconversation.com/files/149664/original/image-20161212-26056-6a8lq0.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=754&h=503&fit=crop&dpr=1 754w, https://images.theconversation.com/files/149664/original/image-20161212-26056-6a8lq0.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=754&h=503&fit=crop&dpr=2 1508w, https://images.theconversation.com/files/149664/original/image-20161212-26056-6a8lq0.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=754&h=503&fit=crop&dpr=3 2262w" sizes="(min-width: 1466px) 754px, (max-width: 599px) 100vw, (min-width: 600px) 600px, 237px"></a>
<figcaption>
<span class="caption">‘Good morning, America.’</span>
<span class="attribution"><a class="source" href="https://www.shutterstock.com/pic-460918720/stock-photo-new-york-jul-16-2016-donald-trump-speaks-during-a-press-conference-on-july-16-2016-in-new-york.html?src=hSNzfW5w-BB0GQ6X4Ym9nw-1-20">JStone</a></span>
</figcaption>
</figure>
<p>In the UK, to take another example, politicians have consistently promulgated teaching reading by comprehensively adopting synthetic phonics, where you start with basic (artificial) sounds and gradually build up to reading words and sentences. This despite a government-commissioned <a href="https://czone.eastsussex.gov.uk/sites/gtp/library/core/english/Documents/phonics/A%20Systematic%20Review%20of%20the%20Research%20Literature%20on%20the%20Use%20of%20Phonics%20in%20the%20Teaching%20of%20Reading%20and%20Spelling.pdf">report</a> that suggested it offered no advantages over other forms of phonetic reading. </p>
<p>More recently UK Prime Minister Theresa May’s obsession with reintroducing grammar schools blithely disregarded <a href="https://theconversation.com/why-the-ghost-of-grammar-schools-keeps-on-haunting-us-63679">the numbers</a>. They have consistently indicated that grammar schools are ineffective vehicles for social mobility or raising attainment. </p>
<p>These examples suggest that other things are at play apart from the facts and that the numbers are often deployed to pursue policies that the numbers themselves don’t support. And in a world of political lies – let’s not call it <a href="https://en.oxforddictionaries.com/word-of-the-year/word-of-the-year-2016">post-truth</a> – media and politicians have created a prudential and moral equivalence between arguments, irrespective of their quality. </p>
<p>It is as if every proposition was as good or bad as every other and only to be discriminated with reference to the arithmetic. And Schleicher and his scoring system contribute significantly to this corrosive culture through the PISA promotion of number scales on a limited set of assessment instruments. </p>
<h2>Schleicher’s list</h2>
<p>Despite the tortured relationship between the arithmetic and policy, governments seem increasingly cowed by the PISA process. The US government invoked these results to justify introducing its “<a href="https://www.whitehouse.gov/issues/education/k-12/race-to-the-top">Race to the Top</a>” programme, for example. This substantially increased assessment, tests and rankings for teachers and students, and forced a false equivalence between very different institutions. It created short-term learning objects of what Beatrix Potter <a href="http://www.mainlesson.com/display.php?author=potter&book=tailor&story=tailor">might have called</a> “snippets and tippets”, rather than a coherent account of the object of study.</p>
<p>Perhaps worse, PISA is deployed in an unending process of “correcting educators’ mistakes”. Thus Michael Gove, the former UK education secretary, <a href="http://www.theewf.org/news/2011/summary-of-the-education-world-forum-2011">introduced</a> his white paper on education in 2011 by explicitly suggesting his urgent intervention to reshape English education was driven by the PISA comparisons. </p>
<p>Underlying the OECD rhetoric is a belief in a singular account of professional education that can be unearthed by the numbers. In truth, high scorers like Singapore, Canada, Hong Kong, Finland, China and Japan have little in common structurally or pedagogically – except perhaps a genuine reverence for education, unlike the UK. </p>
<figure class="align-center zoomable">
<a href="https://images.theconversation.com/files/149663/original/image-20161212-26056-p3t9yf.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=1000&fit=clip"><img alt="" src="https://images.theconversation.com/files/149663/original/image-20161212-26056-p3t9yf.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=754&fit=clip" srcset="https://images.theconversation.com/files/149663/original/image-20161212-26056-p3t9yf.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=600&h=398&fit=crop&dpr=1 600w, https://images.theconversation.com/files/149663/original/image-20161212-26056-p3t9yf.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=600&h=398&fit=crop&dpr=2 1200w, https://images.theconversation.com/files/149663/original/image-20161212-26056-p3t9yf.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=600&h=398&fit=crop&dpr=3 1800w, https://images.theconversation.com/files/149663/original/image-20161212-26056-p3t9yf.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=754&h=501&fit=crop&dpr=1 754w, https://images.theconversation.com/files/149663/original/image-20161212-26056-p3t9yf.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=754&h=501&fit=crop&dpr=2 1508w, https://images.theconversation.com/files/149663/original/image-20161212-26056-p3t9yf.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=754&h=501&fit=crop&dpr=3 2262w" sizes="(min-width: 1466px) 754px, (max-width: 599px) 100vw, (min-width: 600px) 600px, 237px"></a>
<figcaption>
<span class="caption">Educating Singapore.</span>
<span class="attribution"><a class="source" href="https://www.shutterstock.com/pic-34061026/stock-photo-singapore-july-11-students-pointing-towards-the-sky-as-the-red-lion-parachute-team-dives-during-national-day-parade-combine-rehearsal-july-11-2009-in-singapore.html?src=GQtMEoBqt5rihQ4g_Ods3w-1-14">Jordan Tan</a></span>
</figcaption>
</figure>
<p>The OECD also fails to tell us that the numbers are loaded from the outset. They foreground some things at the expense of others. The drive for a common set of test items fails to take into account the subtle relationship between, say, acquiring number skills and the creative uses to which they can be put. </p>
<p>Not all test items are applied to every student, the numbers are tiny and selective, and <a href="http://folk.uio.no/sveinsj/Sjoberg-PISA-book-2007.pdf">there is</a> poor pre-testing practice. Ironically one of the most significant indictments of the PISA is seen in the US and the UK. Despite their mediocre performance, they remain immensely creative and usually venues of choice for university education.</p>
<p>Even in the China Daily, a <a href="http://usa.chinadaily.com.cn/world/2016-12/08/content_27604835.htm">recent article</a> on certain tier-one PISA performers conceded education was more than the sum of a few selective numbers. The problem is not that the numbers tell us nothing. Apart from the many methodological flaws, the big challenge is, they don’t tell us much – and certainly nowhere near as much as politicians, press, pundits and sundry hangers on would have us believe.</p><img src="https://counter.theconversation.com/content/70291/count.gif" alt="The Conversation" width="1" height="1" />
<p class="fine-print"><em><span>James Conroy receives funding from the AHRC, ESRC and the Templeton Foundation.</span></em></p>The direct line between world education policy and Donald Trump looks like this.James Conroy, Professor of Religious and Philosophical Education and Vice Principal, Internationalisation, University of GlasgowLicensed as Creative Commons – attribution, no derivatives.