tag:theconversation.com,2011:/es/topics/university-feedback-14847/articlesUniversity feedback – The Conversation2021-08-18T19:53:25Ztag:theconversation.com,2011:article/1657572021-08-18T19:53:25Z2021-08-18T19:53:25ZFeedback from supervisors can be a good or bad experience. Here’s how to get it right<figure><img src="https://images.theconversation.com/files/416450/original/file-20210817-14-1qrekus.jpg?ixlib=rb-1.1.0&rect=54%2C0%2C6088%2C4010&q=45&auto=format&w=496&fit=clip" /><figcaption><span class="caption">
</span> <span class="attribution"><a class="source" href="https://www.pexels.com/photo/focused-man-working-with-female-colleague-in-office-5325104/">Anna Shvets/Pexels</a>, <a class="license" href="http://artlibre.org/licence/lal/en">FAL</a></span></figcaption></figure><p>Giving good feedback is an art. It can be challenging for supervisors and managers, whether in an educational setting or any other workplace. Our <a href="https://www.tandfonline.com/doi/full/10.1080/02602938.2021.1955241">newly published review</a> of the past decade’s research on this issue confirms the key elements of improving feedback are to make it meaningful, constructive, timely and regular.</p>
<p>Feedback is centred on <a href="https://onlinelibrary.wiley.com/doi/abs/10.1002/bs.3830280103">giving information</a> about actual performance against set requirements. Good feedback enables people to learn from both successes and weaknesses in performance. </p>
<p>Focusing only on people’s shortcomings does not help learning, but <a href="https://hbr.org/2019/03/the-feedback-fallacy?autocomplete=true">hinders</a> it. Bad feedback can be <a href="https://www.sciencedirect.com/science/article/pii/S0007681317300253?casa_token=TkC2_OE8qEUAAAAA:YKvpw-RtnZq-F7FygwAWz_HHAyyEizfVY5uKFD_AFmYFvS_WdOLx7NrRAMDACoFWAym9nfLPxg">destructive</a>.</p>
<hr>
<p>
<em>
<strong>
Read more:
<a href="https://theconversation.com/our-uni-teachers-were-already-among-the-worlds-most-stressed-covid-and-student-feedback-have-just-made-things-worse-162612">Our uni teachers were already among the world's most stressed. COVID and student feedback have just made things worse</a>
</strong>
</em>
</p>
<hr>
<p>An earlier review found <a href="https://psycnet.apa.org/record/1996-02773-003">one feedback intervention out of every three</a> actually decreased performance. Postgraduate students’ experiences of feedback from research supervisors mirrors employees’ experiences of <a href="https://www.gallup.com/workplace/249332/harm-good-truth-performance-reviews.aspx">feedback from managers</a>. Our <a href="https://www.tandfonline.com/doi/full/10.1080/02602938.2021.1955241">analysis</a> of the past decade of academic literature on feedback to postgraduate research students confirms the problem is widespread. </p>
<p>And large numbers of people are affected. Australia has more than <a href="https://www.dese.gov.au/higher-education-statistics/resources/2019-section-2-all-students">66,500</a> higher degree research students. In the US, <a href="https://ncses.nsf.gov/pubs/nsf21308/report/executive-summary">55,703</a> doctorates were awarded in 2019. </p>
<p>Poor feedback to such students leads to a negative experience. But there is not one feedback strategy that works positively for all situations.</p>
<figure class="align-center ">
<img alt="Young woman smiles as she gets advice from another woman" src="https://images.theconversation.com/files/416460/original/file-20210817-6755-1fvkqxs.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=754&fit=clip" srcset="https://images.theconversation.com/files/416460/original/file-20210817-6755-1fvkqxs.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=600&h=400&fit=crop&dpr=1 600w, https://images.theconversation.com/files/416460/original/file-20210817-6755-1fvkqxs.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=600&h=400&fit=crop&dpr=2 1200w, https://images.theconversation.com/files/416460/original/file-20210817-6755-1fvkqxs.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=600&h=400&fit=crop&dpr=3 1800w, https://images.theconversation.com/files/416460/original/file-20210817-6755-1fvkqxs.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=754&h=503&fit=crop&dpr=1 754w, https://images.theconversation.com/files/416460/original/file-20210817-6755-1fvkqxs.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=754&h=503&fit=crop&dpr=2 1508w, https://images.theconversation.com/files/416460/original/file-20210817-6755-1fvkqxs.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=754&h=503&fit=crop&dpr=3 2262w" sizes="(min-width: 1466px) 754px, (max-width: 599px) 100vw, (min-width: 600px) 600px, 237px">
<figcaption>
<span class="caption">Effective feedback is built on a relationship of trust, with the supervisor often likened to a ‘critical friend’.</span>
<span class="attribution"><a class="source" href="https://www.shutterstock.com/image-photo/happy-millennial-intern-student-girl-talking-2007206843">Shutterstock</a></span>
</figcaption>
</figure>
<hr>
<p>
<em>
<strong>
Read more:
<a href="https://theconversation.com/universities-are-failing-their-students-through-poor-feedback-practices-86756">Universities are failing their students through poor feedback practices</a>
</strong>
</em>
</p>
<hr>
<h2>What are the common problems?</h2>
<p>Our study found the problems in giving and receiving feedback related to content, process, people and expectations.</p>
<p>Low-quality feedback with <a href="https://www.emerald.com/insight/content/doi/10.1108/CDI-08-2014-0107/full/html">inadequate information</a> or <a href="https://www.emerald.com/insight/content/doi/10.1108/SHR-03-2017-0016/full/html#sec002">vague content</a> from managers does not lead to better work performance. Equally, managers and supervisors need to find a good balance between overwhelming their supervisees with <a href="https://www.sciencedirect.com/science/article/pii/S0749597811000513?casa_token=SYqYnHT-nR0AAAAA:z6uT8P-ESrlIT6YN6igm6Dl8Kz8YBfJXJfKiKL54TBA5hZOR-78kDWsZ7oYPeJJF7SSlpqcv_A">too much feedback</a> and not providing enough or <a href="https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3215129">infrequent and delayed</a> feedback.</p>
<p>Feedback does not stand alone – it is part of the broader relationship between supervisor and supervisee. A <a href="https://www.talent-quarterly.com/is-hr-missing-the-point-on-performance-feedback/">lack of trust</a> is harmful for the giving and receiving of feedback.</p>
<p>Feedback is a two-way process between the giver and receiver – both parties contribute to the experience. Some individuals <a href="https://www.tandfonline.com/doi/pdf/10.1080/02602938.2020.1733491?casa_token=KuqyVnaWFkMAAAAA:Oj77mPUlwHmNSjmgTS0fwpGzktwgboH9RHMn0ZoBdsZhyapL5iOaM1k1k5Km0mGdeuDMXwrE5Usy">actively seek</a> feedback. Others try to avoid it at all costs.</p>
<p>Not all feedback receivers are willing to <a href="https://www.tandfonline.com/doi/pdf/10.1080/09585192.2018.1443961?casa_token=s5fiAz5ny8kAAAAA:7SHyB6veptUqmD-uVKLTd3ij_wrPklAAhUr-0RdSU52hdm7eogovytYiYA5dG6QNlWVEHKtK4YI6">take feedback on board</a>. On the other hand, many feedback givers lack appropriate <a href="https://journals.lww.com/nursingmanagement/fulltext/2016/05000/give_the_gift_of_feedback.11.aspx?casa_token=7Wdz0pIKiZ8AAAAA:OPFPMCqz9oj0vIECz5fqqkKaczDQKa-6X7MidS3DAAx1FlsFhk-oXRTN4ONydhXoNMIWPrxHnnDKFpIWpvW7ZZzT">feedback skills</a> or awareness of their own style of feedback, including its timing and tone. Often, feedback is less than effective because of a mismatch of expectations between givers and receivers.</p>
<figure class="align-center ">
<img alt="Chart showing 5 sources of problems with feedback" src="https://images.theconversation.com/files/416442/original/file-20210817-23-ykmda3.png?ixlib=rb-1.1.0&q=45&auto=format&w=754&fit=clip" srcset="https://images.theconversation.com/files/416442/original/file-20210817-23-ykmda3.png?ixlib=rb-1.1.0&q=45&auto=format&w=600&h=336&fit=crop&dpr=1 600w, https://images.theconversation.com/files/416442/original/file-20210817-23-ykmda3.png?ixlib=rb-1.1.0&q=30&auto=format&w=600&h=336&fit=crop&dpr=2 1200w, https://images.theconversation.com/files/416442/original/file-20210817-23-ykmda3.png?ixlib=rb-1.1.0&q=15&auto=format&w=600&h=336&fit=crop&dpr=3 1800w, https://images.theconversation.com/files/416442/original/file-20210817-23-ykmda3.png?ixlib=rb-1.1.0&q=45&auto=format&w=754&h=422&fit=crop&dpr=1 754w, https://images.theconversation.com/files/416442/original/file-20210817-23-ykmda3.png?ixlib=rb-1.1.0&q=30&auto=format&w=754&h=422&fit=crop&dpr=2 1508w, https://images.theconversation.com/files/416442/original/file-20210817-23-ykmda3.png?ixlib=rb-1.1.0&q=15&auto=format&w=754&h=422&fit=crop&dpr=3 2262w" sizes="(min-width: 1466px) 754px, (max-width: 599px) 100vw, (min-width: 600px) 600px, 237px">
<figcaption>
<span class="caption"></span>
<span class="attribution"><a class="source" href="https://www.tandfonline.com/doi/full/10.1080/02602938.2021.1955241">Source: Chugh et al, Supervisory feedback to postgraduate research students: a literature review (2021). Image: Shutterstock</a>, <span class="license">Author provided</span></span>
</figcaption>
</figure>
<hr>
<p>
<em>
<strong>
Read more:
<a href="https://theconversation.com/how-our-obsession-with-performance-is-changing-our-sense-of-self-120212">How our obsession with performance is changing our sense of self</a>
</strong>
</em>
</p>
<hr>
<h2>The need for a ‘critical friend’</h2>
<p>Providing effective feedback is essential to improve learning and performance. Managers and research supervisors <a href="https://www.proquest.com/docview/2173863943">continually</a> give and receive feedback. But, before giving feedback, supervisors should manage expectations and negotiate <a href="https://www.tandfonline.com/doi/full/10.1080/03075079.2014.914906">supervision arrangements</a>. These include how often and when to give feedback, as well as the length and depth of feedback content.</p>
<p>In all organisations, supervisors should aim for a <a href="https://psycnet.apa.org/record/2012-03641-003">positive supervisory relationship</a>. Such relationships are based on trust, respect, open communication and shared meaning.</p>
<p>Supervisors’ style of feedback often <a href="http://www.scielo.org.za/scielo.php?script=sci_arttext&pid=S0038-23532015000600023">parallels</a> their own experiences, whether it was helpful or not. As feedback can often be misunderstood, supervisors should critically reflect on their feedback style so it becomes a satisfying two-way process.</p>
<p>Constructive regular feedback should highlight both <a href="https://hbr.org/2019/05/what-good-feedback-really-looks-like">strengths and weaknesses</a>. It should also suggest improvements. <a href="https://hbr.org/2014/01/your-employees-want-the-negative-feedback-you-hate-to-give">Fifty-seven percent</a> of employees prefer to hear corrective feedback that provides suggestions for improvement and points out things that weren’t done optimally.</p>
<p>So, supervisors can assume the role of a “critical friend” who is encouraging and supportive but provides candid feedback on performance.</p>
<p>Using technologies such as videoconferencing, messaging, social media and email can help in providing timely feedback.</p>
<p>Our <a href="https://www.tandfonline.com/doi/full/10.1080/02602938.2021.1955241">review</a> sums up the research findings on the characteristics of effective feedback as:</p>
<blockquote>
<p>“suggestive and constructive, brief, frequent and regular, actionable, specific and tailored, explicit, honest but empathetic and tactful, formal, supportive and encouraging, advising, appreciative and respectful but critical”.</p>
</blockquote>
<figure class="align-center ">
<img alt="chart showing 5 strategies for successful feedback" src="https://images.theconversation.com/files/416448/original/file-20210817-23-sew51.png?ixlib=rb-1.1.0&q=45&auto=format&w=754&fit=clip" srcset="https://images.theconversation.com/files/416448/original/file-20210817-23-sew51.png?ixlib=rb-1.1.0&q=45&auto=format&w=600&h=327&fit=crop&dpr=1 600w, https://images.theconversation.com/files/416448/original/file-20210817-23-sew51.png?ixlib=rb-1.1.0&q=30&auto=format&w=600&h=327&fit=crop&dpr=2 1200w, https://images.theconversation.com/files/416448/original/file-20210817-23-sew51.png?ixlib=rb-1.1.0&q=15&auto=format&w=600&h=327&fit=crop&dpr=3 1800w, https://images.theconversation.com/files/416448/original/file-20210817-23-sew51.png?ixlib=rb-1.1.0&q=45&auto=format&w=754&h=411&fit=crop&dpr=1 754w, https://images.theconversation.com/files/416448/original/file-20210817-23-sew51.png?ixlib=rb-1.1.0&q=30&auto=format&w=754&h=411&fit=crop&dpr=2 1508w, https://images.theconversation.com/files/416448/original/file-20210817-23-sew51.png?ixlib=rb-1.1.0&q=15&auto=format&w=754&h=411&fit=crop&dpr=3 2262w" sizes="(min-width: 1466px) 754px, (max-width: 599px) 100vw, (min-width: 600px) 600px, 237px">
<figcaption>
<span class="caption"></span>
<span class="attribution"><a class="source" href="https://www.tandfonline.com/doi/full/10.1080/02602938.2021.1955241">Source: Chugh et al, Supervisory feedback to postgraduate research students: a literature review (2021). Image: Shutterstock</a>, <span class="license">Author provided</span></span>
</figcaption>
</figure>
<hr>
<p>
<em>
<strong>
Read more:
<a href="https://theconversation.com/informal-feedback-we-crave-it-more-than-ever-and-dont-care-who-its-from-138932">Informal feedback: we crave it more than ever, and don't care who it's from</a>
</strong>
</em>
</p>
<hr>
<h2>A 3-way process of improving feedback</h2>
<p>Improving the feedback environment can lead to benefits that include <a href="https://iaap-journals.onlinelibrary.wiley.com/doi/pdf/10.1111/j.1464-0597.2006.00253.x">higher</a> work satisfaction. For example, in higher education, the triad of institutions, supervisors and students/supervisees can all help <a href="https://onlinelibrary.wiley.com/doi/pdf/10.1111/j.1365-2834.2011.01361.x?">improve</a> feedback processes. The same is true of the <a href="https://psycnet.apa.org/record/2015-44290-001">triad</a> of the organisation, supervisors/managers and employees in other workplaces. Each has a role to play in making feedback effective.</p>
<p>Institutions and organisations can provide administrative, technical and financial support to supervisors. Training, mentoring and personal development opportunities can help both supervisors and supervisees succeed.</p>
<p>Supervisors need to engage in professional development, regularly communicate with their supervisees, be culturally sensitive and use a blend of the previously outlined feedback strategies.</p>
<p>Supervisees should develop reflective skills and engage critically with feedback as integral to their learning and improvement.</p>
<h2>No ‘one size fits all’, but key principles apply</h2>
<p>Every supervisory relationship is different. However, developing a constructive feedback culture is critical. In the supervisor-supervisee relationship, lessons need to be learnt from problems in the process, and a mix of positive feedback strategies can be adopted.</p>
<p>As our study shows, there is no “one size fits all” approach to providing feedback. Ultimately, supervisors and managers should ensure feedback is supervisee-centred, focuses on improvements and is actionable.</p><img src="https://counter.theconversation.com/content/165757/count.gif" alt="The Conversation" width="1" height="1" />
<p class="fine-print"><em><span>The authors do not work for, consult, own shares in or receive funding from any company or organisation that would benefit from this article, and have disclosed no relevant affiliations beyond their academic appointment.</span></em></p>A review of the past decade of research on effective feedback confirms supervisors should aim to fill the role of a ‘critical friend’ who provides constructive and timely feedback.Ritesh Chugh, Senior Lecturer – Information Systems and Analysis, CQUniversity AustraliaBobby Harreveld, Professor and Director, Centre for Research in Equity, Advancement of Teaching & Education (CREATE), CQUniversity AustraliaStephanie Macht, Senior Lecturer in Strategic Management, CQUniversity AustraliaLicensed as Creative Commons – attribution, no derivatives.tag:theconversation.com,2011:article/867562017-11-27T19:09:50Z2017-11-27T19:09:50ZUniversities are failing their students through poor feedback practices<figure><img src="https://images.theconversation.com/files/193018/original/file-20171102-26478-1k773b5.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=496&fit=clip" /><figcaption><span class="caption">For many students, the experience of teacher-led feedback is underwhelming or negative, and they are effectively left to their own devices.</span> <span class="attribution"><span class="source">Author provided</span>, <a class="license" href="http://creativecommons.org/licenses/by-nd/4.0/">CC BY-ND</a></span></figcaption></figure><p>Educators and students often struggle to learn from each other through the use of feedback. Our research into feedback practices has found that students and staff find feedback practices largely unsustainable, de-motivating and without opportunity for improvement. </p>
<p><a href="https://www.routledge.com/Feedback-in-Higher-and-Professional-Education-Understanding-it-and-doing/Boud-Molloy/p/book/9780415692298">Researchers generally describe</a> current feedback practices as lacking in detail, difficult to understand, ambiguous or simply unusable. </p>
<p>The <a href="http://www.graduatecareers.com.au/wp-content/uploads/2016/07/Graduate-Course-Experience_20151.pdf">2015 Graduate Course Experience</a> surveyed over 93,000 students within four months of their graduation. It reported that while close to three quarters of graduates felt the feedback they received was helpful, 16.3% could not decide if the feedback was helpful, while a further 9.7% found the feedback unhelpful. Clearly something is wrong when a quarter of our graduates indicate feedback is not working.</p>
<p>The Australian Government funded Feedback for Learning <a href="http://feedbackforlearning.org">project</a> surveyed 4,514 students and 406 staff across two universities. It revealed that while students are generally satisfied with their feedback, there are a number of cohorts, or practices, that need attention. </p>
<figure class="align-center ">
<img alt="" src="https://images.theconversation.com/files/194120/original/file-20171110-29324-bs30i1.png?ixlib=rb-1.1.0&q=45&auto=format&w=754&fit=clip" srcset="https://images.theconversation.com/files/194120/original/file-20171110-29324-bs30i1.png?ixlib=rb-1.1.0&q=45&auto=format&w=600&h=721&fit=crop&dpr=1 600w, https://images.theconversation.com/files/194120/original/file-20171110-29324-bs30i1.png?ixlib=rb-1.1.0&q=30&auto=format&w=600&h=721&fit=crop&dpr=2 1200w, https://images.theconversation.com/files/194120/original/file-20171110-29324-bs30i1.png?ixlib=rb-1.1.0&q=15&auto=format&w=600&h=721&fit=crop&dpr=3 1800w, https://images.theconversation.com/files/194120/original/file-20171110-29324-bs30i1.png?ixlib=rb-1.1.0&q=45&auto=format&w=754&h=906&fit=crop&dpr=1 754w, https://images.theconversation.com/files/194120/original/file-20171110-29324-bs30i1.png?ixlib=rb-1.1.0&q=30&auto=format&w=754&h=906&fit=crop&dpr=2 1508w, https://images.theconversation.com/files/194120/original/file-20171110-29324-bs30i1.png?ixlib=rb-1.1.0&q=15&auto=format&w=754&h=906&fit=crop&dpr=3 2262w" sizes="(min-width: 1466px) 754px, (max-width: 599px) 100vw, (min-width: 600px) 600px, 237px">
<figcaption>
<span class="caption"></span>
<span class="attribution"><span class="source">Author provided/The Conversation</span>, <a class="license" href="http://creativecommons.org/licenses/by-nd/4.0/">CC BY-ND</a></span>
</figcaption>
</figure>
<p>Feedback underpins students’ effective decision making, and is the basis for improving learning outcomes. Despite heavy financial investment by universities, student experience of feedback continues to be less than desirable, especially for already “at risk” students. Many academics operate with misconceptions about feedback that inadvertently add to the problem. If we are to improve we need to seek out best practices and gain a clearer vision of what feedback should be. </p>
<h2>Feedback is not ‘given to’ or ‘done to’ the learner</h2>
<p>In higher education, the concept of feedback is commonly misunderstood. For example, many academics and students assume that feedback is a one-way flow of information, which happens after assessment submission and is isolated from any other event. In addition, academics and students often feel that the role of feedback is merely to justify the grade. A further misunderstanding is that feedback is something that is done by academics and given to students. These beliefs are deeply held in academic culture. </p>
<p>In contrast, <a href="https://www.routledge.com/Feedback-in-Higher-and-Professional-Education-Understanding-it-and-doing/Boud-Molloy/p/book/9780415692298">leading</a> <a href="http://www.tandfonline.com/doi/abs/10.1080/03075071003642449">researchers</a> <a href="https://akoaotearoa.ac.nz/download/ng/file/group-4/n3469-the-black-box-of-tertiary-assessment---john-hattiepdf.pdf">in the field</a> <a href="http://srhe.tandfonline.com/doi/abs/10.1080/02602938.2013.795518">argue</a> that feedback is not a simple input. Instead, it is a process in which information about the quality of a student’s performance is engaged with by the student, and leads to a change in future work or learning strategies. Ultimately, academics need not be involved at all. </p>
<p>The shift from a teacher-centred perspective also provides a valuable opportunity to re-position the academic as just one actor within the feedback process. Indeed, feedback comments can be from, and instigated by, a variety of sources, including the evaluator, peers, and the learner. </p>
<h2>Feedback must have impact</h2>
<figure class="align-right ">
<img alt="" src="https://images.theconversation.com/files/193015/original/file-20171102-26462-4vukrh.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=237&fit=clip" srcset="https://images.theconversation.com/files/193015/original/file-20171102-26462-4vukrh.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=600&h=705&fit=crop&dpr=1 600w, https://images.theconversation.com/files/193015/original/file-20171102-26462-4vukrh.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=600&h=705&fit=crop&dpr=2 1200w, https://images.theconversation.com/files/193015/original/file-20171102-26462-4vukrh.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=600&h=705&fit=crop&dpr=3 1800w, https://images.theconversation.com/files/193015/original/file-20171102-26462-4vukrh.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=754&h=886&fit=crop&dpr=1 754w, https://images.theconversation.com/files/193015/original/file-20171102-26462-4vukrh.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=754&h=886&fit=crop&dpr=2 1508w, https://images.theconversation.com/files/193015/original/file-20171102-26462-4vukrh.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=754&h=886&fit=crop&dpr=3 2262w" sizes="(min-width: 1466px) 754px, (max-width: 599px) 100vw, (min-width: 600px) 600px, 237px">
<figcaption>
<span class="caption">Effective feedback practices require us to look for impact.</span>
<span class="attribution"><span class="source">Author provided</span>, <a class="license" href="http://creativecommons.org/licenses/by-nd/4.0/">CC BY-ND</a></span>
</figcaption>
</figure>
<p>In contexts other than education, such as technology or biology, feedback is not an input but rather a process within a system. For example, if a blood vessel is damaged, platelets cling to the injured site and release chemicals that attract more platelets, eventually forming a blood clot. In this system, feedback regulates or optimises the output. Applying this metaphor to higher education, feedback can be usefully understood as a process within our complex teaching and learning system, rather than something that needs to be given to an actor in the system. </p>
<p>Under these circumstances, feedback can be identified by its regulating effect or impact. With this in mind, <a href="https://www.routledge.com/Feedback-in-Higher-and-Professional-Education-Understanding-it-and-doing/Boud-Molloy/p/book/9780415692298">Boud and Molloy</a> argue that assessment feedback also should be seen to require some action or change to occur. </p>
<p>In other words, any information without effect is just information. Anything else is a waste of our time. </p>
<h2>Feedback should come before submission</h2>
<p>Feedback is a process in which information about the quality of a performance leads to a change in student work or learning strategies. Arguably, for the most useful impact, feedback should occur prior to the final submission of assessment. This means we are challenged with finding ways to elicit student performance early, and to facilitate feedback that then leads to improved submissions</p>
<p>In response to this, we might consider carefully designing a series of assessments that are connected by well-planned feedback. However, it is useful to note that the initial performance does not need to be the submission of assessment - it could be in the form of a variety of tasks that expose the student’s understanding or skills that may elicit feedback from a variety of sources. </p>
<h2>Feedback as teaching</h2>
<p>For many students, the experience of teacher-led feedback is underwhelming or negative, and they are effectively left to their own devices. In other words, many students learn despite us. However, feedback is arguably the most important form of interaction we can muster as teachers. Effective feedback requires us to seek out and judge the qualities of student performance, and to craft information and responses to have an impact. In addition, we need to find ways to monitor the impact of those interactions. </p>
<p>Most educators agree that feedback is potentially valuable. However, there is considerable push-back at the thought of greater investment in feedback practices, because it adds to an academic’s already heavy workload. This response is understandable if we persist with the assumptions that feedback is something we “give” to students as a secondary practice of, and costed in relation to, assessment grading.</p>
<p>A counter-argument is that we need to re-conceive of feedback in higher education to be a form of teaching just as important as lectures and tutorials. Feedback also doesn’t need to be teacher-centred. Peer, self and automated systems of feedback are well recognised as sustainable models.</p>
<h2>Feedback design</h2>
<figure class="align-center ">
<img alt="" src="https://images.theconversation.com/files/193014/original/file-20171102-26483-1489qyn.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=754&fit=clip" srcset="https://images.theconversation.com/files/193014/original/file-20171102-26483-1489qyn.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=600&h=426&fit=crop&dpr=1 600w, https://images.theconversation.com/files/193014/original/file-20171102-26483-1489qyn.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=600&h=426&fit=crop&dpr=2 1200w, https://images.theconversation.com/files/193014/original/file-20171102-26483-1489qyn.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=600&h=426&fit=crop&dpr=3 1800w, https://images.theconversation.com/files/193014/original/file-20171102-26483-1489qyn.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=754&h=536&fit=crop&dpr=1 754w, https://images.theconversation.com/files/193014/original/file-20171102-26483-1489qyn.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=754&h=536&fit=crop&dpr=2 1508w, https://images.theconversation.com/files/193014/original/file-20171102-26483-1489qyn.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=754&h=536&fit=crop&dpr=3 2262w" sizes="(min-width: 1466px) 754px, (max-width: 599px) 100vw, (min-width: 600px) 600px, 237px">
<figcaption>
<span class="caption">Feedback practices generally can’t be replicated across all contexts.</span>
<span class="attribution"><span class="source">Author provided</span>, <a class="license" href="http://creativecommons.org/licenses/by-nd/4.0/">CC BY-ND</a></span>
</figcaption>
</figure>
<p>While there is a growing body of <a href="https://scholar.google.com/scholar?as_ylo=2013&q=assessment+feedback&hl=en&as_sdt=1,5&as_vis=1&lookup=0">literature</a> surrounding feedback, there continues to be little agreement on the best approach. Certainly, there is no single feedback strategy or model that has been shown to work across all contexts. This is a significant problem for the higher education sector. </p>
<p>The way teachers, students and institutions interact vary in every instance. This includes policy, workload pressures, academic and student culture, and other broader socio-political issues that can significantly influence what might otherwise be regarded as effective feedback strategies. It is no wonder that simple strategies of feedback cannot be replicated successfully from one context to another. </p>
<p>This leaves us with a simple but frustrating truth – every educator needs to engage in feedback practices with an inquiring mind, prepared for repeated development of their own practices.</p><img src="https://counter.theconversation.com/content/86756/count.gif" alt="The Conversation" width="1" height="1" />
<p class="fine-print"><em><span>Support for this research has been provided by the Australian Government Department of Education and Training. The views expressed here do not necessarily reflect the views of the Australian Government Department of Education and Training.
The project team includes: Professor David Boud, Associate Professor Phill Dawson, Dr Michael Phillips, Professor Elizabeth Molloy, Dr Tracii Ryan and Ms Paige Mahoney.
More information can be found: <a href="http://feedbackforlearning.org">http://feedbackforlearning.org</a> </span></em></p>Despite heavy investment by universities, student experience of feedback higher education continues to be less than desirable, especially for at-risk students.Michael Henderson, Associate Professor in Educational Technologies, Monash UniversityLicensed as Creative Commons – attribution, no derivatives.tag:theconversation.com,2011:article/712232017-01-16T15:19:18Z2017-01-16T15:19:18ZHow plugging into well-connected colleagues can help research fly<figure><img src="https://images.theconversation.com/files/152622/original/image-20170113-11191-hv7zq4.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=496&fit=clip" /><figcaption><span class="caption">Getting input from well-connected academics and researchers is crucial to a paper's scientific impact.</span> <span class="attribution"><span class="source">Shutterstock</span></span></figcaption></figure><p>Informal intellectual collaboration <a href="http://jfe.rochester.edu/jointed.pdf">is crucial</a> for good social science research. This includes interactions with colleagues to improve a paper before it is sent to a journal.</p>
<p>Our <a href="https://ssrn.com/abstract=2877586">new research</a> explored the value of informal intellectual collaboration. It highlights the importance of social networks in academia. </p>
<p>What we uncovered suggests the scientific impact of a research paper increases with every additional commenter who provides feedback. This impact is measured by the number of citations over a paper’s lifespan. The same holds when we look at the probability of publishing a paper in top journals.</p>
<p>But here’s the true novelty of our paper: it found that the feedback of more central or connected people is more valuable than less central, less connected ones when it comes to impact. And no, it’s not as simple as just asking your most senior colleague for their input. Seniority isn’t what matters. It’s all about how well connected an academic or researcher is.</p>
<p>This is important information. Our results should encourage university management to actively encourage collaboration among scientists, across departments, and across universities – and to make networking and seeking feedback part of PhD training.</p>
<h2>Connectedness is key</h2>
<p>So how do you define “well connected” in this case?</p>
<p>A researcher is well connected in a social network because they are connected to other well-connected researchers. We used what might sound like a tautological idea in our research: the so-called <em>eigenvector</em> centrality, which posits that if you know important people you are probably important in that field, too. It is the <a href="http://www.math.cornell.edu/%7Emec/Winter2009/RalucaRemus/Lecture3/lecture3.html">same idea</a> that allows Google’s search algorithm to identify relevant websites.</p>
<p>But, as we’ve already pointed out, our findings weren’t about “importance”, or status. These eigenvector central academics are not necessarily the most well known or most senior. And they aren’t always affiliated to the most prestigious universities. Yet in the social network they occupy influential positions. It’s about connections. Think of them as opinion leaders.</p>
<p>Feedback from eigenvector central academics has a much larger impact on a paper’s publication success than feedback from isolated loners. Highly connected commenters may point authors to emerging <a href="http://www.mitpressjournals.org/doi/pdf/10.1162/REST_a_00430">new topics</a> or the most rewarding avenues for future research. </p>
<p>To reach these conclusions, we built the first and most comprehensive view of the social network structure among financial economists. It connects authors and acknowledged commenters from published papers. This is a novel approach because it captures all those that have contributed to a paper, not only authors.</p>
<p>Our innovative approach was to use acknowledgements as a primary source of data. In financial economics, authors often acknowledge from which colleagues they have received helpful feedback. We collected more than 5,800 research papers from six major financial economics journals. About 90% of these acknowledge helpful input by colleagues.</p>
<p>After consolidation we create the network. Two researchers are connected when they have co-authored a paper or one acknowledges the other. This network connects about 7,500 researchers and indicates information flows between them. Then we computed the network positions and ranked individuals according to their eigenvector centrality. </p>
<figure class="align-center zoomable">
<a href="https://images.theconversation.com/files/152626/original/image-20170113-11183-1x3zl1k.png?ixlib=rb-1.1.0&q=45&auto=format&w=1000&fit=clip"><img alt="" src="https://images.theconversation.com/files/152626/original/image-20170113-11183-1x3zl1k.png?ixlib=rb-1.1.0&q=45&auto=format&w=754&fit=clip" srcset="https://images.theconversation.com/files/152626/original/image-20170113-11183-1x3zl1k.png?ixlib=rb-1.1.0&q=45&auto=format&w=600&h=600&fit=crop&dpr=1 600w, https://images.theconversation.com/files/152626/original/image-20170113-11183-1x3zl1k.png?ixlib=rb-1.1.0&q=30&auto=format&w=600&h=600&fit=crop&dpr=2 1200w, https://images.theconversation.com/files/152626/original/image-20170113-11183-1x3zl1k.png?ixlib=rb-1.1.0&q=15&auto=format&w=600&h=600&fit=crop&dpr=3 1800w, https://images.theconversation.com/files/152626/original/image-20170113-11183-1x3zl1k.png?ixlib=rb-1.1.0&q=45&auto=format&w=754&h=754&fit=crop&dpr=1 754w, https://images.theconversation.com/files/152626/original/image-20170113-11183-1x3zl1k.png?ixlib=rb-1.1.0&q=30&auto=format&w=754&h=754&fit=crop&dpr=2 1508w, https://images.theconversation.com/files/152626/original/image-20170113-11183-1x3zl1k.png?ixlib=rb-1.1.0&q=15&auto=format&w=754&h=754&fit=crop&dpr=3 2262w" sizes="(min-width: 1466px) 754px, (max-width: 599px) 100vw, (min-width: 600px) 600px, 237px"></a>
<figcaption>
<span class="caption">A visualisation of the largest connected component of the network of intellectual collaboration in financial economics using publications from 2009 to 20011. Red links connect researchers when they have co-authored a paper, blue links indicate that one acknowledged the other, and purple links indicate the both happened. The darker the node the more important the researcher is in terms of eigenvector centrality.</span>
</figcaption>
</figure>
<p>Such an analysis helps uncover patterns and structures that remain hidden when looking at individual researchers only. </p>
<p>We then used a quasi-natural experiment – the assignment of discussants at top conferences – to show our main argument: getting feedback from a colleague increases the scientific impact of a paper more if the colleague is more eigenvector central in the social network of their profession.</p>
<p>On our <a href="http://www.central-places.net/index">website</a>, we have developed an interactive tool where financial economists can find themselves on our database.</p>
<p>In a <a href="https://ssrn.com/abstract=2709107">companion paper</a> we explore the determinants of the most eigenvector central financial economists. That is, we contrasted their eigenvector centrality rank with their individual characteristics. We found that traditional author metrics such as citation counts or their number of published papers cannot explain which researchers are eigenvector central. </p>
<p>One part of the answer is certainly that citation counts have many <a href="http://www.nature.com/nature/journal/v465/n7300/full/465870a.html">problems</a> and poorly capture true academic strength. Another one is that is generally difficult to identify the opinion leaders unless you know all the network.</p>
<h2>New insights in the sociology of economics</h2>
<p>Our analysis is not exhaustive and research is ongoing. But it is clear that understanding knowledge flows helps in understanding <a href="http://voxeu.org/article/nine-facts-about-top-journals-economics">productivity differentials</a> among scientists. </p>
<p>Hopefully these results will inspire university managers to actively encourage collaboration among scientists, across department and across universities. </p>
<p>Our results also support calls to measure scientific impact <a href="http://blogs.lse.ac.uk/impactofsocialsciences/2015/04/20/scholarly-behaviour-evaluation-criteria-citations/">broader</a>, and not just based on citations. </p>
<p>Finally, our findings highlight the importance of sufficient travel funding for academics, given the crucial role of <a href="http://blogs.lse.ac.uk/impactofsocialsciences/2016/08/16/the-last-great-unknown-the-impact-of-academic-conferences/">academic conferences</a> as a networking opportunity.</p>
<hr>
<p><em>Authors’ note: This article is based on a <a href="http://blogs.lse.ac.uk/impactofsocialsciences/2017/01/10/feedback-helps-increase-the-impact-of-academic-research-even-more-so-when-coming-from-well-connected-colleagues/">post</a> written for the London School of Economics’ blog.</em></p><img src="https://counter.theconversation.com/content/71223/count.gif" alt="The Conversation" width="1" height="1" />
<p class="fine-print"><em><span>The authors do not work for, consult, own shares in or receive funding from any company or organisation that would benefit from this article, and have disclosed no relevant affiliations beyond their academic appointment.</span></em></p>The scientific impact of a research paper increases with every additional commenter who provides feedback – particularly if the comment came from a well-connected academic.Michael E. Rose, PhD Candidate in Economics, University of Cape TownCo-Pierre Georg, Senior Lecturer, African Institute for Financial Markets and Risk Management, University of Cape TownLicensed as Creative Commons – attribution, no derivatives.tag:theconversation.com,2011:article/549902016-02-23T09:37:20Z2016-02-23T09:37:20Z‘Quality assurance’ must be reimagined at Ethiopia’s universities<figure><img src="https://images.theconversation.com/files/112337/original/image-20160222-25855-1asj13r.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=496&fit=clip" /><figcaption><span class="caption">Ethiopian universities may suffer if quality assurance systems are too narrowly focused.</span> <span class="attribution"><span class="source">Adam Jones/Flickr</span></span></figcaption></figure><p>Universities and higher education systems around the world have become fixated on quality over the past two decades. They have tried out any number of initiatives designed to improve quality. They have looked for structures that will produce concrete evidence of <a href="http://www.tandfonline.com/doi/full/10.1080/14719030701340390">effectiveness and efficiency</a>.</p>
<p>Quality assurance has emerged as one of universities’ most significant management tools in this quest. It has become <a href="http://www.tandfonline.com/doi/abs/10.1080/13538322.2010.485728">a feature</a> of every continent’s higher education system. Its major drive is to ensure that universities provide quality higher education based on a minimum set of criteria and standards. At the same time, it is expected to <a href="http://www.tandfonline.com/doi/abs/10.1080/1353832042000230635?journalCode=cqhe20">improve quality</a> within an institution. </p>
<p>But does it work? I undertook <a href="http://www.codesria.org/spip.php?article2531&lang=en">a study</a> in Ethiopian universities as part of my PhD research project to find out the realities, opportunities and impediments of quality assurance. </p>
<p>Ethiopia’s higher education landscape has expanded continuously from about 2004. Then, there were eight public universities in the country. Today there are 36. Private universities are also on the rise, and account for about 15% of the country’s total tertiary student population.</p>
<p>Enrolment rates have climbed too: in 2010/11, Ethiopia recorded 447,693 undergraduates across institutions. By the 2013/14 academic years, this had <a href="http://unesdoc.unesco.org/images/0023/002317/231724e.pdf">risen</a> to 593,571. Most of these students were pursuing technology or engineering related degrees. </p>
<p>Preliminary evidence suggests that their quality of teaching and learning has actually dropped despite the introduction of quality assurance systems. Why is this the case? </p>
<h2>The Ethiopian university landscape</h2>
<p>First, a word about what “quality assurance” entails at universities. The average quality assurance system requires evidence of “quality” at an institutional level. This is measured, for example, through research output, student progression from enrolment to completion, and graduation rates. Lecturers are expected to keep track of such details through overly prescriptive teaching and learning assessment policies.</p>
<p>My research found that the concept of quality assurance has been accepted by Ethiopia’s government and education authorities. It is also accepted as a daily reality at individual institutions. The problem is that there’s no evidence to show any widespread qualitative change in classroom practices or students’ learning experiences.</p>
<p>For the most part, Ethiopian universities <a href="http://www.moe.gov.et/English/Information/Documents/Education%20Statistics%20Annual%20Abstract%202006E.C%20%282013-2014%20G.C%29.pdf">struggle</a> to get basic and essential learning resources like text and reference books and laboratory and workshop equipment and facilities. Students’ academic work most often depends on notes handed out by lecturers. This means that three things become <a href="https://books.google.co.za/books?hl=en&lr=&id=cehvAAAAQBAJ&oi=fnd&pg=PT9&dq=creating+significant+learning+experiences+fink&ots=GBlFqPavzL&sig=5K2DMUnmwxQvbLe_Rm4bFX_e1m8#v=onepage&q&f=false">crucial</a>:</p>
<ul>
<li><p>developing the skills and resources required to learn</p></li>
<li><p>learning independently, and</p></li>
<li><p>cooperation among students</p></li>
</ul>
<p>The problem is that the quality assurance systems being used in Ethiopia have not been developed with the country’s specific educational context - as outlined above - in mind.</p>
<h2>Advantages and problems</h2>
<p>There are some advantages to the system of quality assurance. For example, it provides an agreed upon definition of quality that works almost everywhere. It also establishes a minimal threshold as a standard for quality - a base from which all institutions can work. There are, finally, valuable structural approaches, procedures and processes inherent within any quality assurance scheme. </p>
<p>However, there are several impediments in such a system. These include heightened emphasis on reporting of results, methodological flaws and a lack of concern for context. </p>
<p>Research <a href="https://www.researchgate.net/profile/Michael_Skolnik/publication/43797524_Quality_Assurance_in_Higher_Education_as_a_Political_Process/links/09e4150579e6a3f091000000.pdf">has shown</a> that the efficacy of quality assurance lies in its political and ideological nature. This means that any system of quality assurance relies heavily on adherence to externally imposed definitions and structures. </p>
<p>It’s also all about performing common actions - so a lecturer at an under-resourced but overburdened Ethiopian university would be expected to behave in the same way as a colleague at a wealthy Australian institution to live up to ideals of “quality”.</p>
<h2>Quality assurance doesn’t mean improved quality</h2>
<p>It is obvious that there is room for improvement in Ethiopian universities’ teaching and learning practices. </p>
<p>The problem is that quality assurance systems don’t bring about such improvement - a fact that’s been <a href="https://books.google.co.za/books?id=A-pDBgAAQBAJ&printsec=frontcover#v=onepage&q&f=false">proved</a> by extensive <a href="http://www.tandfonline.com/doi/abs/10.1080/13538321003679457">global research</a>.</p>
<p>These systems tend to have a limited scope of information available to serve as inputs for quality improvement. This is because their emphasis is on “the big picture”, like student graduation numbers, rather than on daily academic or learning experiences. Such systems don’t consider evidence about regular academic practices or students’ learning experiences as being important information that might contribute towards improvement.</p>
<p>Simply put, quality assurance focuses on reporting to meet accountability requirement and undermines the influence of context. It also undermines the complexity of educational outcomes and institutional practices, as well as the conditions that lead to attaining such outcomes.</p>
<h2>Making changes to make systems work</h2>
<p>Quality assurance programmes definitely have a role to play at Ethiopia’s universities. But this role will only be truly positive if programmes are modified to take academic considerations into account. They must also become more flexible about collecting essential data at an individual level rather than just focusing on the institutional level. </p>
<p>Changing the programmes’ focus in this way will mean that higher education quality assurance becomes more concerned with the micro realities of higher education and the academy. Also, they will become more relevant to driving any improvement of quality.</p><img src="https://counter.theconversation.com/content/54990/count.gif" alt="The Conversation" width="1" height="1" />
<p class="fine-print"><em><span>Tefera Tadesse does not work for, consult, own shares in or receive funding from any company or organisation that would benefit from this article, and has disclosed no relevant affiliations beyond their academic appointment.</span></em></p>Quality assurance programmes tend to ignore context - which means important elements of teaching and learning are overlooked and universities miss out on a real chance to improve their practices.Tefera Tadesse, Assistant Professor of Curriculum and Instruction, Jimma UniversityLicensed as Creative Commons – attribution, no derivatives.tag:theconversation.com,2011:article/373702015-02-10T01:23:48Z2015-02-10T01:23:48ZRate my professor’s gender?<figure><img src="https://images.theconversation.com/files/71542/original/image-20150210-24687-1vilyug.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=496&fit=clip" /><figcaption><span class="caption">When rating their classes, students use different words to describe male and female professors</span> <span class="attribution"><a class="source" href="https://www.flickr.com/photos/nationalassemblyforwales/6937339042">National Assembly for Wales</a>, <a class="license" href="http://creativecommons.org/licenses/by-sa/4.0/">CC BY-SA</a></span></figcaption></figure><p>A friend of mine calls me professor of genius studies. It’s a sort of slip of the tongue, as I teach in gender studies, but it‘s also funny because everyone knows that genius is not associated with gender studies, and I’m the wrong gender anyway. A genius has electrified hair, big glasses, problems talking with mere mortals, and is white and male. Disney confirms this repeatedly, as does Christine Battersby in her 1989 study <a href="http://philpapers.org/rec/BATGAG">Gender and Genius</a>.</p>
<p>Now the anonymous online ranking system, <a href="http://www.ratemyprofessors.com/">RateMyProfessors.com</a>, has been subjected to algorithmic sifting to find that genius is a term students apply to male professors at least three times the rate for women, depending on the discipline. Brilliance is also something men do better in university lecture theatres, according to these ratings, and in music male professors are seven times more likely than female professors to be virtuoso performers. That was by over 3 million students. Consistently more knowledgeable and smart, men are also handsome, cute, charming, funny and sensitive.</p>
<p>So we know what’s coming next. As this is a gender mapping, women professors are consistently more likely to be described as feisty, bossy, aggressive, shrill, condescending, rude. You get the picture. We are also ahead on that vanilla descriptor, nice.</p>
<figure class="align-center zoomable">
<a href="https://images.theconversation.com/files/71446/original/image-20150209-13716-icpuqy.png?ixlib=rb-1.1.0&q=45&auto=format&w=1000&fit=clip"><img alt="" src="https://images.theconversation.com/files/71446/original/image-20150209-13716-icpuqy.png?ixlib=rb-1.1.0&q=45&auto=format&w=754&fit=clip" srcset="https://images.theconversation.com/files/71446/original/image-20150209-13716-icpuqy.png?ixlib=rb-1.1.0&q=45&auto=format&w=600&h=642&fit=crop&dpr=1 600w, https://images.theconversation.com/files/71446/original/image-20150209-13716-icpuqy.png?ixlib=rb-1.1.0&q=30&auto=format&w=600&h=642&fit=crop&dpr=2 1200w, https://images.theconversation.com/files/71446/original/image-20150209-13716-icpuqy.png?ixlib=rb-1.1.0&q=15&auto=format&w=600&h=642&fit=crop&dpr=3 1800w, https://images.theconversation.com/files/71446/original/image-20150209-13716-icpuqy.png?ixlib=rb-1.1.0&q=45&auto=format&w=754&h=807&fit=crop&dpr=1 754w, https://images.theconversation.com/files/71446/original/image-20150209-13716-icpuqy.png?ixlib=rb-1.1.0&q=30&auto=format&w=754&h=807&fit=crop&dpr=2 1508w, https://images.theconversation.com/files/71446/original/image-20150209-13716-icpuqy.png?ixlib=rb-1.1.0&q=15&auto=format&w=754&h=807&fit=crop&dpr=3 2262w" sizes="(min-width: 1466px) 754px, (max-width: 599px) 100vw, (min-width: 600px) 600px, 237px"></a>
<figcaption>
<span class="caption">Female professors: much nicer than male professors, according to their students.</span>
<span class="attribution"><a class="source" href="http://benschmidt.org/profGender/#%7B%22database%22%3A%22RMP%22%2C%22plotType%22%3A%22pointchart%22%2C%22method%22%3A%22return_json%22%2C%22search_limits%22%3A%7B%22word%22%3A%5B%22nice%22%5D%2C%22department__id%22%3A%7B%22%24lte%22%3A25%7D%7D%2C%22aesthetic%22%3A%7B%22x%22%3A%22WordsPerMillion%22%2C%22y%22%3A%22department%22%2C%22color%22%3A%22gender%22%7D%2C%22counttype%22%3A%5B%22WordsPerMillion%22%5D%2C%22groups%22%3A%5B%22department%22%2C%22gender%22%5D%7D">Screenshot: Gender and Teacher Reviews</a></span>
</figcaption>
</figure>
<p>These fascinating results are enabled through the work of Assistant Professor <a href="http://benschmidt.org/profGender/">Benjamin Schmidt</a> from Northeastern University, who released an interactive chart that groups results from about 14 million reviews over a couple of months from RateMyProfessors. It’s easy to use: type in the word and the results will be graphed, split by gender across discipline and per million words.</p>
<p>Schmidt notes that: </p>
<blockquote>
<p>not all words have gender splits, but a surprising number do. Even things like pronouns are used quite differently by gender. </p>
</blockquote>
<p>Indeed, even the definite article, the, is applied more to men than women. Note that these results are only distributed by the gender of the rated professor, not by the gender of the reviewing student. And they can also be sorted by whether the review was positive or negative overall.</p>
<figure class="align-right zoomable">
<a href="https://images.theconversation.com/files/71543/original/image-20150210-24700-178jvdn.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=1000&fit=clip"><img alt="" src="https://images.theconversation.com/files/71543/original/image-20150210-24700-178jvdn.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=237&fit=clip" srcset="https://images.theconversation.com/files/71543/original/image-20150210-24700-178jvdn.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=600&h=456&fit=crop&dpr=1 600w, https://images.theconversation.com/files/71543/original/image-20150210-24700-178jvdn.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=600&h=456&fit=crop&dpr=2 1200w, https://images.theconversation.com/files/71543/original/image-20150210-24700-178jvdn.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=600&h=456&fit=crop&dpr=3 1800w, https://images.theconversation.com/files/71543/original/image-20150210-24700-178jvdn.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=754&h=573&fit=crop&dpr=1 754w, https://images.theconversation.com/files/71543/original/image-20150210-24700-178jvdn.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=754&h=573&fit=crop&dpr=2 1508w, https://images.theconversation.com/files/71543/original/image-20150210-24700-178jvdn.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=754&h=573&fit=crop&dpr=3 2262w" sizes="(min-width: 1466px) 754px, (max-width: 599px) 100vw, (min-width: 600px) 600px, 237px"></a>
<figcaption>
<span class="caption">Walt Disney’s stereotypical Professor Ludwig Von Drake. You get the picture - despite him being a duck.</span>
<span class="attribution"><a class="source" href="http://www.google.com.au/imgres?imgurl=http://img3.wikia.nocookie.net/__cb20100629171814/disney/images/3/3a/NewPicture6-1.jpg&imgrefurl=http://disney.wikia.com/wiki/Ludwig_Von_Drake&h=574&w=755&tbnid=nSYbgySlNhWVFM:&zoom=1&docid=IV9tLhrORXgSxM&ei=qE7ZVO_ZEcOymAXtxoG4Cg&tbm=isch&ved=0CB8QMygCMAI">Disney.wikia.com</a></span>
</figcaption>
</figure>
<p>This graphical mapping of language used by American students to rate their professors tells us that gender is repeatedly constructed through the language we use to differentiate behaviours and values, and that women still face systematic obstacles in academia.</p>
<p>This usable dataset has been noticed by <a href="http://thinkprogress.org/education/2015/02/07/3620571/rate-my-professor-sexist/">online media</a> commentators and compared to <a href="http://www.nytimes.com/2015/02/07/upshot/is-the-professor-bossy-or-brilliant-much-depends-on-gender.html?abt=0002&abg=0">other studies</a> that demonstrate that teaching evaluations, citation, promotion and research funding are all highly gendered practices. </p>
<p>This is part of a larger narrative about women and work, about the structural hostility when women enter workplaces that are traditionally occupied by men. Such workplaces have already normalised the authority and historic contributions of men – often they are literally built for men, as in the cases where women’s toilets <a href="http://alga.asn.au/site/misc/alga/downloads/womeninpol/ALGA_WomenInPolitics.pdf">have to be added</a> to buildings (<a href="http://www.australia.gov.au/about-australia/australian-story/austn-women-in-politics">including parliaments</a>).</p>
<p>The mass introduction of women into higher education as academics and as students is only relatively recent. In my university, for example, the ban on employing married women was rescinded only in 1976. These structures take generations of making visible, naming and countering.</p>
<p>It’s still sobering to see evidence of the ways gender – or should we be calling it misogyny - is so deeply embedded in language. Australian feminist <a href="http://dalespender.com.au">Dale Spender</a> has been talking about this since the 1970s: how language is male-centred (man comes to stand in for humanity); words reserved for women are derivations/deviations from the word for men (actress, woman); the sexual double standards (stud and slut); and the lack of words to name sexism, rape, sexual harassment, child abuse – all words which have entered our vocabulary since that time. </p>
<p>Spender attributes this lineage of English partially to the history of dictionary, but the impact is that it limits the ways in which we can construct our social world and speak to each other as gendered social beings.</p>
<p>Because this is just raw data, though, we can find other things about gender in it as well – you can search for whatever word you want, and its antonym. Women are more likely to be caring, helpful and encouraging, as we might expect in a society that continually associates women as carers, but we’re also more likely to be uncaring, unhelpful and discouraging – again consistent with the higher expectation of women as carers.</p>
<p>With a different set of terms, though, women are much more likely to be described as feminist, creative, fabulous, amazing, wonderful. Men are consistently ahead on crude, old, vulgar, outdated, misogynist. </p>
<p>I wonder what would happen if we made this the lead story: </p>
<blockquote>
<p>Women professors are fabulous, amazing, wonderful; men crude, vulgar, old. </p>
</blockquote>
<p>It doesn’t fit with the stories we tell about women, or universities, but it’s there in the data as well. And it doesn’t contradict the systemic oppression story; in a whole different way it supports it.</p><img src="https://counter.theconversation.com/content/37370/count.gif" alt="The Conversation" width="1" height="1" />
<p class="fine-print"><em><span>Alison Bartlett does not work for, consult, own shares in or receive funding from any company or organisation that would benefit from this article, and has disclosed no relevant affiliations beyond their academic appointment.</span></em></p>A friend of mine calls me professor of genius studies. It’s a sort of slip of the tongue, as I teach in gender studies, but it‘s also funny because everyone knows that genius is not associated with gender…Alison Bartlett, Associate Professor in Gender Studies, The University of Western AustraliaLicensed as Creative Commons – attribution, no derivatives.