'Punishment by statistics': the father who foresaw A-level algorithm flaws | A-levels | The Guardian US edition US edition UK edition Australian edition International edition The Guardian - Back to home Search jobs Sign inSearch News Opinion Sport Culture Lifestyle ShowMoreShow More News US news World news Environment Soccer US politics Business Tech Science Newsletters Opinion The Guardian view Columnists Letters Opinion videos Cartoons Sport Soccer NFL Tennis MLB MLS NBA NHL Culture Film Books Music Art & design TV & radio Stage Classical Games Lifestyle Fashion Food Recipes Love & sex Home & garden Health & fitness Family Travel Money Make a contribution Subscribe Search jobs Digital Archive Guardian Puzzles app The Guardian app Video Podcasts Pictures Inside the Guardian Guardian Weekly Crosswords Search jobs Digital Archive Guardian Puzzles app Education Schools Teachers Universities Students A-levels This article is more than 5 months old 'Punishment by statistics': the father who foresaw A-level algorithm flaws This article is more than 5 months old Huy Duong had long warned 39% of grades between A* and D would be lower than teacher assessments Huy Duong (right), with his son Thanh. The IT consultant calculated that 39% of grades would be downgraded. Ofqual has since confirmed he was right. Photograph: Graeme Robertson/The Guardian Huy Duong (right), with his son Thanh. The IT consultant calculated that 39% of grades would be downgraded. Ofqual has since confirmed he was right. Photograph: Graeme Robertson/The Guardian Helen Pidd Fri 14 Aug 2020 05.16 EDTLast modified on Fri 14 Aug 2020 23.36 EDT When the English exams regulator set out how grades would be awarded following the cancellation of GCSE and A-levels this summer, there was suspicion of its “standardisation model” – a mysterious algorithm designed to avoid grade inflation. Ofqual probably reckoned on a bit of pushback from the Labour party and maybe the teaching unions too. But in the end it was a statistics-savvy dad worried about his own son’s chances of getting into Oxbridge who correctly predicted that standardisation would result in 39% of students being downgraded, with disadvantaged pupils worst affected. Huy Duong came to the UK as a teenager with his parents in the 1980s, a refugee from communist Vietnam. Now a father of three and an IT consultant, he told the Guardian that his formative years made him inherently opposed to anything which felt opaque and undemocratic. And that included “collective punishment by statistics”. His eldest child, Thanh, attends Matthew Arnold school, a large, non-selective comprehensive in Oxford rated “good” by Ofsted. Smart and with a mathematical mind, this summer Thanh should have been taking A-levels in maths, further maths and chemistry. He had a conditional offer to study engineering at Cambridge, which required two A*s and an A – a tough ask, but one father and son agreed was eminently achievable. Then came lockdown. Exams were off and Thanh’s teachers were asked to award grades on the basis of mocks and assessments, and told to rank students from best to worst in each subject. This data was then fed into Ofqual’s computer system, designed to eliminate teacher bias. The algorithm checked the results weren’t wildly different from the past and the expected national grade distribution for the subject, downgrading outliers. Ofqual tried to reassure students by saying the national results would be “broadly in line with previous years” and might be slightly higher. But Duong wondered what it would mean for Thanh and his classmates. Using what scant information Ofqual had made public on its methodology, and an understanding of statistics gained from his PhD in physics, he analysed the 2017-2019 A-level data for Matthew Arnold school. With a little help from his sister, a statistician at the Medical Research Council, he came to the conclusion that that there was “virtually no chance of providing grades to the students in a way that satisfies the double criteria of being fair to the individuals and controlling grade inflation nationally”. He warned the education select committee that 39% of grades between A* and D would be lower than the teacher assessments, and last week he shared his findings with the Guardian. He repeatedly sent emails to Ofqual only to receive a stock reply. Publicly, Ofqual insisted that Duong was wrong. In what may now appear to be a carefully worded statement, a spokesperson said: “We expect the majority of grades students receive will be the same as their centre assessment grades.” But on Thursday morning Ofqual admitted that nearly two in five (39.1%) pupils in the country saw their A-level grades downgraded from their teachers’ estimates. Duong had been bang on. At home in Oxford, he was in no mood to celebrate being right, however. Shortly after 8am, Thanh received his results: an A in maths, A* in further maths and As in chemistry and physics. The maths results being the wrong way round meant he missed his Cambridge offer. A “ridiculous” set of grades, said his father: “[Thanh] has consistently scored A* in tests and both mocks in maths. He finds mathematics much easier than further maths. He has also done quite well in the UK Maths Challenge [a national contest] and mathematical olympiad.” “Ofqual wants to control grade inflation, which is fair enough,” said Duong. “But how confident are they that they are not downgrading the wrong people?” Ofqual had a “public duty” to be open about how confident it was about its data, said Duong – “otherwise it would be like the Department of Health and Social Care imposing a vaccination programme on all school children without any data on toxicity and efficacy”. He had another analogy to explain his objections to standardisation: “Suppose you have 100 cars travelling on a motorway and 41 of them broke the speed limit. In the interest of road safety, as a matter of principle, it is correct to fine the 41 speeding drivers. “However, suppose that in practice, for whatever reason, the speed trap catches the wrong car, say, 25% of the time. As a result, about 10 of the drivers caught by the speed trap weren’t speeding at all but have been wrongly accused. What sort of democratic society would accept that? It sounds like collective punishment by statistics.” Ofqual continues to insist its standardisation arrangements “are the fairest possible to facilitate students progressing on to further study or employment as planned”. On Thursday it published a 319-page document explaining its methodology. It was little solace to Thanh, even after Cambridge decided to accept him regardless. Opening the email with his results was a “stomach-dropping experience”, he said. “These results will affect us for at least the next 10 years, getting my first job, although I recognise my situation isn’t the worst … The results can’t be as reliable [as previous years] because it’s something … manufactured rather than what we have set out to do and then achieved.” Duong’s crusade for transparency continues. “I came here as a teenager with my parents in the 1980s. In that time, Vietnam was a communist country. We came here as political refugees to a democratic country, so I have a very strong idea of how democratic societies should work and this doesn’t look like it,” he said. Topics A-levels Exams Ofqual Sixth form Schools news Reuse this content Education Schools Teachers Universities Students News Opinion Sport Culture Lifestyle About us Contact us Complaints & corrections SecureDrop Work for us Privacy policy Cookie policy Terms & conditions Help All topics All writers Digital newspaper archive Facebook YouTube Instagram LinkedIn Twitter Newsletters Advertise with us Guardian Labs Search jobs Back to top © 2021 Guardian News & Media Limited or its affiliated companies. All rights reserved. (modern)