Two students submit the exact same essay for an exam. One was given the family name Abbeloos as a gift from his parents, the other goes by the name Winckelmans. Who gets the best rating?
In an ideal world, their last name wouldn’t matter one bit. But in the real world, student Abbeloos might receive higher points and better comments than student Winckelmans.
Three researchers from the University of Michigan examined more than thirty million exam scores of American students that were entered into an online learning platform. They also had access to the order in which the tasks and tests submitted on the platform were scored.
Their analysis shows that students who were assessed later received lower scores overall. The ten students who received their score in fiftieth or later achieved an average of 3.5 percent less than the fellow students who were assessed in the first ten. The difference in points between the first ten and later students increased systematically.
Bad luck for whose exam form ends up at the bottom of the pile. Since the corona crisis, online learning platforms have also been used more often for evaluation and that has not improved the situation. On the contrary. Because that means the same students are invariably the unlucky ones. Platforms such as Canvas or Blackboard sort students by surname by default. Most teachers do not change that alphabetical order, the researchers found. The result: anyone with a surname starting with A to E scores on average 0.6 out of 100 more than fellow students with a name from U to Z. The effect is exactly the opposite for the small group of teachers who take the exams in reverse alphabetical order. corrects.
Shuffle copies
The researchers also had access to the comments given by teachers. The order also plays a role there. A text analysis shows that students who were assessed later received more negative and unfriendly comments.
“We suspect that fatigue is an important explanatory factor,” said Jaixin Pei, one of the researchers, in a press release from his university. “If you work on something for longer, you lose your attention and your cognitive skills decrease.”
The order effect is greater in the human sciences than in the exact and medical sciences. “In the human sciences, the assessment of a task is often more subjective because there is no one correct answer,” the researchers said.
The results do not surprise educational economist Kristof De Witte (KU Leuven). He trains dozens of future teachers and economics teachers every year. “We point out this danger to future teachers. That is why we recommend improving question by question, and changing the order of the exam copies after each question so that the same students do not have an advantage each time. We also recommend correcting the first ten copies in pencil, and only proceeding with the real correction after the entire stack has been completed.”
