‘Failed research also deserves a prize’

The first Trial and Error Award for ‘failed’ research was recently awarded to the Dutch psychologist Floor van Meer. This happened during the General Members Meeting of the Utrecht Young Academy (UYA). The prize is a tribute to the essential place of failure in science and is intended to encourage the publication of research with methodological areas for improvement and negative or null results. The prize is a joint initiative of those involved in the Journal of Trial and Error (JOTE) and several members of the Utrecht Young Academy (UYA).

An award for failed research

It is important that trial and error – or trial and error – is recognized as an essential part of the scientific process. After all, publishing failed research and studies with negative and null results has many advantages: it leads to better meta-analyses, it can prevent scientists from making the same mistakes again, and it provides a more realistic picture of the scientific research field.

Above all, trial and error is a natural part of science that should be celebrated. After all, knowledge that shows that something is not the case is also knowledge. With this message, the initiators hope that less emphasis will be placed within science on publications with positive results. The ultimate goal is a healthier research field and a healthier publication culture.

Driving leads to more snacking, but not due to stress

Floor van Meer won the first Trial and Error Award for her research into the effect of distracted eating on eating behavior. Together with her colleagues, the psychologist discovered in her research that people who ate chips during a driving simulation subsequently eat more chips, as predicted. The team also encountered some unexpected null results. It turned out that this effect could not be explained by a difference in the perceived taste intensity or stress, something that the researchers expected.

Challenges like these highlight how difficult it is to research human behavior. For example, the researchers should have taken into account how hungry the participants felt and how much experience they had with driving simulations and games. Studies like these, where results do not match the hypotheses, can therefore still provide many valuable methodological insights.

You can find the conditions here.

By the way, the main hypotheses of the study initially seemed to be confirmed and the article was submitted to a scientific journal other than JOTE. To be open and transparent about the results, the authors also shared all data and analysis scripts with the reviewers. However, one of the reviewers found an error in the analysis script. One of the most important effects observed in the study therefore changed from ‘significant’ to ‘just not significant’, and the article was rejected.

For her publication, Van Meer received a work of art created for this occasion by Isolde Eickmans. Together with fellow students from Sint Lucas College in Boxtel, Eickmans designed a series of works that depict the concept of failure.

The Journal of Trial and Error

As initiators of the Trial and Error Award, we saw that failure in science was not yet sufficiently praised. That is why we created the Trial and Error Award, named after the open access journal that initiated the award: the Journal of Trial and Error.

Since its founding in 2019, JOTE has been publishing articles with negative and null results, as a response to the positive publication bias in science. Because scientists are mainly rewarded for impressive, positive results, studies without these results remain unpublished on scientists’ desks. However, these studies may contain sensible methodological improvements or indications against certain leading theories. JOTE is convinced of the productive role of errors in science. Making mistakes and publishing them is a fundamental part of learning and discovering in science.

In addition to failed research, JOTE also publishes articles reflecting on (methodological) errors and challenges, and scientific studies into science itself – a field also known as ‘metascience’.

Structural recognition of unsuccessful research

As organizers, we want to offer the Award annually. We are currently discussing doing this at the international Open Science Conference next year. The idea is to expand the nomination process so that anyone who has been brave enough to share unexpected results or failed research can be nominated.

As initiators, we hope that the prize will contribute to a cultural change within the academic community, adopting a more accepting attitude towards failure in science. Ultimately, this should ensure that all types of research results – and not just successful ones – are valued and encouraged.

Nominees

The nominees for the Trial and Error Award were:

• Florencia Abinzano – The Complexity of Joint Regeneration: How an Advanced Implant Could Fail by Its In Vivo Proven Bone Component
• Wendy Ross – Rewilding Cognition: Complex Dynamics in Open Experimental Systems
• Caroline Fischer – Real-effort survey designs: Open-ended questions to overcome the challenge of measuring behavior in surveys
• Tamal Kumar De – Classical Conditioning for Pain: The Development of a Customized Single-Case Experimental Design
• Joshua W. Maxwell – Do Carryover Effects Influence Attentional Bias to Threat in the Dot-Probe Task?
• James Bartlett – No Meaningful Difference in Attentional Bias Between Daily and Non-Daily Smokers
• Trisha Harjani – Gamified Inoculation Against Misinformation in India: A Randomized Control Trial
• Floor van Meer – Driven to Snack: Simulated Driving Increases Subsequent Consumption