A confession from my review hall of shame

This short reflection follows on from the last episode, a replay of my 2017 conversation with Gloria Mark in honour of her just having published her book called “Attention span: a groundbreaking way to restore balance, happiness and productivity”. I make a confession here that comes from my reviewing hall of shame, about when I was a reviewer of one of the key papers leading to this book, a paper authored by Victor González and Gloria Mark. And how I (very wrongly!) argued for rejection. Luckily good colleagues saved me from myself and the paper was accepted but I use this as an example to urge us all to be more reflective about the biases we bring to reviewing and position this also against the broader challenges around reviewing in our increasingly hypercompetitive publication culture. I share this story with Victor González and Gloria Mark’s permission.

Full transcript pdf for download

Overview (times approximate):

0:05 Introduction to changing academic life.

1:31Introduction of the story – paper related to Gloria’s new bo

3:07 Rigorous fieldwork and data collection by Victor Gonzalez.

5:07 Arguing for rejection, discussing the paper in the corridor.

7:15 Judging a paper on its merits.

9:07 The coincidence of other conversations about reviewing eg Life in Academia Seminar

11:11 Review bias eg quantitative vs qualitative research, and Big Q vs little Q qualitative research.

13:27 The broader critiques of the review process.

15:57 Unsustainability of review effort - CHI 2023 example.

17:19 The need to radically rethink peer-review and publication practices.

19:45  End

Related Links:

Victor González, Sperientia

Gloria Mark, UC Irvine and the replay of the interview with Gloria

[Their paper] Victor M. González and Gloria Mark. 2004. "Constant, constant, multi-tasking craziness": managing multiple working spheres. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '04). Association for Computing Machinery, New York, NY, USA, 113–120. https://doi.org/10.1145/985692.985707

[Academic paper] Aczel, B., Szaszi, B. & Holcombe, A.O. A billion-dollar donation: estimating the cost of researchers’ time spent on peer review. Res Integr Peer Rev 6, 14 (2021).

[Academic paper] Moore, S., Neylon, C., Paul Eve, M. et al. “Excellence R Us”: university research and the fetishisation of excellence. Palgrave Commun 3, 16105 (2017).

[Academic paper] Park, M, et al, Papers and patents are becoming less disruptive over time. Nature 613, 138–144 (2023)

[Twitter thread - pointers to academic papers/books] Virginia Braun and Victoria Clarke - twitter thread on their Big Q little q distinctions in qualitative research with links to relevant papers

[Webinar] Life in Academia webinar by Edward Lee 24.1.2023: Toxic culture of rejection

[Blog article] Edward Lee The toxic culture of rejection in computer science. 22 Aug 2022

[Blog article] Nesta, Reducing bias in funding decisions (“Nesta The UK's innovation agency for social good”)

COARA: Coalition for Advancing Research Assessment

DORA: The Declaration on Research Assessment

Acknowledgements:

Thanks to Sabrina Burtscher for cleaning up the otter.ai transcript.