Jeremy Birnholtz on sustainability of reviewing, queer research and being curious
Jeremy Birnholtz is an associate professor with a joint appointment in the Communication Studies and the Electrical Engineering and Computer Science Departments at Northwestern University in Chicago in the US. He also directs the Social Media Lab. The trigger for this conversation was the recent discussion with John Tang about reviewing and Jeremy continues this discussion, looking at issues around authoring and service asymmetries, the unsustainability of the current review and publication models based on what he calls the perpetual motion machine that pushes researchers to churn out more and more papers. He calls for a greater focus on quality of papers instead of numbers and to identify quality signifiers beyond just publications. We discuss his role as conference chair of the upcoming CSCW conference, which is traditionally about distributed online collaboration, and moving the conference online. Shifting topics, we also talk about his personal coming out and the pivot of his research to explore topics around gender and sexuality. Through all of these discussions, Jeremy’s curiosity and care comes through again and again. Much to ponder on here.
“If you are playing the long game, eventually it [career] does work out.”
“I’ve come to believe that you just need a smaller number of very very good papers to make your contribution and a name for yourself.”
“On search committees, writing tenure letters, it is in our collective interest to not be obsessed with numbers and to focus on the contribution and on the quality of the work.”
“As a junior person it is easy to get caught up in the perpetual motion machine mentally where you are constantly spinning out new papers.”
“We’re smart observant people. If we look around and have these conversations we can come up with a way to fix this. This is a solvable problem but it takes stepping back, noticing and talking about it.”
“There’s something I really enjoy about throwing myself in a situation where all of my assumptions are very likely to be wrong and trying to figure out where to go from there.”
Overview (times approximate):
02:00 Jeremy’s background and career path to date
07:50 Reflecting on reviewing and service challenges
36:35 Shifting to queer research topics
53:00 Values & superpowers
59:09 End
In a little more detail… or download the full transcript here
Background:
02:00 Jeremy gives an overview of his background and career to date.
04:30 Jeremy discusses why he moved from Cornell to Northwestern, right before he was up for tenure, and early career choices and challenges.
“If you are playing the long game, eventually it [career] does work out.”
Reviewing & service:
07:50 We shift to the ongoing discussion around reviewing (building on the conversation with John Tang).
08:20 One issue is the arms race in CV length, the pressure to publish lots of papers, and the volunteer service required, which can put the emphasis on the writing that we must do and not the service that we can do.
“In order to get the papers published, people need to review. But if that is getting de-prioritised it is getting harder to find people to review.”
09:50 The cost structure of reviewing – once written the cost of submitting is very little and the cost of reviewing it is pushed onto the community. A sustainability issue.
11:10 We discuss the impact on quality and how large numbers of papers do not equate to quality, and what he finds more compelling when on a search committee.
“I’ve come to believe that you just need a smaller number of very very good papers to make your contribution and a name for yourself.”
13:20 We talk about where publication numbers do have impact, in the filtering process of initial applications. And he talks about looking for other quality signals. But you have to look for it.
15:00 Jeremy discusses the debates about highly selective conferences but not as much of a shift as he would like to see – a hard thing to let go of if you have been arguing for selective conferences all your career. And acceptance rates being arbitrary.
17:10 We note the impact on younger academics and career pressures and I Ask about alternative suggestions? Jeremy talks about not being obsessed with numbers, focusing on quality, and re-thinking deadlines and acceptance rates.
“On search committees, writing tenure letters, it is in our collective interest to not be obsessed with numbers and to focus on the contribution and on the quality of the work.”
“As a junior person it is easy to get caught up in the perpetual motion machine mentally where you are constantly spinning out new papers.”
“If we focused on writing a smaller number of better papers, accept rates might go up.”
The shift to multiple deadlines or being able to submit anytime means you can submit when the work is ready.
21:00 Jeremy talks about it as a classic social dilemma problem and possibly experimenting with charging but then the equity issues this opens up, and also issues around review karma. And saying no to review requests.
26:55 Jeremy talks about stepping back from being an associate editor because of how hard it was to get reviewers and doing more reviewing again. The asymmetry of power to say no, and of information around reviewing and service - the lack of transparency into what other service people are doing when they do say no.
29:50 Jeremy discusses how we can become blind to some processes and assumptions when we come into a field and need to stop and look around.
“We’re smart observant people. If we look around and have these conversations we can come up with a way to fix this. This is a solvable problem but it takes stepping back, noticing and talking about it.”
31:00 Jeremy talks about planning the next CSCW conference and the experiments they are trying, and how to promote more social interaction.
33:00 They have appointed a virtual attendance task force to think about how to address this e.g., through more structured activities, ways to have random encounters, but not trying to replicate face to face.
34:30 We discuss CSCW as the area that has been researching distributed collaboration since the 80s and dealing with distributed and online now in the pandemic. And finding the ‘beyond being there’ moments.
Queer research:
36:35 Jeremy discusses his shift in research from more pragmatic interests (publishable and fundable) to sexuality and gender studies. Also coming out when he was 25. And starting on this new research area looking at CraigsList ads and then Grindr (leading to Charlie’s undergrad thesis) and the encouragement of Fred Turner to look at this as research.
42:45 Jeremy discusses some of the research studies he is doing in this space, including working with collaborators in India and learning so much.
47:00 Jeremy responds to the question about whether there have been personal challenges in coming out and he says not really and the HCI community being very open and welcoming.
49:30 Jeremy reflects on how the community could still change and do better. “There are some hard conversations ahead and hopefully they can be productive conversations.”
“There’s something I really enjoy about throwing myself in a situation where all of my assumptions are very likely to be wrong and trying to figure out where to go from there.”
Values & superpowers
53:00 Jeremy reflects on other values driving his work – addressing real problems in a way that can impact broader understanding, thinking about Pasteur’s Quadrant. And the superpowers he brings – a naïve curiosity and being willing to ask questions at every stage.
57:20 Wrapping up.
59:09 End
Related Links
Acronyms:
CHI Computer Human Interaction
CSCW 2021 conference - Computer Supported Cooperative Work
People:
John Tang’s recent podcast episode on reviewing
Gillian R. Hayes: Inclusive and Engaged Research. CHI2019 SIGCHI Social Impact Award talk
Mary L. Gray Microsoft Research, Harvard Uni Klein Center for Internet and Society
Fred Turner, Stanford
Jed Brubaker, Colorado
Papers/Books:
Jim Hollan and Scott Stornetta. 1992. Beyond being there. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '92). Association for Computing Machinery, New York, NY, USA, 119–125.
Katta Spiel, Os Keyes, Ashley Marie Walker, Michael A. DeVito, Jeremy Birnholtz, Emeline Brulé, Ann Light, Pınar Barlas, Jean Hardy, Alex Ahmed, Jennifer A. Rode, Jed R. Brubaker, and Gopinaath Kannabiran. 2019. Queer(ing) HCI: Moving Forward in Theory and Practice. In Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems (CHI EA '19). Association for Computing Machinery, New York, NY, USA, Paper SIG11, 1–4. DOI:https://doi.org/10.1145/3290607.3311750
Donald E. Stokes, 1997, Pasteur’s Quadrant: Basic Science and Technological Innovation. Brookins Institution Press.
