Lone Malmborg on academic performance measures, benchmarking and strategies

Lone Malmborg is an Associate Professor and heads the Interaction Design Research Group and the People and Computational Things Section at the IT University of Copenhagen in Denmark. She talks about what is happening in Denmark and ITU around performance measures for academics. She reflects on the impacts of what gets counted and how counts get benchmarked and what this means for things like publication strategies and stress levels. She also shares strategies that she has tried out in her own section to turn individual counts into cooperative activities, as well as her personal strategies.

[On performance measures:] “We know that big ideas take a lot of failing but we can’t afford failing. If we’re stuck with performance measures, we have to get the foundation of the model right or fair so we’re not measuring ourselves against something completely impossible. What is giving people stress is not having given tasks, but always having tasks you can never fulfil.”

She talks about (times approximate) … 

01:30 Her early studies, then working, and making a decision to pursue a PhD opportunity that was offered to her; PhD on limitations of formalisations

06:35 What drives her now in her research – wanting to do things that makes a difference to people, working with seniors, having agendas with a political and social part around technologies

10:00 Getting funding resources for these sorts of societal challenges being easier than for basic research; Challenges with different value systems between funding agendas with expectations vs seeing seniors as resourceful people and quality of life – a fight to get this agenda on board

12:37 Not being able to get the big funding for these areas in the same way as other areas of science

13:48 What’s going on in Denmark now re benchmarking all research and measures at IT University (all faculty ‘have to’ spend 1 million DKK (Correction: should be 878,000 DKK) each year) but being difficult when you don’t need to spend money on expensive equipment

15:34 Impacts of benchmarking of funding on culture in the university; Measurements being about efficiency, and not trusting people to do their best – introduced in all sectors; and what makes an ‘efficient academic’

18:00 Started counting teaching, ‘student production’ and various bonuses eg if students graduate on time, and how this can lead to lower requirements at exams

19:30 Now counting publication - all publication channels grouped into levels and credits/points for publication channels, and numbers of authors; Measuring how much people teach, publish, how much external funding they attract.

21:23 Long process set up by Ministry of Science and Education; being a member of one of the groups setting up the publication channels/levels; implications for new publication venues; and difficulties arguing for HCI conferences.

24:44 Creates all kinds of strange publication strategies – rather than picking exactly the right journal in terms of the topic, you pick the publication venue that gives you the highest credit; Universities then having very different ways of handling this – ITU ‘harvest’ twice a year to see points for publications, but no points or visibility for publications rejected or proposals rejected – how much work that is not recognized

27:10 Very unhealthy in terms of allowing yourself to fail, risky to try out new ideas, and supports research strategy where you never fail, but good research requires a lot of failing – see the consequences of this is boring research

28:35 Strategies for helping people in her section to deal with the stress – moving to a collective model to give people the feeling of helping each other out; Series of workshops with all section faculty once/year to discuss funding strategies and having access to an external company that helps write the applications; value for junior faculty to learn the process

32:00 Retreat idea first invented as a paper writing retreat around a conference deadline – structured writing activities – coming with an idea and leaving with a draft paper; taking shared responsibility; value of support of external companies

35:44 strategies for helping people manage stress

39:40 Her strategies as head of section – writing retreat, creating new relationships between people as side benefit, changing the way people work together

41:37 Personal strategies for dealing with this – having something very tangible as an output that satisfies her in another way e.g., cooking, eating together, something you can see the immediate result of

43:50 Being quite seriously ill giving her a new perspective, to focus on what is important; liking her job, her colleagues, but work being never ending and able to work for 12hrs/day easily; so buying a country house after illness and being immersed in picking up the weeds or painting the house that keep her attention on other activities

48:30 Going into a new managerial position and trying to make some decisions about how to be a good manager and not put stress on her colleagues; one example is avoiding sending emails to people in the evenings, also leaving office at 4:30 then people shouldn’t feel bad if they leave at 4:30; People shouldn’t work for free but can’t see how performance measures can be done in 37 hr work week

51:08 Finding arguments for other ways of benchmarking our research, not against national average but other departments in our area; a way of compromising that if we have to have benchmarks, they need to be more realistic; No other area with this amount of quality control that we have – acceptance rates of conferences as a form of quality control so why do we need another one

53:33 “Everybody is doing it the best they can. We are so longing for the honour of being a good researcher. It’s what drives us, we want to be the best.”

54:07 We have so many ways of making sure that people are doing good work. We are just creating stress that prevents people from doing deep thoughts; We know that big ideas take a lot of failing but we can’t afford failing

55:00 If we’re stuck with performance measures, we have to get the foundation of the model right or fair so we’re not measuring ourselves against something completely impossible. “What is giving people stress is not having given tasks, but always having tasks you can never fulfil. … We have to take this feeling away one way or another. … It’s so unhealthy.”

57:29 End

Related Links

Lone's blog https://lonemalmborg.wordpress.com/

Give&Take Project http://givetake.eu

This podcast uses the following third-party services for analysis:

Chartable - https://chartable.com/privacy