Talk - Representing epistemic and disciplinary diversity in open research

Talk – Representing epistemic and disciplinary diversity in open research

On 10 February 2024, I gave a lightning talk at FOSDEM 2024's Open Research Online Devroom titled “Representing epistemological and disciplinary diversity in open research discourse” (slides shared here). It was informed by incredible feedback I received from various open research communities. There's so much good stuff I couldn't fit them into a 10-minute lightning talk, so I'm putting them here.

I'm posting:

a transcript of my talk;
additional discussions that didn't fit; and
other resources/further reading that couldn't fit (scroll down to see!).

I'll try to clean up this post with more context and details on a best-effort basis.

Transcript

My very sciency background started in ecology and environmental research 15-ish years ago, including evaluating impacts from a major marine oil spill to wildlife. During those years, I heard about a fellow PhD student getting their thesis criticised by a committee member (i.e. examiner) because their work is not “reproducible” and not structured with explicit “hypotheses” and tests of those hypotheses.

I remember myself wondering why should scientific research be defined by reproducible experiments? The oil spill I studied is fundamentally not reproducible. And even if it were, it's probably not ethical to reproduce it! I also didn't conduct any experiments. Does that mean I am doing bad science?

In the years since, I've become an advocate for open science as a way to do good science, where I think what makes good science different from alchemy is best summed up by Cory Doctorow, who said: “Alchemists kept what they knew a secret for 500 years. They didn’t advance the art very much… and each of them learned in the hardest way possible that drinking mercury is a bad idea.”

I also learned that the Latin origins of the word science comes from Latin “scientia”, i.e. “knowledge”.

This prompted me to reflect more deeply on “open research” instead of just “open science”; and how open research could encompass diverse ways of learning and organising what we know.

For the purposes of my lightning talk, this is what I mean by epistemic diversity. My fear is that conversations about open research is not representative of epistemic diversity.

For example, we encourage people to share open data, but what does that mean to, say, a scholar of medieval literature?

There is much focus on “reproducibility”, but would a law professor care about this?

We have made progress on publishing open access papers, but does that mean anything to a musicologist?

I think it is possible for us to shoehorn these concepts into these disciplines (e.g. perhaps medieval books or sheet music are the “data”, and the law professor should document their reasoning and arguments in a “reproducible” way), but why should we? What if these researchers get to define research – and open research – in their terms instead? What might open research look like then?

Another way to look at this is to ask: who gets to decide how to conceptualise and talk about research? Who holds the epistemic power to define the terms of the conversation? In my view this is not an abstract problem.

Last year, I had a long conversation with a new history professor who has an interest in open research. And he told me about how groups about open research are often dominated by researchers from very STEM-focused subjects like the life sciences. And he doesn’t see any researcher who looks like him. He told me that there is resentment and a sense of exclusion among other historians.

In other words, despite good intentions, the epistemic power of the loudest voices in open research discourse might have accidentally caused epistemic injustice which systemically excluded some researchers.

People often ask “what can I do as an individual”?

A core part of open research is diversity and inclusion and over the years I've learned much (and have much more to learn!) on how words matter when it comes to dimensions like race, gender, or accessibility. I suggest applying that same sensitivity to epistemic diversity.

For example, the word “manuscript” could mean a paper submitted to a scientific journal, or a physical piece of ancient paper that a historian studies. I hear the terms “lab” and “PI” a lot in open research communities, but there are research disciplines whose social structures are very different and don't use these terms at all. Or, sometimes I hear talk of “STEM” vs “non-STEM” research, but that is itself a very STEM-centric view.

And, of course, the conflation of “science” and “research” as if they're the same thing. If we can be mindful of epistemic diversity when talking about open research, then maybe we can start to avoid excluding people from the conversation.

Finally, as someone from a very sciency background, I am in a privileged position. My imagination is constrained and it's not my place to authoritatively declare what we should do. That's why I'm hesitant to prescribe “say x instead of y”.

Instead, there may be value in elevating under-represented groups through gatherings like focus groups or workshops, where we can learn from epistemologically diverse researchers directly on how to make open research more inclusive.

Over the past weeks, I received amazing suggestions on where this topic could go, and my lightning talk is just the tip of that iceberg.

In the mean time, I’d like to give thanks to them, including: The Turing Way community, Framework for Open and Reproducible Research Training, Nowhere Lab, Gathering for Open Science Hardware, and the organisers of this FOSDEM Open Research Devroom!

Additional discussions

In no particular order (don't have time to organise ATM), here are some other ideas which came up in the Turing Way, FORRT, Nowhere Lab, or GOSH communities (acknowledgements at the end of this post):

Start with 'the idea of open scholarship and then narrow to “open science” or “open research” if needed, depending on who I'm talking to'.
”...it’s not just an issue in the open science/research movements, but interdisciplinary fields in general and any inter-/multi-disciplinary attempts to change research practices need to adopt epistemological flexibility & tolerance towards other”
There is interest from the FORRT community to write something about this.
Also possible for me to re-present this lightning talk at an upcoming Nowhere Lab meeting, maybe in March 2024.
Classification can be confusing for some researchers, especially those who don't fit in typical “STEM” boxes.
Sabina Leonelli's work on open science and philosophy of science, linked to below.
Harding’s sciences from below and Fricker’s epistemic injustices are also useful reading (see below).
There is a Digital Humanities group for the UK and Ireland: https://digitalhumanities-uk-ie.org
- Including research software engineering: https://digitalhumanities-uk-ie.org/community-interest-groups/research-software-engineering/
There are different levels of abstraction when talking about epistemic diversity, which is a separate deep dive on its own.
'Beware the “pageant effect”: you're likely to learn the amazing successes of an unfamiliar discipline before learning about its flaws and failures.'
Connections to science, technology, and society (STS) studies, link to epistemic power and (in)injustice.
We had a conversation about the good intentions, usefulness, but also limitations of the CReditT taxonomy for research contributor roles (readings below). Though there is further work on improving it like SCoRO.

More resources

Event by the UK Reproducibility Network: How relevant is the open research and scholarship agenda to the arts, humanities and social science disciplines? (warning: YouTube link)

Humanities Commons: https://hcommons.org/

Replicable History Project: https://ljmu.libcal.com/event/4130747

Works by Karen Barad: https://en.wikipedia.org/wiki/Karen_Barad

Joint meeting of the European Association for the Study of Science and Technology (EASST) and the Society for Social Studies of Science (4S): https://www.easst4s2024.net/

FOSDEM talk: FLOSS meets Social Science Research (and lived to tell the tale): https://archive.fosdem.org/2021/schedule/event/open_research_floss_meet_social_science/

SCoRO, the Scholarly Contributions and Roles Ontology: http://purl.org/spar/scoro

Research Software Engineering in the Arts and Humanities: https://digitalhumanities-uk-ie.org/community-interest-groups/research-software-engineering/

Unless otherwise stated, all original content in this post is shared under the Creative Commons Attribution-ShareAlike 4.0 license (CC BY-SA 4.0).

naclscrg