From The Turing Way

Anne reminded me the importance of recognising/avoiding what I would call the trap of “performative objectivity”, i.e.:

...traditional 'social science' (which can often fall prey to the same ideas of being 'objective' and 'removed' from the communities they are studying or working with, in their desire to be more aligned with the hard as opposed to 'soft' sciences)

This then reminded me of the excellent book Data Feminism:

https://data-feminism.mitpress.mit.edu/

This book was a great read, and contains — among other things — great examples of projects which avoid that trap, and creative ways to share and conceptualise “data”.

With that in mind, here's a good example of community-led research:

https://grassrootsjusticenetwork.org/resources/community-action-guide-on-community-led-research/

Engaged and public anthropology can be creative in its outputs (as opposed to “traditional” academic outputs), such as an open city documentary festival:

https://opencitylondon.com/about-us/

Or a set of walking tours:

https://open-city.org.uk/events

Anne also mentioned that Paz Bernaldo from Open Life Sciences and Beth Duckles from “organisational mycology” (so curious what this term means!) may have insights, too.

Beth later responded with some very useful examples, saying that:

...participant action research (PAR) and community based participatory research (CBPR) are another set of methodological approaches that seeks to reconsider subject/object of the research by including the folks “being studied” in the research leadership, essentially trying to lessen the power inequalities and make the research more community led. It comes out of a social justice lens, particularly Paolo Friere's work.

One of which is a piece of open participatory research that makes “the research process, data collection and analysis open to those who were able and interested in being a part of the process”:

https://zenodo.org/records/8015576

There are also online repositories for publishing open social science outputs, such as SOCARXIV and SOAR:

https://socopen.org/

https://www.gesis.org/ssoar

Beth also linked to a discussion about “open social science”:

https://blogs.lse.ac.uk/impactofsocialsciences/2022/01/11/eight-components-for-open-social-science-an-agenda-for-cultural-change/

And Angela Okune worked for a long time with a community in Kenya, published on the Platform for Experimental, Collaborative Ethnography (PECE: pronounced “peace”):

https://worldpece.org/

Anne also curated a list of tools for social science researchers:

https://open-source-social-science.github.io

Even though it's a list of tools, tools affect the questions we could ask and I think Anne's list can serve as an inspiration for what kinds of open research one could do!

From FORRT

Priya reminded me of the wonderful work she and others at the UK Reproducibility Network did to collate examples of open research across disciplines:

https://www.ukrn.org/disciplines/

https://doi.org/10.31219/osf.io/3r8hb

Aleksandra shared a platform her lab established to share psychological methods/tools translated into Serbian. To me, this is a great example not because it's about psychology research, but because that it's a community effort at translation, making resources accessible to a different audience:

https://www.repopsi.f.bg.ac.rs/en/

Flavio shared a great paper about “Teaching open and reproducible scholarship: a critical review of the evidence base for current pedagogical methods and their outcomes”. An important reminder that pedagogy/teaching is a key component of open research:

https://doi.org/10.1098/rsos.221255

Also, FORRT resources on adopting open research and replication:

https://forrt.org/adopting

https://forrt.org/replication-hub/

On a more meta level, Flavio noted that FORRT itself might be a good social sciences example of people coming together as a community to build something.

From NASA TOPS

Christine shared two very cool resources.

SEEKCommons, which seeks to promote “the 'commons' in science and technology with an emphasis on collaborative socio-environmental research”:

https://seekcommons.org/about.html

And the ICPSR (Inter-university Consortium for Political and Social Research), which “provides leadership and training in data access, curation, and methods of analysis for the social science research community”:

https://www.icpsr.umich.edu/web/pages/about/

Citizen science

There's also lots of work in the citizen/community science circles that may be good examples of open research in social sciences.

For example, the classic story I always tell is about Public Lab:

https://publiclab.org/

Where their famous open source balloon mapping kit — originally for mapping the spread of the 2010 Deepwater Horizon oil spill — was adapted by those in the Bourj Al Shamali refugee camp to see their urban space from above for the first time:

https://placesjournal.org/article/camp-code/

The story in this article is an inspiring example of open research. And, the article itself is open research by Claudia Martinez Mansell, sharing her work in a public way.

Other resources

A great talk about open qualitative research by Natasha Mauthner, Professor of Social Science Philosophy and Method at Newcastle University: https://oercommons.org/courseware/lesson/134043/overview

Qualitative research software

Qualcoder is an open source replacement for proprietary qualitative research software like NVivo:

https://github.com/ccbogel/QualCoder

Some Qualcoder learning resources:

• https://guides.library.illinois.edu/c.php?g=997192&p=10050831#s-lg-box-31727131
• https://shsulibraryguides.org/az/scworkshops/qualitative-data-coding-intro-to-taguette-and-qualcoder
• https://ndporter.github.io/open-qualitative-research-qualcoder/04-qualitative-data-analysis.html
• https://guides.temple.edu/qda/qualcoder

And there's also Taguette which seems much more user-friendly:

https://www.taguette.org/

To complement Tageutte, I also learned of a new tool which allows you to view your codes in different ways:

https://qdb.n.gardella.cc/

Acknowledgements

In alphabetical order.

The Turing Way

Anne Lee Steele, Beth Duckles

Framework for Open and Reproducible Research Training (FORRT)

Aleksandra Lazić, Flavio Azevedo, Priya Silverstein

NASA TOPS community

Christine Kirkpatrick

#openresearch

Unless otherwise stated, all original content in this post is shared under the Creative Commons Attribution-ShareAlike 4.0 International license

The slides are published on Zenodo with DOI 10.5281/zenodo.11051128 listed under the “30 minute version”. I will try to gather here:

• the video recording;
• short summary;
• further reading collected when developing the talk; and
• a transcript of the talk.

I'll try to clean up this post with more context and details on a best-effort basis.

Video recording

There is a live video recording made during my 22 November 2024 talk which is viewable on the Internet Archive. The video is also embedded here (click the “CC” icon for subtitles):

Short summary

Please see the notes for my original “AI” talk for additional information.

Aware of the irony, I was curious how a large language model (LLM) could take the transcript of my talk (see below) and infer a short summary. The following is what Claude 3.5 Sonnet produced, with some edits by me:

This talk came from my conversation with Jennifer Ding at the Turing Institute about which underlying issues around “AI” technology deserve more attention versus the overhyped aspects. While I acknowledge that new technologies like “AI” can bring positive changes – such as a helpful Speech Schema Filling Tool that helps chemists record experimental metadata in real time as they run experiments – I wanted to focus on several key concerns.

The first observation I made is how “AI”-generated content is affecting academia. I shared examples including a published paper that began with “Certainly, here's a possible introduction...” (clearly ChatGPT-generated) and most amusingly, a paper featuring an anatomically incorrect lab rat with comically oversized genitals that somehow made it through peer review. I've also noted evidence of academics using “AI” tools for both writing and reviewing papers, and even PhD programs where applicants and reviewers use “AI” to convert application letters between bullet points and prose.

I emphasized that words really matter in this discussion. “AI” has become more of a marketing term than a technical term of art, and I pointed to how papers from just before the “AI” hype rarely used the term for the same technologies. I argue that this ambiguous language serves as a smokescreen, shifting power to those who control these tools.

This led me to discuss how “AI” often masks human exploitation. I shared examples including Kenyan sweatshop workers traumatized by moderating graphic content for ChatGPT, their Indian counterparts manually tracking purchases in ostensibly automated Amazon Fresh supermarkets, and bus drivers in “driverless” buses who must remain hypervigilant for that 1% chance of needing to intervene. As Kate Crawford notes, “AI” is “neither artificial nor intelligent” – it's not replacing labor but rather making it more invisible (which Lilly Irani also discussed in depth).

For scientific research, I see several concerns. There's a growing trend of papers proposing to replace human participants with large language models or suggesting complete automation of the scientific process – with one paper proudly claiming it could produce entire research projects from ideation to paper publication for just USD 15 each. I warn that building science on top of opaque and unaccountable “AI” systems risks turning science into alchemy.

While some suggest banning “AI” in academic publishing (following incidents like the well-endowed lab rat paper), I caution that focusing solely on “AI” (“solely” being the key word) might entrench deeper problems like the broken peer review system and publish-or-perish culture. For example, publishing companies might offer proprietary “AI”-generated paper detection tools, which would make us more reliant on them and further consolidating their power without tackling why researchers feel pressured to publish fake papers in the first place.

My key message is that “AI” often highlights existing problems rather than creating new ones. Instead of fixating on “AI” itself, we should address underlying issues in research culture, from job security to toxic workloads. I concluded by recommending resources like the Mystery AI Hype Theater 3000 podcast and the book “AI Snake Oil” for those interested in deeper exploration of these themes.

P.S. Note that a newer book, “The AI Con”, is about to be published in 2025: https://thecon.ai/

Further reading

Please see the notes for my original “AI” talk for links and references in addition to what's here.

• [report] Amazon’s AI Cameras Are Punishing Drivers for Mistakes They Didn’t Make: https://www.vice.com/en/article/amazons-ai-cameras-are-punishing-drivers-for-mistakes-they-didnt-make/
• [report] Amazon Fresh kills “Just Walk Out” shopping tech—it never really worked: https://arstechnica.com/gadgets/2024/04/amazon-ends-ai-powered-store-checkout-which-needed-1000-video-reviewers/
• [report] Look, no hands! My trip on Seoul's self-driving bus: https://www.bbc.co.uk/news/business-68823705
• [podcast] Mystery AI Hype Theater 3000: https://www.dair-institute.org/maiht3k/
• [editorial] The advent of human-assisted peer review by AI – in Nature Biomedical Engineering: https://doi.org/10.1038/s41551-024-01228-0
• Words matter, they affect the way we think about issues:
  • [essay] Stefano Quintarelli is a former Italian member of parliament who said that instead of “AI”, we could call those technologies “Systematic Approaches to Learning Algorithms and Machine Inferences (SALAMI)”: https://blog.quintarelli.it/2019/11/lets-forget-the-term-ai-lets-call-them-systematic-approaches-to-learning-algorithms-and-machine-inferences-salami/
  • [podcast] Completely randomly, I heard another “AI” replacement term “Technical Oriented Artificial StupidiTy (TOAST)” coined by Chris Roberts in the middle of a gaming podcast (19:31 into the video): https://www.youtube.com/live/ADYB-QJGheA?feature=shared&t=1171
• I didn't get to talk about the environmental costs of scaling (or the urge to scale up) “AI” technology, Timnit Gebru of the DAIR Institute touches on this and other issues in this interview (57:58 into the video):

Books

Hanna, A., & Bender, E. M. (2025). The AI Con—How to fight big tech’s hype and create the future we want. Harper. https://thecon.ai/

Narayanan, A., & Kapoor, S. (2024). AI Snake Oil: What artificial intelligence can do, what it can’t, and how to tell the difference. Princeton University Press. https://press.princeton.edu/books/hardcover/9780691249131/ai-snake-oil

Academic literature

Argyle, L. P., Busby, E. C., Fulda, N., Gubler, J. R., Rytting, C., & Wingate, D. (2023). Out of one, many: Using language models to simulate human samples. Political Analysis, 31(3), 337–351. https://doi.org/10.1017/pan.2023.2

Bender, E.M., Gebru, T., McMillan-Major, A., Shmitchell, S. (2021) On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜. In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency (FAccT '21). Association for Computing Machinery, New York, New York, United States, 610–623. https://doi.org/10.1145/3442188.3445922

Gu, J., Liu, L., Wang, P., Theobalt, C. (2021) StyleNeRF: A Style-based 3D-Aware Generator for High-resolution Image Synthesis. arXiv, 2110.08985. https://doi.org/10.48550/arXiv.2110.08985

Transcript

This started from my conversation with Jennifer Ding at the Turing Institute. And we were talking about: what are some of the underlying issues around “AI” technology that we feel should be surfaced a little more rather than some of the stuff that we think is a little overhyped? And I'm gonna go over a lot of those problems today.

Before I get into it, I want to do something I always emphasize in talks like this, which is that I think for any kind of technology, it can bring about a lot of change in how we do things and how we organize ourselves. And it's not a matter of saying: oh, you know, let's just not use it. There's a potential for “AI” technologies, right? Because if you think about it, when the printing press came around, you don't want to ban the printing press just because you're afraid that the scribes are gonna go out of business. We hopefully can work together to find a way to realize the potential of a new technology.

And I think a positive example that I'd like to share before jumping to everything else is this tool that Shern Tee shared with me. It's called the Speech Schema Filling Tool. So it was developed by chemists for use in their experiments. And what happens is that as you do your experiments, you talk into the microphone on your computer and the large language model on it will use your audio input to do a speech to text conversion and fill in your lab notebook with what you're saying. But what's really cool about it is that the tool will also parse what you're saying and record relevant metadata into a structured data format to go with your lab notebook. So there's a very well-structured metadata set to go with the particular experiment that you're doing. And I think as long as you're happy to talk through your experiment as you're doing it, this tool is so helpful for you to improve the quality of the data that you're capturing, helping make your experiments more reproducible and so on, right?

So there are certainly really good uses, of what people are calling “AI” technologies these days. Having said all of that, obviously there's also a lot of concern that we've seen over the past couple of years, such as in terms of how people publish papers, right? This is a classic one I think Marcus shared a while back where if you look at the paper, starting right from the first sentence in the introduction, it says: “Certainly, here's a possible introduction for your topic.” And I think it's pretty clear that this probably came from ChatGPT, which is one of the more commonly used so-called “AI” tools today to generate text.

However, this is not my favorite one. So my favorite paper is this one. I don't know if some of you have seen it. I see some of you smiling, so you know what I'm getting to. First of all, this was published in Frontiers back in February [2024]. If you look at the text, a lot of it looks fairly generic and probably “AI”-generated. But the most dramatic part is one of the figures which was a lab rat. And most of the lab rat looks kind of like a normal rat, but it's got these giant genitals sticking out of it. For the phallus, it's so long that it extends beyond the figure.

I just love how a figure like this would get past the peer reviewers, it gets past the editors, it gets past the copyeditors of the journal and gets published. Now, for the record, it was retracted by the publisher pretty soon afterwards. But not after everyone on the internet got copies of the PDF first and then archived it. That's how I was able to get this amazing picture of this lab rat, which I love. And you can also see a lot of weirdly spelled words that annotate this figure. So definitely check it out. I think this is one of the classics that's come out of some of the papers we've seen over the past couple of years.

And in addition to generating these papers, we are also seeing some evidence that academics are using these tools to generate the peer reviews that they write. And to be honest, I can kind of relate to what these academics are going through because who has time, right, to do a really good peer review these days? And in higher education, of course, we know that some students feel really tempted to use these sort of [large] language models to generate their essays, and we're also seeing that some instructors are using the same tools to grade and mark the essays.

You know, there's an anecdote I heard for a PhD program that was recruiting students, I think it was in the US, they found that a lot of the applicants to the PhD program didn't have time to write so many cover letters in the application. So they would write a few bullet points saying what they want in their cover letters. They use a large language model, turn it into the cover letter. And then when the professors on the program, they have so many applications to sift through, they ask the same tool to translate it back into bullet points so that it's quicker for them to skim through.

So a lot of interesting use cases here, but I just wanna use this to set the stage to talk about three things today. So the first one is that I think words really matter when we talk about so-called “AI” technologies because there's a lot of ambiguity in the language. And that can become really problematic because it allows so-called “AI” to become a smokescreen that distracts us from what I think a lot of the underlying issues are. That's more important to tackle. And lastly, I will try to bring all of this back to scientific research and think about what this means for scientific research and maybe what it doesn't mean.

Okay, so what do I mean by words matter? Well, I think it's very important for us to realize that so-called “AI”, as we colloquially use it today, is very much just a marketing term and not a technical term of art!

To illustrate this point, I really like this paper. It's called “A style-based 3D-aware generator for high-resolution image synthesis.” And you can see that you can use this tool to generate very realistic-looking photos of people. And I use this example because I searched through the whole paper, including the title, and other than one of the affiliations of the first author, there's no mention of “artificial intelligence” in this paper at all.

And if you look at the publication date, it's 2022, just before all of the hype around “AI” started. And I think if this paper is published just a year later, the text is going to be filled with references to “artificial intelligence”. And I think this is really important because it comes back to the point that a lot of the terminology we're using today around these technologies are marketing terms, like hallucinations or reasoning skills or training these models.

First of all, it really anthropomorphizes this technology, and it gives us a sense kind of like how humans have a tendency to recognize faces in things. And I feel using this terminology misleads us into recognizing intelligence in these tools as well. And I think that can be really problematic.

Another way to think about it is that when we are using our word processors to type up our papers, there's spellcheck, right? And spellcheck is basically a statistical model that takes an input and infers, in this case, the possible correct spelling for the word you're trying to spell. And this is not to minimize the amazing amount of work that's gone into these artificial intelligence technologies, but roughly speaking, large language models are also a very, very sophisticated form of statistical modeling that takes text as input and infers a natural-looking output.

And I think Emily Bender describes it really well when she calls these models “stochastic parrots”, because parrots, they might repeat words back to you, but they are literally incapable of understanding what it's saying. And this also applies to all of these “artificial intelligence” technologies.

And I think this ambiguous language is the feature, not the bug, because it's not just a matter of linguistics or semantics or nitpicking, but we know from history that ambiguous language shifts power to people who hold control over those tools and technologies. And I feel that the powerful people behind so-called “AI” is using this ambiguous language as a smokescreen to distract us from the very real problems underneath it.

So just, I think it was last year where there was this union that was formed in Kenya, because there were so many sweatshop workers in Kenya that were hired by the company behind ChatGPT and also Facebook and other companies to, well, as you can see here, to make the models less toxic.

So what they do is that you're constantly looking at outputs for the most egregious stuff, such as descriptions of sexual abuse, murder, suicide, and other really graphic details. And they're basically tweaking the model inputs whenever something really graphic comes out [so that] the statistical inferences from these large language models are slightly less offensive.

And they're so traumatized by this and doing this kind of sweatshop work all day, every day, trying to keep ChatGPT working that they were able to actually form a union. And I think this is important because that chemistry example I gave you earlier was one of the “AI” assisting humans, right? But actually, a lot of the exploitation comes in when you have a human-assisted “AI”, such as these sweatshop workers.

Another one is, of course, Amazon Fresh. I took this picture of the Amazon Fresh store. This one is just south of Aldgate East Station in London. And I know some of you know this... So the selling point for Amazon Fresh is that you walk in, pick up whatever you wanna buy, and you just walk out. And they use really advanced “artificial intelligence” to all of the cameras in the shop will figure out what you bought and automatically charge your Amazon account.

But it also came out in the news this year [2024] that all of the so-called “artificial intelligence” was actually Amazon hiring sweatshop workers in India whose sole job is to watch all of those cameras and manually tag what people are buying in these shops when everyone is thinking that's actually the “artificial intelligence” technology doing all of those things.

And actually, Amazon shut down the whole thing soon afterwards, and they're actually shifting Amazon Fresh to one where, rather than having all of those cameras watch you, whenever you grab an item, you have to manually scan it into your cart before you take it out.

And the other example that I think is very, very telling is this piece of news that was in the BBC earlier this year [2024] about this new driverless bus route that was started in Seoul in South Korea. So what happens is that this bus is supposed to be completely driverless, right? And you can see a picture of this guy sitting on the [driver's seat].

So I like this picture, by the way, of how this person actually also has his feet up to indicate that he doesn't even have his feet on the pedal. And I wanna use this example to say that all of what I've been showing to you so far are cases of human-assisted “AI”.

And what this driver has to do, you might be asking, “Okay, if this bus is completely driverless, why do you still need someone to sit there?” So what happens is that this driver will sit in the driver's seat. They don't usually have to do anything, like 99% of the time they can just sit and watch the bus drive itself, but this bus driver has to be super vigilant the whole time. Just in case, you know, in that 1% of the situations where the driverless bus makes a mistake, this driver has to immediately react and come in and actually make an adjustment to whatever the bus is doing.

So this driver is actually more vigilant than they usually have to be if they were just driving a regular bus. And this is what we're also seeing, of course, of the Amazon delivery drivers who are [used by] the so-called “artificial intelligence” system. You know, it's constantly watching the drivers on these trucks as they make their deliveries.

And they're under so much pressure because on one hand, Amazon is constantly pressuring them into making their delivery quotas. On the other hand, this “artificial intelligence” disciplinary system is constantly watching their behavior, such as watching their eyeballs [to track] where they're looking. There's also some evidence that the camera is watching their lips because apparently some drivers, they would whistle or sing a tune as they're driving, and apparently that's a bad thing and you'll get marks taken off and you might not get your bonus at the end of the week. So they're constantly being disciplined like this.

Or they have to deal with these inhuman competing demands. And in these examples, it's like, you know, us humans, we're basically mindless bodies where the “AI” acts as the head to discipline us and make us do exactly what it wants us to do.

And it comes back to my point where if we think of it as an “artificial intelligence”, then we attribute agency to this technology. And that distracts us from the Jeff Bezos-es behind the technology who's actually using them to exert that power over us. And I think that's really dangerous, right?

And I think Kate Crawford describes it really well, where so-called “artificial intelligence” is neither artificial nor intelligent. And the use of this technology in the ways that I just described, you know, it's not really replacing labor. It is displacing labor and making it even more invisible to us.

And this is why I think words matter because they have so much epistemic power over how we think about things. And often the use of language in “artificial intelligence” distract us from all of these underlying problems. Because, you know, if the “AI” on that driverless bus, you know, let's say hallucinates and makes a mistake, who are you gonna blame? We might blame, you know, that driver who wasn't vigilant enough to catch that 1% chance of the bus making a mistake, but is that really the issue here?

And that's where I'd like to try to bring this back to scientific research. So what does what we do as academic scientists have anything to do with this, right? Well, first of all, I'm kind of concerned about how even in academic scientific research, there is already sometimes a tendency to exploit.

So this is a paper that I actually cited in my previous research where it talks about, crowdsourcing the work that we do in science, whether it's data collection or data processing to online volunteers. And I want to first say that sometimes this can be done really well. For instance, a lot of this is integrated into science outreach and science education and science engagement, where as part of your engagement activity, the participant, they get to do part of the science and help you analyze data. And they can be mutually beneficial, but in papers like this, you often see language like, crowdsourcing, right? Which allows all of these free labor that you hired to shorten the time to perform the work for you, or it lowers the cost of labor for the academic who's running the project.

And I think there's a little bit of a danger here where we are perpetuating some of the exploitation, especially now where I am actually asked to review papers over time about this kind of crowdsourcing work and the way they talk about the participants make me concerned about where this is going in terms of various technologies where we might accidentally perpetuate this smokescreen that I keep talking about.

The second thing is that because the language around “AI” is so misleading we get papers like this who are, of course, it's basically saying it's so costly and labor intensive to recruit participants in your project. So why don't we replace them with large language models who will never get tired of our interview questions? We don't need to give them any compensation and we can get as many participants as we want in our study because, you know, they're as good as the real thing anyway, right? So I think that's pretty problematic.

Another one is talking about human assisted peer review in “AI” where they actually want to use these models to do peer reviews. And of course, proposing this particular editorial in this Nature Journal is that they're claiming: “oh, it's gonna save so much work for the actual peer reviewer because the 'AI' is gonna do all of it” and then the human, they just need to come in at the end and briefly check that peer review to see if it's okay.

But this sounds so much like that bus driver to me, and I feel we're seeing a lot of really high profile papers like this. There's one that I didn't get to stick into the slide in time, which is literally proposing, using “AI” to completely take over the scientific discovery process, where you're gonna use the large language model for question generation to design and conduct the experiment, analyze the result, write a paper, and then get another large language model to come into peer review that paper.

And at the end of the abstract, so I really wish I should put the abstract here, but at the end of the abstract it says, this saves so much money: “We calculated on average that if you outsource this entire thing to our 'AI' tool, it will be able to produce all of that scientific research for you at a cost of $15 per paper.” And I think that says a lot about how there's so much misunderstanding and hype around these technologies that high profile papers like this are starting to appear.

And I think Lisa Messeri described it really well where if we develop this kind of reliance and we think that “AI” technology is actually, sentient and intelligent, then by doing science this way, it will give us illusions of understanding. And this is a fantastic paper I suggest you check out.

Okay, now as someone who has been an open research advocate for a long time, another thing that's talked about, in “AI” circles right now is that we should really make a lot of these “AI” tools open source. And I think there are good reasons for that. But in the context of open research, there's a lot of messiness there as well.

So you might have heard of LLAMA 2, one of the large language models released by Meta last year. Then they called it an “open source” large language model. But if you actually click on download the model, it actually comes with a ton of restrictions on what you can do with it and a lot of limitations. And a lot of parts of it are completely opaque and you're not allowed to see what the model is doing. So it certainly doesn't meet the industry definition of open source as it has been established for software.

Now, the Open Source Initiative has been working on this issue for a long time. And actually just a few weeks ago, they released the first version of an open source “AI” definition. And I think it's really important for academic researchers to be part of this process as well.

But in any case, what happens in practice is that there was another study published earlier this year where they looked at dozens of the popularly used, large language models these days and scored them using 14 different criteria on their openness. And the overwhelming majority of them comes not only with a ton of restrictions, but also a lot of black boxes where you're not really allowed to know what's actually happening inside these models.

So you can see that ChatGPT is right there on the very bottom as one of the most black box large language models that there is that we're using. And I think there's a real danger here for... with all of this hype around so-called “artificial intelligence” and all the talk about completely integrating that into the science that we do. We're building all of the science on top of this “AI” technology.

I think what's gonna happen is that we won't end up doing science anymore. We will be doing alchemy! Because it's built on top of this completely opaque system. And I think that's a fundamental danger to the future of doing science.

And I want to quickly bring us back to this very well-endowed lab rat that I mentioned at the beginning, because I know that in response to papers like this, some people are saying, okay, so of course, you know, we should certainly ban the use of “AI” technologies in the creation of papers. So maybe we should just completely cut “AI” out of the paper writing process, right?

And I think that's understandable to a large degree, but I think there's a concern about if and what kind of problems are we actually solving if we focus on dealing with the “AI” part of it. Because I'm concerned that fixing “AI” might actually entrench deeper problems.

In this case, the broken peer review system, the publish-or-perish culture, right? Where these publishing monopolies... because I wouldn't be surprised, given what we've seen in higher education in terms of finding fake essays written by students. I wouldn't be surprised if one of those big publishers, they release some proprietary “AI” tool saying, “hey, if you publish a journal with us, then we'll let you use our proprietary 'AI' tool to detect fake paper submissions.”

That might seem to superficially solve the problem, but I think the deeper risk of thinking about “AI” is that in this example, we will become even more reliant on these huge publishers and cede even more power to them, right? And I think that's what I'm really concerned about because, solutions like this, don't really get at the actual problems leading to why people want, well, not necessarily want, but feel pressured into publishing those fake papers.

So I think a core message that I've got from these examples is that “AI” highlights existing problems that we have. And it's important for us to be aware of deeper problems in our research culture. And it could be really long standing issues like job security or the toxic workloads that we have to put up with, right? And think about all of those lecturers who have to live in tents because they can't afford anything more than that.

And it's important to realize that “AI” didn't create these problems just as “AI” didn't create the sweatshops that I mentioned earlier.

So to wrap things up, I think the main messages I want to send today is that words really matter when we talk about these technologies. And we should be very sensitive in understanding what those words really mean. And instead of thinking about “AI”, we should think about these deeper underlying issues that have plagued us for so long because, you know, very often “AI” is NOT the problem. It highlights existing problems and we should reflect on and focus on those underlying issues.

If we only focus on “AI”, it risks making those problems even worse. Okay, so that's the bulk of my talk, but if I've piqued your interest a little bit, I will leave you with some further reading, one of which is this one about generative “AI” and the automating of academia. The lead author is Richard Watermeyer based right here in Bristol. It's a fantastic read.

But if you're tired of reading yet another paper, I mentioned Emily Bender earlier. So Emily Bender and Alex Hanna host an incredible podcast called Mystery AI Hype Theater 3000, where every week they look at one of these so-called “AI” papers like the ones that I just showed you and tear it apart. And it's both very depressing and very entertaining at the same time.

Or if you'd like to read, these two Princeton professors, they wrote a book called “AI Snake Oil,” again, along the veins of what I'm talking about today. And I think it's really informative in terms of how we think about how we want to adapt our research culture in light of this new technology.

So that's some additional material that I think is useful. And in the interest of doing open research, I've published these slides, the transcript, additional notes, and all of the references to Zenodo. So you can look at that and remix and use it if you want.

And I also want to just give a shout out to Jennifer Ding from the Turing Institute and Shern Tee, and everyone from the Turing Way community who's helped me develop this talk.

So that's what I have for you today. And thank you for coming.

#talks #AI

Unless otherwise stated, all original content in this post is shared under the Creative Commons Attribution-ShareAlike 4.0 International license

...mathematical model scores feature which is an objective measure of novelty for scientific work. https://pharmaceuticalmanufacturer.media/pharma-manufacturing-news/latest-pharmaceutical-manufacturing-news/desci-labs-launches-novelty-scores-for-scientific-manuscript/

The article says:

...evaluating the novelty of scientific manuscripts and grant applications takes centre stage in the scientific peer review process. The primary reason work is rejected by editors of high-impact journals or funding agencies is because referees think it is not novel enough. However, the current peer review process is subjective, slow, labour-intensive, and prone to bias and inaccuracy [...] The release of these novelty scores [...] means there is now an objective, automated measurement of one of the core parts of the peer review process.

As a general principle, I assume goodwill. With that in mind, it is with genuine, all due respect that I find this development to be deeply alarming.

First of all, how can there possibly be an “objective” measure of novelty?????

Secondly, while it's great to see on DeSci Labs's about page some laudable goals like enabling FAIRness, open science, developing open source software, and preserving scientific outputs (which I care deeply about), the same page also speaks of securing USD 6.5 million in “seed funding”, accelerating science, using “Web3” technology, and to “accelerate growth and enhance customer loyalty”. To me, this reeks of techno-solutionism and -accelerationism.

Third, the underlying math is published in Nature:

https://doi.org/10.1038/s41467-023-36741-4

To me, all three of the above speak volumes about the state of scientific research culture, and not in a good way... 😩

Contrast this with the excellent essay on “The Limits of Data” by C. Thi Nguyen recently shared with the Turing Way community by Shern Tee:

https://doi.org/10.58875/LUXD6515

Which reminds us:

...policymakers and data users should remember that not everything is as tractable to the methodologies of data. It is tempting to act as if data-based methods simply offer direct, objective, and unhindered access to the world—that if we follow the methods of data, we will banish all bias, subjectivity, and unclarity from the world. The power of data is vast scalability; the price is context. We need to wean ourselves off the pure-data diet, to balance the power of data-based methodologies with the context-sensitivity and flexibility of qualitative methods and local experts with deep but nonportable understanding. Data is powerful but incomplete; don’t let it entirely drown out other modes of understanding.

I hope the work on reforming academic research culture and #metaresearch could include diverse and skeptical voices in addition to simply developing new quantitative “metrics”.

Unless otherwise stated, all original content in this post is shared under the Creative Commons Attribution-ShareAlike 4.0 International license

• Recording
• Transcript
• Further reading/resources

Recording

I've given several variations of this talk with multiple recordings. For now, here is the recording of an early iteration I gave at the Edinburgh Open Research Conference in mid-2023 (click on the “Presentation Video” link on the page):

https://doi.org/10.2218/eor.2023.8112

I will try to put other recordings here on a best effort basis.

Transcript

I will put a transcript of the talk here as soon as I can.

Further reading/resources

• The official Open Source Hardware Definition: https://www.oshwa.org/definition/
• OreSat open source cubesats: https://www.oresat.org/
• Public Lab is the group which developed the open source balloon mapping platform in response to the 2010 Deepwater Horizon oil spill: https://publiclab.org/
• Story about capturing photographic evidence of dumping toxic waste in the Mississippi River: https://publiclab.org/notes/eustatic/05-28-2013/kite-photos-of-ongoing-coal-pollution-in-plaquemines-parish-la
• Claudia Martinez Mansell is a humanitarian worker and independent researcher who worked at the Bourj Al Shamali refugee camp in Lebanon. It's the community there that remixed the Public Lab balloon mapping platform for use in their camp. Relevant reading:
  • https://placesjournal.org/article/camp-code/
  • https://publiclab.org/notes/clauds/04-28-2016/camp-code-how-to-navigate-a-refugee-settlement

Peer-reviewed papers

• Arancio, J. (2023). From inequalities to epistemic innovation: Insights from open science hardware projects in Latin America. Environmental Science & Policy, 150, 103576. https://doi.org/10.1016/j.envsci.2023.103576
  • Associated article: https://sparcopen.org/impact-story/often-overlooked-sharing-of-hardware-is-a-missing-link-in-open-science-puzzle/
• Burke, N., Müller, G., Saggiomo, V., Hassett, A. R., Mutterer, J., Ó Súilleabháin, P., Zakharov, D., Healy, D., Reynaud, E. G., & Pickering, M. (2024). EnderScope: A low-cost 3D printer-based scanning microscope for microplastic detection. Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, 382(2274), 20230214. https://doi.org/10.1098/rsta.2023.0214
• Collins, J. T., Knapper, J., Stirling, J., Mduda, J., Mkindi, C., Mayagaya, V., Mwakajinga, G. A., Nyakyi, P. T., Sanga, V. L., Carbery, D., White, L., Dale, S., Lim, Z. J., Baumberg, J. J., Cicuta, P., McDermott, S., Vodenicharski, B., & Bowman, R. (2020). Robotic microscopy for everyone: The OpenFlexure microscope. Biomedical Optics Express, 11(5), 2447–2460. https://doi.org/10.1364/BOE.385729
• Grant, S. D., Cairns, G. S., Wistuba, J., & Patton, B. R. (2019). Adapting the 3D-printed Openflexure microscope enables computational super-resolution imaging (No. 8:2003). F1000Research. https://doi.org/10.12688/f1000research.21294.1
• Hsing, P.-Y., Johns, B., & Matthes, A. (2024). Ecology and conservation researchers should adopt open source technologies. Frontiers in Conservation Science, 5. https://doi.org/10.3389/fcosc.2024.1364181
• Pearce, J. M. (2020). Economic savings for scientific free and open source technology: A review. HardwareX, 8, e00139. https://doi.org/10.1016/j.ohx.2020.e00139
• Thaler, A., Sturdivant, K., Neches, R., & Levenson, J. (2024). The OpenCTD: A low-cost, open-source CTD for collecting baseline oceanographic data in coastal waters. Oceanography. https://doi.org/10.5670/oceanog.2024.60

Useful guides

• UNESCO Open Science Toolkit guide on “Supporting open hardware for open science”: https://doi.org/10.54677/LUMO4515
• Report – Creating an Open-source Hardware Ecosystem for Research and Sustainable Development: https://doi.org/10.5281/zenodo.8301858
• Report – Supporting Open Science Hardware in Academia: Policy Recommendations for Science Funders and University Managers: https://doi.org/10.5281/zenodo.8030028
• Open Know-How is a specification for including detailed metadata with your open source hardware project so that its designs are more machine readable, interoperable, and reproducible: https://www.internetofproduction.org/openknowhow
• DIN SPEC 3105 is a specification for good practices in publishing and peer reviewing open source hardware designs: https://www.beuth.de/en/technical-rule/din-spec-3105-1/324805763

Relevant organisations

• Gathering for Open Science Hardware (GOSH): https://openhardware.science/
• Open Source Hardware Association: https://www.oshwa.org/
• Open Science Hardware Foundation: https://opensciencehardware.org/
• Internet of Production Alliance: https://www.internetofproduction.org/
• Open Hardware Makers provide mentoring and training on how to develop and support open source hardware: https://openhardware.space/
• IO Rodeo sells open source hardware for scientific research, including the OpenFlexure microscope: https://www.iorodeo.com/

#talks #opensource #openresearch

Unless otherwise stated, all original content in this post is shared under the Creative Commons Attribution-ShareAlike 4.0 International license

The workflow is organised below into three stages: Pre research; during research; and post research. Within each one I'll write down what would be good to happen at that stage.

Pre research

Start with a research “data” management plan. I'm using the term data very broadly here to mean the artefacts that result from a research projects, which could be (but not limited to) general notes, numerical data, interview transcripts, audio/video recordings, artwork, lab notebooks, etc.

When writing the plan, think about:

• What artefacts do you anticipate from the research? Which will be shared and preserved? Remember things may be be produced throughout, not just at the end.
• How will artefacts be shared and preserved? Any anticipated barriers? How might they be overcome? How could this be done in a way that ensures, as much as possible, that they can be human and machine readable years later?
• Where will they be preserved? Make sure the appropriate digital repositories are in place.
• When do you expect each output to be produced? Will the “how” be ready at those times? Closely related is for how long (what timescales) do you hope for they to be preserved? 10 years? 20 years? 100 years?!
• Who will take on the responsibility of carrying out this plan?

From experience, I know that a big challenge is not just coming up with such a plan, but to budget the time, resources, and labour to implement it. In academic research, I think this is an underappreciated point. At least from my scientific background, there are many scientists who scramble to prepare and publish data (usually because an academic journal requires them to publish data) at the last minute, and end up doing a poor job at digital preservation.

During research

During the course of a research project, remember to do good documentation. In my view, it is especially important to write down things like spontaneous learnings (“what are we learning along the way?”) or to note deviations from the research plan.

Documentation could also be informal, like rehearsal notes for performing arts or daily lab notebooks for an experimental scientist. Blog posts are also good.

Regularly check in with the original data management plan to see if it is being followed or if changes are needed.

Post research

In my view, a post-mortem is a critical exercise in any research project. This is true, too, for reflecting on how well a project's digital preservation plan/data management plan worked. Some questions to ask:

• Did we produce the digital artefacts we anticipated at the beginning?
• What was the experience of sharing and preserving those artefacts? Any points of friction?
• What would we do differently next time?
• How will we preserve and shared what we learned from this post-mortem to inform future efforts?

Another meta issue I see in academic research is the lack of appreciation, and highlighting of, the reuse of digitally preserved material. At least from what I've seen, there's lots of talk in #openresearch circles about sharing and how to do it well, but far less on using what others have shared!

I think if we do a good job of telling stories about the use of shared stuff, then we can more effectively make a case for digitally preserving said stuff and reducing #intellectualpoverty.

Unless otherwise stated, all original content in this post is shared under the Creative Commons Attribution-ShareAlike 4.0 International license

Visualisations

My original prompt was:

Visual accessibility question: Is converting color to greyscale (or black and white) an adequate test of color blind accessibility (e.g. if I convert a data visualization with color to greyscale and check if I can still understand it)? If no, what's a good test or rule of thumb?

Short answer: No.

Here are the useful responses I got for which I'm grateful (bold emphasis mine):

• Liz Hare: Good question! I don't think that would work because of the way colors are perceived. There are a few different approaches you could take. You could use some secondary code like texture or text labels. Also, it depends on how you are working. I know there are colorblind-friendly color palette packages for R. And don't forget the alt text.
• Alycia Crall: I’ve always found this testing tool very helpful: https://webaim.org/resources/contrastchecker/
• Hao Ye: Why this doesn't work:
  • converting color to grayscale is dimensional reduction (3 color axes –> 1 axis of brightness)
  • the conversion method is probably based on the perceptual attributes of an average human with 3 cone types
  • someone who deviates from that, e.g. by not having a particular cone, will perceive relative brightness differently than what the grayscale conversion produces
• DavidPS: If you open the image with Firefox, and right-click on it you will see a “inspect accessibility properties” button. Clicking on it you will see a simulate button. There you can try many different types of colour accessibility issues.
• Anne Lee Steele: @DavidPS – I've previously used this extension in another context: https://addons.mozilla.org/en-GB/firefox/addon/let-s-get-color-blind/, I didn't realise that this is now built in to the browser, amazing!
• Shern Tee: Given that a “grayscale-legible” chart is not necessarily color-accessible — what about the reverse? That is, do schemes that account for different colour perceptions also tend to make charts more grayscale-legible? Or is that not generally ensured? I ask because I don't have different colour perception (as far as I know!), but I do frequently print papers in grayscale. I'm guilty of assuming that grayscale legibility would equal colour accessibility. I've often encouraged my students to consider being as thoughtful as possible — not just using colour palettes but line-dashing, symbol shapes, and explicit labels to clarify information — but I wonder now if there's no overlap, or some overlap!
  • Hao Ye: @Shern Tee – I think so, based on arguments that a set of colors that are distinct under different color perception modes, would probably have to rely on brightness that is agnostic to any specific color channels, and thus render as distinct when converted to grayscale. I would probably have to do some linear algebra to check for sure!

Fonts

Original prompt:

As a follow up, over the years I've noted some open source fonts designed for accessibility: * Atkinson Hyperlegible by the Braille Institute (source code): https://brailleinstitute.org/freefont (expanded forks here and here) * Inclusive Sans (source code): https://www.oliviaking.com/inclusive-sans (now here: https://www.oliviaking.com/inclusivesans/feature) * OpenDyslexic: https://github.com/antijingoist/opendyslexic My question is: While I like the idea of accessible fonts (e.g. I like good distinction between 0,o,O or 1,i,l), I don't know how to critically evaluate them. What should one consider when choosing a font for visual accessibility?

Liam McGee gave a useful response from the perpsective of dyslexic accessibility. The short version is that ostensibly dyslexic accessible fonts might not be that useful after all.

With regards to dyslexia (according to Liam):

• They don't have much evidential backup... [see] https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5629233/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5934461/
• In general, using particular fonts are shown to help particular people with dyslexia (sans-serif is better for some, some like comic sans, distressingly) but I have yet to see evidence for a single font being generally helpful.
• https://www.linkedin.com/pulse/dyslexic-myths-presented-truths-gareth-ford-williams/ is worth reading on the subject.
• also https://www.linkedin.com/posts/christophestrobbe_at-the-bbc-20-fonts-were-tested-for-readability-activity-7001490480043540480-7idF/
• And... https://dyslexiaida.org/do-special-fonts-help-people-with-dyslexia/
• https://link.springer.com/article/10.1007/s11881-018-0164-z
• And, more nuanced: https://onlinelibrary.wiley.com/doi/10.1002/dys.1527

More generally:

• ...distinguishability is important, as is kerning.
• A guide to understanding what makes a typeface accessible – And how to make more informed design decisions: https://medium.com/the-readability-group/a-guide-to-understanding-what-makes-a-typeface-accessible-and-how-to-make-informed-decisions-9e5c0b9040a0
• Don't overlook more general typography such as leading and margins. https://en.wikipedia.org/wiki/The_Elements_of_Typographic_Style is an excellent reference for this.
  • Which informed a thesis style: https://bitbucket.org/amiede/classicthesis/wiki/Home

Liam also insightfully noted that “Accessibility is just aesthetics with a more sensitive gauge… where the consequence of a lack of clarity, harmony and structure is greater to some people than to others... But good typography and layout is definitely an accessibility aid.” Great point!

Liam also mentioned the 2:3 aspect ratio which is “12mm off the side of A4 (so 198x297)”, where “2:3... cut in half, it's 3:4... cut in half, 2:3. Like a musical harmonic.”

Other than the above, I note that SIL publishes various open source fonts, including Charis SIL (“optimized for readability”) and Andika (for the needs of “beginning readers”), both with wide character coverage for various languages. What's cool is that SIL hosts a TypeTuner which allows you to customise font features (e.g. whether to have slashes through 0s and 7s) and download their fonts with those features enabled.

Also, Atkinson Hyperlegible had a new release in early 2025 called Next (wider character coverage) and Mono (official monospace version!): https://www.brailleinstitute.org/freefont/

Alt-text

Great guide on how to compose alt-text for images: https://www.perkins.org/resource/how-write-alt-text-and-image-descriptions-visually-impaired/

Which has a great visual example:

Unless otherwise stated, all original content in this post is shared under the Creative Commons Attribution-ShareAlike 4.0 International license

Context

We've been doing many interviews over the past 1.5 years with different stakeholders in academia, and one of the most common barriers to changing behavior (such as doing more open research or changing research culture) is that “no one else is doing it and it doesn't benefit me”, but actually if everyone does it, then everyone benefits. Is solving such collective action problems something that has been studied in the academic context? If so, where and by whom?

I posed this question to the Turing Way and NASA TOPS Slack groups. Here's my attempt at collecting the responses so far.

Turing Way

So far, I haven't heard from someone who knows of research specifically about collective action problems in academia. But, a few theoretical frameworks were suggested as ways to examine the problem.

Agent-based modelling of individual vs collective behaviour

(from Shern Tee)

• You may find the Stanford Encyclopaedia of Philosophy entry useful: “Agent-Based Modeling in the Philosophy of Science” https://plato.stanford.edu/entries/agent-modeling-philscience/#TheoDiveInceStruScie_1
  • Unfortunately it doesn't directly answer the question of collective action problems. But (because I am a straitjacketed physicist) I find myself thinking about these situations as agent-based: a model simulation that shows agents doing things that are individually rational, but as a whole cause problems for science, is a demonstration of one possible model of collective action failure.
• This paper is a more concise overview of the above link: https://compass.onlinelibrary.wiley.com/doi/10.1111/phc3.12855
• An agent-based model of peer review, studying how scientists might want to trade-off work publishing papers with work reviewing papers: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6096663/
• This PhD thesis describes agent-based modelling of the academic publishing system — actors being journals and scientists: https://nova.newcastle.edu.au/vital/access/services/Download/uon:31339/ATTACHMENT01

This is interesting to me in the sense that I first of agent-based modelling in my intro ecology course during undergrad, but haven't considered it in the context of collective human behaviour.

Organisational theory

(from Liam McGee)

• Off the top of my head, and this might be a bit too general, but I quite like https://uk.sagepub.com/en-gb/eur/a-very-short-fairly-interesting-and-reasonably-cheap-book-about-studying-organizations/book276268 as a rapid intro to the competing models in organisational theory. Lots of references to explore. There's a book from the same series on “Studying Leadership”, which may also be of interest in approaching this problem.

Economic theories

(from Liam McGee)

• Other route might be down alternate economic theories (ones that seek to resolve the tragedy of the commons, like doughnut economics, or Ole Bjerg's stuff). Or Kahneman and Tversky from a Experimental Psych/CogSci perspective.

Religions

(from Liam McGee)

• Another interesting direction is to understand how the pro-social behaviour of various world religions works — a practical use case on that here: https://www.linkedin.com/pulse/church-cheese-frog-hugh-mason/ — comments on that article worth a dig too.

The Collective Action in Science Committee

(Julien Colomb) You may ask the people behind. https://freeourknowledge.org/committee/

Note:

I see that it proposes the model of “We will all do X (the ‘action’) when Y people have pledged (the ‘threshold’)”. This reminds me of the National Popular Vote Interstate Compact in the United States: https://en.wikipedia.org/wiki/National_Popular_Vote_Interstate_Compact

Thinking about common pool resources

(from Jonah Duckles)

A bit of a different tack on collective action, but still VERY related is Elanor Ostrom's work on Common Pool Resources, detailed in her book “Governing the Commons”. If you think about the work of an academic as working to advocate for and gather common pool resources (grant money) for themselves, I think it is an informative model for imagining a way that grant money could be considered less “contested” and more of a common pool of resources. The open science movement does kind of implicitly treat information as a common pool resource. Ostrom's work, I think, helps think about ways to build structures and systems around governing it for the benefit of many. A summary of “Governing the Commons” is her 8-point “Design principles illustrated by long-enduring Common Pool Resource (CPR) institutions” which is under the Research header on the Wikipedia page about her.

Thinking about “doers” and “thinkers”

(from Anne Lee Steele)

I think there are a few ways to approach this question: as sometimes the people doing the collection action & organising may not necessarily being the ones studying it, and vice versa. (Similarly for example: the work of community management is different from the act of studying communities!)

Regarding broader theories and ideas of social change at the individual level, the trans-theotical model (coming from medicine) is a very popular one: https://sphweb.bumc.bu.edu/otlt/MPH-Modules/SB/BehavioralChangeTheories/BehavioralChangeTheories6.html

There's also the studies of 'innovation diffusion' that talks about how systems change more broadly, studied by quite a few folks: https://en.wikipedia.org/wiki/Diffusion_of_innovations#Process

Regarding the tension between 'doers' and 'thinkers' (which of course is not necessarily cut and dry), it might be helpful to think through a few examples:

Organisers of collective action (for example – there are so many!): * https://movementecology.org.uk/ * https://scienceforthepeople.org/ * https://techworkerscoalition.org/

Studies of collective action:

• The Logic of Collective Action: Public Goods and the Theory of Groups by Mancur Olson is one I've heard cited quite a bit
• Elinor Ostrom (as @Jonah Duckles mentioned!) also wrote about collective action theory: https://academic.oup.com/edited-volume/28345/chapter-abstract/215160451?redirectedFrom=fulltext
• Institutional ethnography has been used to understand the roles, rituals and practices of all sorts of different environments, including academic spaces: https://blogs.lse.ac.uk/highereducation/2023/11/17/are-we-proper-institutional-ethnographers/
• More broadly, I've also seen how some studies of neoliberalism in academic institutions affect collectivising practices – stumbled upon this interesting piece: https://discovery.dundee.ac.uk/en/publications/revealing-the-manifestations-of-neoliberalism-in-academia-academi

Hope this helps!

Note:

Interestingly, the innovation diffusion model by Rogers is cited in the Center for Open Science's theory for behaviour change: https://www.annualreviews.org/content/journals/10.1146/annurev-psych-020821-114157#f3

NASA TOPS

Similar to the Turing Way responses, nothing specific to academia here. But there's a very interesting one about learning from climate action suggested by Jamaica Jones:

This is such an interesting question! I am not sure if it's exactly what you are looking for, but you might find Sheila Jasanoff's work to be informative. She contributed a chapter to a book called Human Choice and Climate Change that may be relevant. I also found another climate change-focused citation that seems similarly aligned: the article is called “Doing What Others Do: Norms, Science, and Collective Action on Global Warming”, by Bolsen et al.

Here's the Bolsen et al. paper: https://web.archive.org/web/20240522100857/http://eprints.lse.ac.uk/64670/1/Leeper_Doing what others do_2016.pdf

For the book Human Choice and Climate Change, it's available to borrow online from the Internet Archive:

https://archive.org/details/humanchoiceclima0001unse

It reminds me of my past life studying environmental sciences and learning about the concept of collective action problems and the tragedy of the commons. I wonder if anyone's done research on how to take lessons solving collective action problems in one domain (e.g. climate action) and applying them to another (e.g. academia)?

#metaresearch #ideas

Unless otherwise stated, all original content in this post is shared under the Creative Commons Attribution-ShareAlike 4.0 International license

The slides are published on Zenodo with DOI: 10.5281/zenodo.11051128

I also tweaked this talk linking it to reproducibility in science at the Reproducibility by Design symposium on 26 June 2024 (at the life sciences department at the University of Bristol), kindly organised by Nick and Fiona.

I will try to gather:

• general notes;
• other resources/further reading collected when developing the talk; and
• a transcript of the talk (with reproducibility addendum).

I'll try to clean up this post with more context and details on a best-effort basis.

There is a video recording (of the April 2024 version) which is saved in a Zenodo item and viewable on the Internet Archive. The video is also embedded here (click the “CC” icon for subtitles):

Further reading

The talk cites various people and resources:

• Open Source Initiative's community process for defining open source “AI”
  • https://opensource.org/deepdive
• The Turing Way community
  • https://book.the-turing-way.org/
• Infamous paper with figure of lab rat with giant genitals (later retracted) (full citation below)
  • PDF: https://web.archive.org/web/20240324051904/https://cdn.arstechnica.net/wp-content/uploads/2024/02/fcell-11-1339390-1.pdf
  • https://arstechnica.com/science/2024/02/scientists-aghast-at-bizarre-ai-rat-with-huge-genitals-in-peer-reviewed-article/
• Kate Crawford on “Artificial intelligence is neither artificial nor intelligent”
  • https://link.springer.com/article/10.1007/s43681-021-00115-7
  • https://www.technologyreview.com/2021/04/23/1023549/kate-crawford-atlas-of-ai-review/
  • https://www.theguardian.com/technology/2021/jun/06/microsofts-kate-crawford-ai-is-neither-artificial-nor-intelligent
  • https://nicospage.eu/unethical-academics-ai-and-peer-review
  • https://www.technologyreview.com/2021/04/23/1023549/kate-crawford-atlas-of-ai-review/
• “Invisible” Kenyan sweatshop workers keeping Meta and OpenAI's tools running
  • https://time.com/6247678/openai-chatgpt-kenya-workers/
  • who have now unionised: https://time.com/6275995/chatgpt-facebook-african-workers-union/
• Lilly Irani on “AI” displacing instead of replacing labour
  • https://www.publicbooks.org/justice-for-data-janitors/
  • https: