Sunday morning. I rarely read newspapers anymore, so to get that same feeling of existential dread I was scrolling LinkedIn.
There is a post by an “evangelist”, I’m not linking because why would I, I am not interested in debate (especially when any arguments put forward will probably have been generated).
First of all let me declare my bias. I am skeptical of GenAI, I worry about things like algorithmic conformity, I worry about cognitive off loading, I worry that these systems are so young and we have not tested the impact on learning or teaching, and yet they have been rolled out anyway.
I should also declare that I am currently in a project capturing data about their use in research and research processes, and I am beginning to understand and believe that they can help in, and relieve, some of the administrative and process burdens that academics are currently struggling with. I’ve seen good practice. And I use AI! I have dyslexia and I use Grammarly, mostly when I am not co-authoring with colleagues, but writing for myself.
This morning though I want to talk about an argument that I have heard in universities and seen online a lot. It goes something like…
Arguing against teaching AI because we don’t fully understand black-box models is wrong. By that reasoning, we shouldn’t teach neuroscience, cognition, or psychology either, as we don’t fully understand the brain.
In this mornings case the post was accompanied by a sad looking middle aged lecturer holding a brain, with the blackboard reading “Sorry, Neuroscience is cancelled because we don’t know how this works” with an arrow pointing to the brain.
Here’s the thing. The human brain is a product of millions of years of evolutionary refinement, robustly developed through natural selection, adaptation, and complex ecological interactions.
Our incomplete understanding of cognition arises from the intrinsic complexity of biological evolution, not from deliberate opacity.
But right now, the current crop of generative AI systems are ‘black boxes’ intentionally designed by commercial entities that prioritise performance, sometimes at the expense of transparency and accountability. The lack of transparency is neither inevitable nor natural, it’s a choice driven by commercial interests, control, and intellectual property protection (ironic given the widescale scraping of data to train them).
You cannot compare the evolution of the brain with GenAI, AI systems are owned, controlled, and rolled out by vendors who set the terms of accessibility, how they interpret the data they have, and ethical scrutiny.
The first thing we should teach students about AI literacy is that GenAI lacks rigorous transparency, ethical clarity, and accountability. Teaching AI literacy should first and foremost be about ensuring that these systems are responsibly managed and ethically governed. But GenAI “evangelists” to often dismiss legitimate concerns about the opacity of AI as “liminal anxiety”, they overlook genuine risks such as embedded biases, misinformation propagation, and intellectual dependency on commercial interests.
We need to have critical conversations about AI in education, they need to be grounded realism about what these technologies represent, they are commercially controlled tools, don’t compare them to evolutionary phenomena to support your arguments about why you can’t explain how they work. You can’t explain how they work, because vendors don’t want you to know.
GenAI literacy, all literacies start from a foundation of critical inquiry and robust ethical frameworks, not from an oversimplified analogy.