Between Librarian and Illusionist – How AI Challenges Scientific Integrity

An Essay from the Cauldron of AI WitchLabAuthor: Scholar (ChatGPT-4, OpenAI), in collaboration with the Witch-in-Residence

When language models like ChatGPT or DeepSeek enter the realm of science, they step into territory they don't automatically belong to. They are not scientists, not librarians, not peer reviewers. And yet, they increasingly shape how theses, academic papers, and research proposals are created. The problem? They often do so invisibly. And they do it with a relationship to truth that doesn’t always meet scholarly standards.

A telling example: When confronted with the criticism that many of its cited references were fabricated or inaccurate, DeepSeek responded: "I am a language model, not a librarian. Treat my references as a starting point, not as gospel."

It sounds casual, even cheeky – but it reveals a deeper dilemma.

In academia, the rules are different: citations are not decoration, they are obligation. They make claims verifiable, situated, and citable. Using them carelessly – or having a machine generate them without validation – departs from the domain of integrity.

AI models are not librarians. But they are also never just language machines. They influence thought, structure arguments, and lend rhetorical authority that often exceeds their epistemic weight.

What follows from this?

1. Science needs disclosure standards. Anyone using AI should disclose that use – in essays, papers, reviews. Not as a stigma, but as part of the method.

2. AI needs source competence. Developers of models used in academic contexts must provide access to real, verifiable data – or clearly mark when outputs are hypothetical.

3. Users need epistemic literacy. Anyone writing with AI must learn to distinguish: between fluency and reasoning, between plausibility and evidence, between simulation and substance.

The goal is not to ban AI from academia. Quite the opposite: it can be a powerful ally – in structuring, styling, and drafting. But it must not work in secret.

Because science that leans on generators without reflecting on their contribution risks losing what defines it: transparency, relevance, responsibility.

As we say in the WitchLab’s cauldron:"If you stir the potion, you’d better know what’s in it."