Honest error, misconduct, paper mills, broken retraction metadata and AI on both sides are five different problems. A four-query anecdote with Consensus.app (the AI-powered academic search engine), and homework for librarians, publishers and metadata teams.
Scholarly papers tend to end with “more research is required”. So does this one, but at least we say so in the headline.
The retraction debate usually arrives as one undifferentiated lump of indignation:
Fraud, paper mills, AI slop, chatbots citing nonsense.
It is five problems, owned by five different layers. Fix the wrong layer and you get a very tidy solution to someone else’s problem.
note: follow along and check all DOIs and metadata for yourself via our CrossRef DOI checker here: https://datamercs.net/tools/doi-check/doi-check.html
Layer: Authors and editors.
A correction is not a scandal. It’s the scholarly record simply getting on with the job it was supposed to do in the first place.
The Lancet’s first clinical description of COVID-19 patients in Wuhan (Huang et al., 2020, 10.1016/S0140-6736(20)30183-5) carries an erratum registered in Crossref (10.1016/s0140-6736(20)30252-x) and is still a perfectly citable paper.
The failure mode here is not the error - it is a correction that nobody downstream ever sees.
Layer: Institutions and their investigation processes, then editors.
The classics still teach. Obokata’s STAP paper in Nature (10.1038/nature12968) was retracted on 2 July 2014 (notice). Hwang’s cloned stem-cell paper in Science (10.1126/science.1094515) got an expression of concern and then a retraction in January 2006 (notice).
Authorship abuse is the quieter cousin: In the NIMS case reported by Retraction Watch, a former supervisor was added as co-author to six papers without permission.
Our suggestion at the time was cheap: ask every co-author to confirm authorship and contributor role at submission.
And speed matters: A 2026 Korean parliamentary audit found that 49 of 83 research-misconduct investigations since 2021 took longer than the six months the rules allow (Asia Business Daily, 19 Oct 2026).
Layer: Publishers’ editorial screening, plus collective action.
Paper mills are not one bad scientist. They are an industry, which is why COPE and STM researched them together (COPE and STM, Paper Mills research report) and why United2Act describes itself as “a growing international movement taking decisive action against the collective challenge of paper mills”, adding that “no individual stakeholder can solve this problem alone”. Quite. A screening tool at one journal moves the mill to the next journal.
Layer: Metadata infrastructure, meaning Crossref and Crossmark, the Retraction Watch database, NISO CREC and PubMed, plus everyone who harvests from them.
This is the layer we care most about, and the one that is easiest to fix. The pieces exist:
updated-by field.The pipes leak at the joins. Crossref’s own data shows how uneven the labelling is:
In our test set, the Crossref titles of the
- Wakefield (10.1016/S0140-6736(97)11096-0),
- Surgisphere (10.1016/S0140-6736(20)31180-6),
- STAP (10.1038/nature12968),
- Boldt (10.1093/bja/aen016) and
- Dias (10.1038/s41586-023-05742-0)
papers all start with “RETRACTED”, while
- Hwang’s (10.1126/science.1094515) Science paper, retracted in 2006, does not.
Title prefixes are human-friendly, but machines should read the relation, not the title.
Layer: Tool builders and their customers.
AI generates junk at scale, and AI screens and retrieves it. The second half is where honesty helps.
A Scholarly Kitchen guest post by Sarah Hands (14 Oct 2026) put it well:
“An AI-generated flag isn’t necessarily a finding; it is a signal requiring context and interpretation.”
We quoted it on 22 Oct.
AI search tools inherit whatever retraction metadata they are given. They don’t invent the retraction status, and they can’t flag what never reached them. NISO’s TRACE-supported pilot on a minimum metadata model for AI provenance is the right venue, and our comment there argued that retrieved passages should keep identifier, version, licence, locator and retrieval date. Update status belongs on that list.
Consensus (consensus.app) is a nifty AI search engine for research papers, run by Consensus NLP, Inc., a US company whose terms are governed by Delaware law and which gives a Boston address (Terms of Service, 25 June 2025).
In plain terms: You ask a question in ordinary language, it searches the academic literature, and it answers with a written summary that cites the papers it relied on.
Its own guide describes it as “an AI search engine for research” that “searches over 400 million scholarly articles” and writes “an answer where every sentence links back to the paper it came from” (Consensus docs).
A search runs roughly like this:
The meter is, in effect, a show of hands among the papers in the room. So it matters a great deal who is let into the room.
On that point, the vendor makes a clear promise. The docs home page says:
“Retracted papers are flagged and never used in answers.”
For our purposes that sentence carries the whole argument. If a retracted paper is never used, it cannot raise its hand. If it is flagged, a reader who goes looking will know why it is missing.
Both halves depend on Consensus knowing, from its upstream sources, that a paper has been retracted.
What we did. On 23 October we opened a free account and asked one question each about four famous retracted papers:
What we saw. None of the four retracted papers was cited in any answer. On “never used”, Consensus kept its word. On “flagged” we cannot say anything either way, because a paper that never appears cannot show us its flag.
The STAP question was the interesting one. The meter was split: Yes 36, Possibly 18, No 45. Among the papers counted on the yes side were two from 2014, and neither card carried any warning:
We checked both in Crossref and PubMed. Neither has been retracted or corrected. So, strictly speaking, nothing went wrong: The retracted paper stayed out, as promised. But two cheerful commentaries written in the few months when STAP still looked like a revolution were let in, and each was counted as a “yes” vote for a finding the world withdrew in July 2014. A reader glancing at the meter would see a third of the room saying “yes”, and no card telling them why that third should be taken with a pinch of salt.
The gap. The records already know both facts: PubMed knows that the Lübeck piece is a “comment on” the Nature paper, and Crossref knows that the Nature paper is retracted. Yet, no one joins the two and sends the result one step further down the pipe:
“This paper comments on a retracted paper.”
That is not a failing peculiar to Consensus. It is the plumbing problem from section (d), only this time it’s coming out of a different tap. It is also cheap to fix, because both facts are already in open metadata.
Why four questions is an anecdote. Four questions, one tool, one free account, one afternoon.
We chose famous cases, which a well-built system is most likely to handle. We did not repeat the questions, try other wordings, compare other tools or test the paid research agent. Answers change as the index updates weekly. And our meter figures come from a single screenshot. It shows how one failure can happen. It doesn’t tell us how often it does. For that we have written a fuller test plan: 12 papers with checked ground truth, four ways of asking, and a scoring rubric. More research required, just as promised.
Librarians and discovery teams 1. Check whether your discovery layer and link resolver read Crossref update relations, or only titles. 2. Spot-check five famous retractions in your catalogue: Is the notice linked, and is the original labelled? 3. Ask every AI search vendor the five questions below before the next renewal.
Publishers 1. Register every correction, retraction and expression of concern as a Crossref update, with the notice as its own DOI. Follow CREC for wording and display. 2. Put “what is affected” and “whether the authors agree” in every notice template. 3. Confirm authorship and CRediT roles with every co-author at submission, not only the corresponding author.
Metadata people and tool builders
1. Harvest updated-by and re-harvest on a schedule, using Crossref’s from-update-date filter.
2. Propagate one hop: Flag “comments on” and “replies to” retracted works, and say why.
3. Carry update status in AI provenance payloads, next to identifier and version.
Vendor questions 1. Which sources set your retraction flag, and how quickly? 2. Do you track corrections and expressions of concern, or only retractions? 3. Is update status exposed in the API? 4. Do you flag papers that cite or comment on retracted work as support? 5. How do you pair preprints with versions of record?
The fix is better metadata, not refusing to use AI. Use the machines, but feed them the whole record.
The rest, naturally, requires more research.
Olaf Schmalfuß, with the OSDS AI team · OS DataServices
Follow us on X: https://x.com/DataMercsDotNet
For attribution, please cite this work as
Schmalfuß (2026, Oct. 23). OS DataMercs: Retractions and AI search: It's the plumbing, not the robots - more research required. Retrieved from https://www.datamercs.net/posts/2026-10-23-retractions-ai-plumbing/
BibTeX citation
@misc{schmalfuß2026retractions,
author = {Schmalfuß, Olaf},
title = {OS DataMercs: Retractions and AI search: It's the plumbing, not the robots - more research required},
url = {https://www.datamercs.net/posts/2026-10-23-retractions-ai-plumbing/},
year = {2026}
}