Picture this: you ask an AI tool to help with your literature review, and it hands you a perfectly formatted reference, correct author names, a real-sounding journal, a plausible year. You paste it straight into your bibliography. There’s just one problem. That paper doesn’t exist.

This isn’t a rare glitch anymore. It’s become common enough that major journals, publishers, and research-integrity teams are now treating it as one of the biggest quiet risks in academic writing today. If you use AI for research, essays, or a thesis, this is worth five minutes of your time.

What is a hallucinated citation?

A hallucinated citation is a reference that an AI tool generates that looks completely real but doesn’t actually exist, or that misattributes real findings to the wrong paper. Researchers studying this problem call the worst examples “Frankenstein” citations, because they stitch together fragments of genuine papers (a real author, a real-sounding title, a real journal name) into something that was never actually published.

The dangerous part is that these fake references rarely look fake. They’re usually formatted correctly, attributed to real researchers, and dated plausibly. Unless you actually go and check, there’s often no obvious red flag.

How big is this problem, really?

Bigger than most people realize, and it’s growing fast. A Nature news feature published in April 2026 reported that tens of thousands of papers published in 2025 may contain invalid, AI-generated references.

A separate analysis is even more specific about the scale. Researchers led by Maxim Topaz at Columbia University audited nearly 2.5 million PubMed-indexed papers and published their findings as a letter to The Lancet in May 2026, reported in detail by Retraction Watch. They found that about 1 in every 277 papers published in the first seven weeks of 2026 referenced a paper that doesn’t exist. That’s a sharp jump from 1 in 458 in 2025, and 1 in 2,828 back in 2023, a roughly 12-fold increase in fabricated citations in just two years. The researchers traced the sharpest rise to mid-2024, right around when AI writing tools became widely used.

One more detail worth knowing if you’re writing any kind of review paper: the study found review articles had a fabrication rate 57% higher than other paper types, likely because they cite so many sources at once.

Why do AI tools make up references in the first place?

General-purpose AI chatbots like ChatGPT, Gemini, or Claude are built to predict the next most plausible piece of text, not to look things up in a verified database by default. When you ask one to “give me three sources on X,” it generates something that fits the pattern of a real citation, without necessarily checking whether that exact paper exists. It’s the same underlying issue behind AI giving wrong factual answers generally, which we cover in more depth in our guide to why AI sometimes gives wrong answers.

Researcher Maxim Topaz, who led the Lancet analysis, made an important point in his interview with Retraction Watch: most of the cases his team found weren’t researchers deliberately faking sources. 91% of the flagged papers had only one or two fabricated references, which he said are “likely honest mistakes by authors who used AI tools without verifying the output.” In other words, this usually isn’t dishonesty. It’s trust placed in a tool that was never designed to guarantee factual citations.

How to check every AI-generated citation

The good news is that verifying a citation only takes a minute or two once it’s a habit. Here’s a simple process:

• Search the exact paper title in quotation marks on Google Scholar or PubMed. If nothing comes up, that’s your first warning sign.
• Check for a DOI, and paste it into Crossref’s search tool to confirm it resolves to a real, matching paper.
• Open the actual source. Don’t just trust that the AI’s summary of a paper matches what the paper really says, skim the abstract yourself.
• Be extra careful with review articles and papers that cite many sources at once, since that’s exactly where this analysis found the highest fabrication rate.

Quick tip: if an AI tool gives you a citation you can’t verify within two minutes of searching, treat it as fake until proven otherwise, not the other way around.

From my own experience working on websites and digital tools, this is really the same instinct as checking a suspicious link before you click it. You don’t assume something is safe by default, you look for confirmation first. Citations deserve the same habit.

Tools that reduce this risk

Not all AI research tools carry the same risk. Some are built specifically to ground their answers in real, searchable sources rather than generating text freely. If you’re doing a literature review, our step-by-step guide to literature reviews with AI covers tools like Elicit and Semantic Scholar, which pull directly from real paper databases and show you the actual source, rather than describing one from memory. Similarly, our guide to AI tools for thesis writing and our walkthrough of summarizing research papers with AI both lean on tools that link back to the original document, so you can check the source yourself in one click.

Free citation managers like Zotero also help here, not because they use AI themselves, but because they store the actual paper alongside the reference, making it easy to double-check what you’re citing before you submit anything.

What this means if you’re writing a thesis, paper, or report

If you’re a student or researcher using AI to speed up your work, this isn’t a reason to stop. AI is genuinely useful for finding starting points, summarizing dense papers, and organizing your reading list, our beginner’s guide to AI covers the basics if you’re still getting comfortable with these tools. The real takeaway is simpler: treat every AI-generated citation as a draft that needs verifying, not a finished fact. That one habit is the difference between using AI well and ending up in a retraction story.

Common Questions

Can AI research tools like NotebookLM or Elicit still invent citations?

They’re much less likely to, because they’re designed to ground answers in the specific documents or database you give them rather than generating references from general knowledge. But no tool is risk-free, so it’s still worth spot-checking anything that goes into a formal paper.

Is using a fake AI-generated citation considered academic misconduct?

Opinions among researchers and publishers differ, and it depends on intent and how central the citation is to your argument. Most experts agree it’s treated far more seriously if you didn’t bother to check the source at all, so verifying every reference protects you either way.

How can I quickly tell if a citation is fake?

Search the exact title in quotation marks on Google Scholar or PubMed, and check the DOI on Crossref. If the paper doesn’t turn up, or the DOI doesn’t resolve to a matching title, treat it as unverified until you find it yourself.

Final takeaway

AI can genuinely speed up research, but it can also hand you a citation that looks completely real and isn’t. The fix isn’t complicated: search the title, check the DOI, and open the actual source before it goes anywhere near your bibliography. That one habit keeps AI a useful research assistant instead of a liability.