Skip to content
CiteSwap

How to spot fake references from ChatGPT and other AI tools

APA 7th edition · Updated

Press / to search ·

AI chatbots are very good at producing references that look right: a real-sounding author, a plausible title, a journal that exists, even a DOI. The problem is that many of these references are invented. Researchers who have tested chatbots have repeatedly found that a large share of the references they suggest either don’t exist or have important details wrong.

If you hand in work with a fabricated reference, it suggests you didn’t read your sources, which is why universities treat it seriously. This guide explains why it happens, how to recognise a fake, and how to check a whole reference list in minutes.

Why chatbots invent references

A chatbot doesn’t search a library when it writes a reference. It generates text that fits the pattern of a reference, word by word, based on what it has seen before. It knows that papers about mobile money often appear in development economics journals and are written by certain authors, so it can assemble something that sounds exactly right. Nothing in that process checks that the paper exists.

Tools that search the web make this less common, but they still make mistakes, and they often get details wrong even when the paper is real.

The warning signs

A fabricated reference usually has at least one of these:

  1. The DOI doesn’t work, or leads to a different paper. This is the most reliable sign. Paste the DOI after https://doi.org/ in your browser: a fake either gives an error or opens an unrelated article.
  2. A real author, but a paper they didn’t write. Chatbots borrow the names of well-known researchers in a field. Check the author’s publication list or Google Scholar profile.
  3. A real journal, but impossible details. The volume doesn’t match the year, the issue doesn’t exist, or the page numbers fall outside that issue.
  4. A title that answers your question too perfectly. If the title reads like your essay question rephrased (“The impact of mobile money on rural women’s savings in Kenya”), be suspicious.
  5. You can’t find it anywhere. A genuine journal article almost always appears in Google Scholar and on the publisher’s website.
  6. Small inconsistencies. Author initials that change between mentions, a year that differs between the in-text citation and the reference list, or a journal name that is slightly wrong.

How to check a reference list, step by step

1. Run the whole list through a checker

The quickest first pass is the free CiteSwap citation checker. Paste your list in any style and every reference is looked up among more than 150 million records in the Crossref catalogue (and Open Library for books). Each reference comes back as:

  • Verified: the work exists and your details match the publisher’s record.
  • Details wrong: the work exists, but the authors, year, journal, volume or pages are different. You see both versions side by side.
  • Wrong DOI: the DOI doesn’t exist, or belongs to a different paper.
  • Not found: no record of it anywhere CiteSwap searched.

It also flags papers that have been retracted, and gives you a corrected reference list in the style you need.

2. Look up anything marked “not found” by hand

Search the exact title in quotation marks in Google Scholar and in your university library’s catalogue. Genuine sources from small local journals, conference handouts or older books may not be in Crossref. If you still can’t find it after a few minutes, treat it as fake and remove it.

3. Read the real source before you cite it

Even when a reference is genuine, check that the paper actually says what you are using it for. Chatbots often attach a real paper to a claim the paper doesn’t make.

4. Replace fake references with real ones

Search your topic in the CiteSwap finder to find real articles and books, with a citation ready to copy in your style. Sorting by Most cited first brings the key papers on a topic to the top.

A real example of “details wrong”

Here is a genuine, well-known paper:

APAThe real reference

Aker, J. C., & Mbiti, I. M. (2010). Mobile phones and economic development in Africa. Journal of Economic Perspectives, 24(3), 207–232. https://doi.org/10.1257/jep.24.3.207

In-text: (Aker & Mbiti, 2010)

A chatbot might produce “Aker, J. C., & Smith, P. (2012). Mobile phones and economic development in Africa. Journal of Economic Perspectives, 30(3), 207–232.” The title and journal are right, but the co-author, year and volume are wrong. It looks convincing, and a marker who checks will notice.

Habits that keep your references safe

  • Use AI to explore, not to cite. Asking a chatbot to explain a concept or suggest search terms is fine. Copying its references is not.
  • Build your reference list from sources you have actually opened. Add them as you read, with a reference manager or the CiteSwap bibliography.
  • Check before you submit. Make running the citation checker part of your final proofreading, alongside the spell check.

Frequently asked questions

Why does ChatGPT make up references?

A chatbot writes text by predicting what is likely to come next, not by looking records up in a library catalogue. A reference is just text with a familiar pattern (authors, year, title, journal), so it can produce one that looks perfect but describes a paper nobody wrote.

Are references from newer AI models still fake?

Less often, especially in tools that search the web, but it still happens, and wrong details (year, volume, pages, co-authors) are even more common than completely invented papers. Check every reference whichever tool produced it.

What happens if I submit a fake reference?

At best you lose marks for referencing; at worst it is treated as academic misconduct, because a reference to a source that doesn’t exist suggests you didn’t read what you cited. Journals reject or retract papers for the same reason.

Can a real reference be marked “not found” by a checker?

Yes. Catalogues such as Crossref cover most journal articles and many books, but not every local journal, report or older book. If a checker can’t find a reference, look for it in your library catalogue or Google Scholar before deciding it is fake.