How Accurate Are AI Detectors? What Stanford, Chicago and Other Studies Found

Summary: AI detectors are accurate on long, unedited AI text but unreliable as proof of authorship. The best tools in a 2025 University of Chicago benchmark flagged almost no human passages, yet a Stanford study found seven detectors wrongly flagged 61% of essays by non-native English writers, and edited or paraphrased AI text slips past most tools.

Can software reliably tell whether a person or a chatbot wrote a paragraph? Teachers, editors and hiring managers face that question daily, usually right after a checker returns a score like "87% AI." So, are AI detectors accurate? The answer depends on three things: which tool is used, what kind of text it reads, and how much that text was edited.

This guide pulls together peer-reviewed and university research on detector accuracy, from OpenAI's withdrawn classifier in 2023 to a 2025 University of Chicago benchmark of commercial tools, and keeps vendor claims separate from independent results. For a wider look at how detection fits into publishing and education, see the AI Business Weekly overview of AI content detection.

💡 Not sure which AI tool is actually right for your business?

Get the free guide, Which AI Tool Should You Actually Use? It is a straightforward breakdown of the leading AI tools to help you pick the right one for your needs.

Subscribe to AI Business Weekly for the guide, plus daily coverage of AI trends, acquisitions, and product launches.

Get the Free Guide →

AI detectors are accurate only in narrow conditions: long passages of unedited text from a mainstream chatbot, checked by a top commercial tool. Outside those conditions, accuracy falls fast. Results vary widely by vendor, and even the strongest tools produce some false positives, so a detector score works as a signal for review, not as proof.

The research record shows a clear split between early tools and today's best commercial products. OpenAI's own classifier correctly labeled only 26% of AI-written text as "likely AI-written" and wrongly flagged human text 9% of the time on its English challenge set. OpenAI withdrew the tool on July 20, 2023, citing "its low rate of accuracy."

The same year, a team led by Debora Weber-Wulff tested 14 detectors, including Turnitin and GPTZero, on 54 documents. The authors wrote that the tools were "neither accurate nor reliable," with all scoring below 80% accuracy and only five above 70%. The study appeared in the International Journal for Educational Integrity in December 2023.

By 2025, the top tier looked very different. A University of Chicago Booth School of Business benchmark by Brian Jabarian and Alex Imas tested four detectors on 1,992 human passages and AI versions from GPT-4.1, Claude Opus 4, Claude Sonnet 4 and Gemini 2.0 Flash. At a 0.5 threshold, Pangram flagged 0% of human passages, Originality.ai 0.11% and GPTZero 0.71%, while an open-source RoBERTa model flagged 95% of human text as AI, according to the NBER working paper (No. 34223, September 2025). Readers checking Claude output specifically can see our guide to whether AI detectors can detect Claude.

The improvement is real, but it comes with a catch. The 2025 benchmark measured clean, unedited chatbot output, which is the easiest case a detector will ever see. Real classroom and workplace text is usually messier.

Most AI detectors are classifiers: machine learning models trained on large sets of human and AI text to recognize the statistical fingerprints of each. Early tools leaned on perplexity, a measure of how predictable each word choice is to a language model. A third method, watermarking, embeds a hidden signal when text is generated, but only works for participating models.

GPTZero, one of the best-known tools, originally defined perplexity as how likely an AI model would be to choose the exact same words, and burstiness as how much that predictability varies across a document. An update note on its own explainer post says that since autumn 2023 it no longer relies on those two measures and has moved to a deep-learning architecture.

Watermarking takes a different route. Google DeepMind open-sourced its SynthID text watermark in October 2024 alongside a paper in Nature, after testing it on around 20 million watermarked and unwatermarked Gemini responses. MIT Technology Review reported that it was less reliable when text was rewritten or translated, and that it only works on output from Google's models.

Both approaches rest on the same probability math that powers large language models themselves. That shared foundation explains the core weakness: a detector is guessing from patterns, and patterns can be shifted by editing, prompting or simply writing in a plain, predictable style. Code checkers raise similar questions, covered in the guide to the AI code detector category.

The first two signals help explain why simple, formal writing gets flagged. A writer with a limited English vocabulary tends to choose common, predictable words, which is exactly what a perplexity-based check reads as machine-like.

AI detection false positives are rare for the best tools on long, native-English text, but common for specific groups of writers. Turnitin, as a vendor claim, reports a document-level false positive rate under 1% and a sentence-level rate near 4%. Independent research found far higher rates for non-native English writers, and US survey data shows uneven flagging by race.

Turnitin's Chief Product Officer Annie Chechitelli wrote in June 2023 that the company's document false positive rate is less than 1% for documents with 20% or more AI writing, and its sentence-level rate is around 4%. The same post said 54% of falsely flagged sentences sit right next to actual AI writing. Turnitin now hides scores between 1% and 19%, showing an asterisk instead, to avoid false positives.

Independent numbers tell a harsher story for some writers. Stanford researchers led by James Zou ran 91 TOEFL essays by non-native English speakers through seven detectors. On average, the tools misclassified 61.22% of them as AI-generated; 18 of 91 were flagged by all seven, and 89 of 91 by at least one. The same detectors were near-perfect on essays by US eighth-graders.

A Common Sense Media survey released in September 2024, covering 1,045 US parents of teens aged 13 to 18, plus one of their teens, found that 20% of Black teens said their work had been incorrectly flagged as AI-generated, compared with 10% of Latino teens and 7% of white teens, Education Week reported. These are self-reported experiences, not measured detector output.

A 1% error rate sounds small until it meets volume. Vanderbilt University noted it submitted 75,000 papers to Turnitin in 2022, meaning a 1% rate could have mislabeled around 750 student papers; it disabled Turnitin's AI detector in August 2023. That arithmetic scales with adoption, which AI Business Weekly tracks in its AI in education statistics.

AI detectors work poorly once AI text is edited, paraphrased or run through a "humanizer" tool. In independent testing, accuracy dropped from roughly three in four correct on raw AI output to about one in four on machine-paraphrased text. Only a few commercial tools stayed robust against humanizers in the most recent large benchmark.

The Weber-Wulff study broke results down by document type. Detectors identified human-written text correctly 96% of the time and unedited AI text 74% of the time, but accuracy fell to 42% for manually edited AI text and 26% for machine-paraphrased text. Human text written in another language and machine-translated into English saw accuracy drop by about 20%.

The RAID benchmark, presented at ACL 2024 by Liam Dugan and seven co-authors, tested 12 detectors on more than 6 million generations across 11 models, 8 domains and 11 adversarial attacks. The authors found detectors are "easily fooled by adversarial attacks" and lose accuracy on unseen models and different sampling settings.

The 2025 Chicago benchmark also ran AI text through the humanizer StealthGPT. Pangram still caught nearly all longer passages, Originality.ai's miss rate rose to about 5% on long text and 21% on short text, and GPTZero's miss rate climbed to about 50% or higher in most genres. In the Stanford study, one extra prompt asking ChatGPT to "elevate" its writing cut detection of AI-written college admission essays from 100% to 13%.

The practical takeaway: detectors mostly catch unedited copy-and-paste output. A writer who uses ChatGPT for a first draft and then rewrites heavily will often pass, while a careful human writer with a plain style can still be flagged.

No single AI detector is most accurate in every test. In the largest recent independent benchmark, from the University of Chicago in 2025, Pangram was the only tool to meet the authors' "policy-grade" standard across all four AI models. Earlier academic studies rated Turnitin, Copyleaks and Originality.ai highly. Rankings shift as models and detectors update.

Jabarian and Imas placed Originality.ai and GPTZero in a second tier, with GPTZero showing a lower false positive rate and Originality.ai a higher overall score on their AUROC measure. They also estimated cost per correctly detected AI passage at about $0.0228 for Pangram, $0.0416 for Originality.ai and $0.0575 for GPTZero. A 2023 study by William H. Walters in Open Information Science tested 16 detectors and found Copyleaks, Turnitin and Originality.ai highly accurate across all three of its document sets.

Vendors dispute these rankings. In a January 2026 post, GPTZero said the Chicago researchers used the wrong API field and that its re-run with model version 2025-12-18-base produced 99.5% accuracy and a 0.05% false positive rate. That is a vendor claim and has not been independently confirmed.

The Chicago authors stressed that detector performance will change over time and needs routine audits. Any "most accurate" ranking is a snapshot of one dataset, one set of models and one month.

GPTZero is accurate on unedited AI text and rarely flagged human writing in the largest recent independent benchmark, but it struggles with humanized text. In the 2025 Chicago benchmark it flagged 0.71% of human passages at a 0.5 threshold, yet missed about half of StealthGPT-processed passages in most genres.

A smaller June 2025 preprint by Selin Dik, Osman Erdem and Mehmet Dik tested GPTZero on 28 AI-generated and 50 human-written essays. Most AI essays drew 91% to 100% AI probability, while the authors reported "a handful of false positives" among human essays and concluded it is less reliable at clearing human writing than at catching AI text.

ZeroGPT has weaker independent evidence behind it than the top commercial tools. In the Weber-Wulff study, it scored 59%, 67% or 72% accuracy depending on the scoring method, ranking fifth or sixth of 14 tools. It was not among the four detectors in the 2025 Chicago benchmark.

A study published in January 2026 in Acta Informatica Pragensia tested ZeroGPT and GPTZero on abstracts from seven Persian-language library science journals. Both tools caught AI-generated abstracts well but produced high false positive rates on human-written abstracts, and the authors called for detectors that work across languages and cultures.

No public tool replicates Turnitin's proprietary model, so no checker can predict exactly what a Turnitin report will show. In independent research, Copyleaks and Originality.ai landed in the same high-accuracy group as Turnitin in Walters' 2023 study, and Turnitin posted the single highest score, 81%, under one of the Weber-Wulff study's three scoring methods.

There is no single answer, because AI detection in Blackboard Learn depends on which tools each institution connects. Many schools add Turnitin, whose AI writing report reaches users through learning management system integrations. Anthology, the company behind Blackboard, also piloted GPTZero inside Blackboard's SafeAssign originality tool in 2023 with a small group of US clients.

Turnitin's Blackboard LTI 1.3 release notes state that AI writing detection became available to all non-student users who can access Similarity Reports on April 4, 2023, including through LMS integrations. GPTZero announced on June 1, 2023 that a beta let instructors at a few dozen US-based Blackboard Learn clients use its detector within SafeAssign; that is a vendor announcement, and no broader rollout was described.

For students, the reliable way to know what a course uses is the syllabus or the instructor, not a third-party "Blackboard checker" site.

AI detector results should be treated as one piece of evidence that prompts a closer look, never as the sole basis for an accusation. Even vendors build in caution: Turnitin hides low scores to limit false positives, and OpenAI withdrew its own detector. Corroborating evidence, such as drafts and version history, carries far more weight.

The institutional record backs this up. Vanderbilt concluded in August 2023 that "we do not believe that AI detection software is an effective tool that should be used," citing false accusations, bias against non-native writers and a lack of transparency about how Turnitin's detector works. Teachers choosing a tool for their classroom can see our guide to the best AI detector for teachers.

Detectors share a trait with the generative models they check: both are probabilistic systems that can be confidently wrong, a pattern documented in AI Business Weekly's AI hallucination statistics. A sensible review process looks like this:

For publishers and marketers, the question is shifting from "was AI used?" to "is this accurate and useful?" AI Business Weekly's AI content creation statistics track how widespread AI-assisted writing has become, and a detector score says nothing about quality or factual accuracy.

AI detectors range from highly accurate to barely better than chance. In a 2025 University of Chicago benchmark, the best tools flagged under 1% of human passages, while a 2023 study of 14 detectors found all scored below 80% accuracy. Editing, paraphrasing, short text and non-native English writing all lower accuracy.

ZeroGPT is moderately accurate at best in independent testing. The 2023 Weber-Wulff study measured its accuracy at 59% to 72% depending on the scoring method, ranking it fifth or sixth of 14 tools. A 2026 study on Persian-language abstracts found high false positive rates on human writing.

GPTZero is accurate on unedited AI text and flagged only 0.71% of human passages at a 0.5 threshold in the 2025 University of Chicago benchmark. Its weakness is humanized text: after processing with StealthGPT, it missed about half of AI passages in most genres tested.

Yes, AI detectors regularly flag human writing as AI-generated. A Stanford study found seven detectors misclassified 61.22% of TOEFL essays by non-native English speakers on average. Turnitin itself reports a sentence-level false positive rate of around 4%, and hides scores between 1% and 19% to limit errors.

No AI detector reproduces Turnitin's results, because its model is proprietary. In independent research, Copyleaks and Originality.ai performed in the same high-accuracy group as Turnitin in a 2023 study of 16 detectors published in Open Information Science, making them the closest comparable tools on available evidence.

Most AI detectors cannot reliably detect paraphrased AI text. In the 2023 Weber-Wulff study, accuracy fell to 26% on machine-paraphrased AI text, compared with 74% on unedited output. The 2024 RAID benchmark of 12 detectors similarly found they were easily fooled by adversarial attacks.

Are AI detectors accurate? The best commercial tools are now accurate on long, unedited chatbot text: a 2025 University of Chicago benchmark found Pangram, Originality.ai and GPTZero flagged under 1% of human passages. Accuracy drops sharply once AI text is edited, paraphrased or humanized, with one 2023 study measuring just 26% accuracy on paraphrased text.

Human writers also get flagged. Stanford research found an average 61% false positive rate on essays by non-native English speakers, and Turnitin reports about 4% of flagged sentences are human-written. OpenAI withdrew its own detector in 2023 for low accuracy. A detector score is a reason to look closer, not proof of how a text was written.

Written by the AI Business Weekly Editorial Team. Reviewed, edited and published by Sameer Khan. This article was AI-assisted, then reviewed by Sameer Khan before publishing.