Are AI Detectors Accurate? What the Research Shows

A Monday morning

Imagine Meera, an English teacher (she is an example, not a real case). She runs the class essays through an AI detector. One comes back flagged as likely AI-written. It belongs to Arjun, a quiet student who moved from Hindi-medium schooling two years ago.

Meera has two choices. She can treat the score as the answer, or she can treat it as a question. To choose well, she needs to know how often detectors are right. Here is what the research says.

Think of it like a smoke alarm

A smoke alarm is useful. But when it goes off, you check the kitchen before you call the fire brigade, because burnt toast sets it off too. A detector score works the same way. It tells you where to look, not what happened.

A detector score works the same way. It tells you where to look, not what happened.

What the studies found

OpenAI gave up on its own detector. In 2023 OpenAI released a classifier that, on its own test set of English texts, caught 26% of AI-written text and wrongly flagged human writing 9% of the time. On 20 July 2023 OpenAI withdrew it “due to its low rate of accuracy” (OpenAI).

A Stanford study raised the alarm for students like Arjun. Researchers ran 91 essays written by non-native English speakers for the TOEFL exam (from 2020 or earlier, before ChatGPT) through seven detectors. On average the detectors labelled 61.22% of these human essays as AI-written. All seven agreed on 18 of them, and at least one flagged 89. The authors call it a small pilot, and most of the detectors were built on the older GPT-2 model. Turnitin was not one of them (Liang et al.; Stanford HAI).

The biggest test found the tools unreliable, with one bright spot. A team of researchers tested 14 tools, including Turnitin, and concluded the tools are “neither accurate nor reliable”. Their main error was missing AI text, not accusing humans. Turnitin did best of all 14 and made no false accusations, but it still caught only 2 of 9 AI texts that had been run through a paraphrasing tool. False accusations ranged from 0% (Turnitin) to 50% (GPTZero) (Weber-Wulff et al.).

A 2024 study tried simple tricks. Researchers tested several detectors, including Turnitin, on 114 texts. On unedited AI text the detectors were right about 40% of the time on average (39.5% as reported). After simple changes, such as adding spelling mistakes, that fell to about 22%. The detectors also falsely flagged human writing in 9 tests, though the study had only 10 human texts. The authors say the tools cannot currently be recommended for deciding misconduct (Perkins et al.).

What Turnitin says

Turnitin reports a false positive rate below 1% for documents with 20% or more AI writing, and about 4% for individual highlighted sentences, which is per sentence, not per student. It tells teachers to use results to “initiate a conversation, not to draw a conclusion” (Turnitin). It shows scores between 0% and 20% as an asterisk, because false positives are more common there (Turnitin iThenticate). Its own 2023 analysis found no significant bias against English language learners for documents of 300 words or more, though this is the vendor testing its own tool (Turnitin).

Why small error rates still matter

Vanderbilt University did the maths. If Turnitin’s launch claim of a 1% false positive rate held, its 75,000 papers from 2022 would have meant around 750 students wrongly flagged. Citing this and other concerns, including privacy and bias, it switched the detector off in August 2023 (Vanderbilt).

Back to Meera

Meera treats the flag as a question. She:

1. Checks what the task said about AI use.
2. Opens Arjun’s drafts and version history, knowing they can help but are not proof, since drafts can be generated too.
3. Asks him, kindly and in private, to walk her through one paragraph, and offers to do it in writing if that is easier.
4. Follows her school’s process if a real concern remains.

If Arjun wrote it, she has avoided a false accusation that could have damaged his trust in her. If he did not, she now has evidence a detector could never give her.

For the full approach, see what to do instead of an AI detector, process-based assessment and a copyable AI-use policy. This is guidance, not legal advice.

What we still do not know

• How today’s tools perform. These studies tested 2023 and 2024 versions. I searched for newer independent tests and could not verify one.
• How far the numbers travel. The samples are small. Treat the percentages as warnings, not as rates for your class.
• Mixed work. Most real student work is part human, part help, and the studies only partly cover that.

Questions teachers ask

Are AI detectors accurate? It depends on the tool and the text. Weber-Wulff et al. (2023) found 14 tools “neither accurate nor reliable” overall, while Turnitin did best with no false accusations. Perkins et al. (2024) found about 40% accuracy on unedited AI text.

Does Turnitin’s AI detection give false positives? Turnitin reports under 1% at document level for documents with 20% or more AI writing and about 4% per highlighted sentence. Weber-Wulff et al. found none on their human samples.

Are AI detectors biased against non-native English speakers? A 2023 Stanford pilot found seven detectors flagged 61.22% of TOEFL essays on average. Turnitin, which was not tested there, reports no significant bias for longer documents. The question is open, so a score alone should not decide any student’s case.

What should I do if a student says they did not use AI? Listen, look at their drafts, ask them to explain part of the work, and follow your school’s process.

Where Edudojo fits

Edudojo is an AI coach that asks students questions instead of writing for them. It saves drafts and the student’s coach conversation inside an assignment and shows them to the teacher as a read-only Replay, with no score. It does not prove who wrote the work, it is not a detector, and it cannot see AI used in other tabs.

Sources (retrieved 2026-10-20)

• OpenAI, New AI classifier for indicating AI-written text (update dated 20 July 2023).
• Liang, Yuksekgonul, Mao, Wu and Zou, GPT detectors are biased against non-native English writers (2023), and the Stanford HAI summary (15 May 2023).
• Weber-Wulff et al., Testing of Detection Tools for AI-Generated Text (2023).
• Perkins et al., GenAI Detection Tools, Adversarial Techniques and Implications for Inclusivity in Higher Education (arXiv, 28 March 2024).
• Turnitin, Understanding the false positive rate for sentences of our AI writing detection capability (14 June 2023).
• Turnitin iThenticate, AI writing detection in the new enhanced Similarity Report view.
• Turnitin, New research: Turnitin’s AI detector shows no statistically significant bias against English language learners (26 October 2023).
• Vanderbilt University, Guidance on AI detection and why we’re disabling Turnitin’s AI detector (16 August 2023).