Why AI Detectors Are Not Working Properly: Evidence, Technical Reasons and Real-World Cases

AI text detectors are often presented as tools that can tell whether a piece of writing was produced by a human or by a large language model. In practice, they do not work reliably enough for high-stakes decisions. The main problem is not that detectors never catch AI writing. It is that they are unstable, easy to evade, biased in some settings, and frequently wrong on ordinary human writing. Even the companies and universities closest to these tools have publicly acknowledged serious limits. OpenAI discontinued its own AI text classifier in July 2023 because of its low accuracy. In its published evaluation, the classifier correctly identified only 26% of AI-written text as “likely AI-written” while falsely labeling human writing as AI-written 9% of the time.

Core thesis

AI detectors fail in real use because they are trying to solve an unstable classification problem. Modern writing is often a mix of human drafting, AI suggestion, paraphrasing, grammar correction, translation help, and revision. That means the question “Was this written by AI?” usually has no clean yes-or-no answer. On top of that, detectors are trained or tuned on narrow datasets, then deployed on new models, new writing styles, new domains, and new languages. As soon as the data distribution changes, accuracy drops. A 2025 NAACL Findings paper found that benchmark scores can look good while real-world usefulness remains poor, and that moderate adversarial prompting can significantly evade detection.

Why AI detectors fail technically

1. They rely on statistical patterns, not proof of authorship

Most text detectors do not “know” who wrote a passage. They infer from surface statistics such as predictability, token probability, sentence regularity, or model-specific stylistic patterns. That is not authorship proof. It is a guess based on resemblance. Stanford HAI researchers explained that many detectors depend heavily on perplexity-like signals, which correlate with how predictable a text is. That creates a structural problem: writing that is simple, formulaic, or highly polished can be misread as AI-like even when it is fully human.

2. They break under distribution shift

Detectors often perform best on the exact model families, prompts, genres, or datasets they were tested on. But real deployments involve new LLMs, new prompt styles, and real student or workplace writing. The 2025 NAACL Findings paper found that evaluation on unseen models and tasks is much harder than benchmark reporting suggests, and that common metrics like AUROC can overstate practical usefulness. The paper argues that the more relevant question is performance at very low false-positive rates, because real institutions cannot tolerate many false accusations.

A separate 2024 study on short-form AI detection reached a similar conclusion: existing zero-shot detectors were inconsistent with prior benchmarks, were vulnerable to simple changes such as sampling temperature, and even specialized detectors struggled to generalize to new human-written texts.

3. Human editing destroys the signal

The moment AI text is revised by a person, paraphrased, shortened, translated, or blended with original writing, detector performance drops. Real-world writing is rarely raw model output pasted untouched. The NAACL paper explicitly tested practical adversarial prompting and found that moderate efforts can evade detection. The Stanford analysis also noted that simple prompt engineering can bypass detectors.

4. There is a built-in tradeoff between catching AI and avoiding false accusations

To catch more AI-written text, a detector usually has to become more aggressive. But aggressiveness raises false positives. To reduce false positives, vendors often accept more false negatives. Inside Higher Ed reporting noted that Turnitin may miss roughly 15% of AI-generated text in trying to avoid false positives.

Turnitin’s own guidance says there is a higher incidence of false positives between 0% and 19%, which is why lower-range percentages are now suppressed and shown with an asterisk instead.

5. Mixed authorship makes the label itself ambiguous

A student may outline manually, use AI to brainstorm, rewrite everything, then run grammar correction. Another may draft alone but use translation assistance. Another may write fully human text that resembles the statistical profile of machine output. In all of these cases, “AI-generated” is not a precise category. That ambiguity makes detector scores unsuitable as stand-alone evidence in disciplinary settings. This is one reason Vanderbilt University disabled Turnitin’s detector.

The strongest empirical evidence

OpenAI’s own detector was withdrawn

This is one of the clearest pieces of evidence. OpenAI discontinued its AI classifier on July 20, 2023 due to low accuracy. In the company’s published evaluation, it caught only 26% of AI-written texts as “likely AI-written,” while incorrectly flagging human text 9% of the time. That is far too weak for serious enforcement.

False positives are not rare, especially in some populations

A Stanford-led study found that detectors were near-perfect on essays by U.S.-born eighth graders but classified 61.22% of TOEFL essays written by non-native English students as AI-generated. It also found that 97% of those TOEFL essays were flagged by at least one detector. This is one of the most important findings in the literature because it shows not just error, but unequal error.

Universities have backed away from these tools

Vanderbilt’s official guidance explains why it disabled Turnitin’s AI detector. MIT Sloan teaching guidance likewise states that AI detectors do not work reliably enough for serious use and points to false accusations as a major concern.

Real documented cases

You asked for real cases. The examples below are documented, public cases that show how unreliable AI detection can create real harm.

1. Australian Catholic University case

ABC News reported that Australian Catholic University used AI technology to accuse about 6,000 students of academic misconduct in 2024. The report says many students had done nothing wrong. One nursing student, Madeleine, was cleared only after six months, during which her transcript showed “results withheld,” and she believed that hurt her employment chances. ABC also reported that around one-quarter of all referrals were dismissed after investigation, and that any case where Turnitin’s AI detector was the sole evidence was dismissed immediately. The same report says ACU later abandoned the Turnitin tool after finding it ineffective.

This case matters because it shows the full failure chain: unreliable software, institutional overreliance, burden shifted onto students to prove innocence, delayed resolutions, and real career consequences.

2. Vanderbilt disables Turnitin AI detection

Vanderbilt’s published explanation said Turnitin’s AI detector was enabled with little advance notice, without transparency into how it worked, and with serious reliability concerns. Vanderbilt calculated that if a 1% false-positive rate were applied to roughly 75,000 papers submitted there in 2022, around 750 papers could be incorrectly labeled. Vanderbilt disabled the feature for the foreseeable future.

3. University at Buffalo student case

Spectrum News reported that a University at Buffalo student said Turnitin flagged her work as AI-generated even though she had not used AI, causing stress and putting graduation at risk.

4. Yale lawsuit over alleged improper AI use

Yale Daily News reported that a Yale School of Management student sued Yale after being accused of using AI on a final exam and suspended. Even without deciding the merits of that specific case, it shows that AI-use accusations now have serious legal and academic consequences.

5. Massachusetts high school discipline case

Reuters reported on a Massachusetts high school student whose punishment for AI use was upheld by a federal judge. This is not purely a detector-failure case, but it shows how schools are already imposing significant consequences in disputes over alleged AI misuse.

Why these failures keep happening

Scale turns “small” error rates into large harms

Even when vendors advertise a low false-positive rate, the number becomes large at institutional scale. Vanderbilt’s example is powerful: 1% sounds small until it is applied to tens of thousands of assignments. A low percentage can still mean hundreds of innocent students investigated or stigmatized.

Vendors themselves warn against sole reliance

Turnitin’s own guidance says AI reports may not always be accurate and should not be the sole basis for punishment. The ABC report on ACU suggests that institutions can still over-rely on these tools despite those warnings.

The tools are least fair in exactly the settings where schools use them most

Student writing often includes second-language writing, formulaic discipline-specific phrasing, technical descriptions, revision help, and stylistic unevenness. Those are precisely the conditions under which detectors are more likely to misfire. The Stanford study makes this especially clear.

Benchmarks do not match deployment

Many studies test detectors on neat pairs of human and AI text under controlled conditions. Real life is messier: short answers, mixed editing, paraphrased outputs, domain-specific jargon, translated text, and novel models. The 2024 arXiv study and the 2025 NAACL Findings paper both show that real-world performance falls short of benchmark claims.

Common misconceptions

“If the detector says 80% AI, that is proof”

No. A detector score is not proof of authorship. It is a probabilistic output from an imperfect model, and vendors themselves caution against using it alone.

“The tools are getting better, so the problem is basically solved”

Recent research does show progress on some benchmarks, but that is not the same as dependable real-world use. The NAACL paper and the 2024 short-form detection study both report brittleness on unseen tasks and vulnerability to editing or humanization.

“False positives are rare enough not to matter”

That is wrong in two ways. First, some populations face much higher false-positive rates, especially non-native English writers, as shown by the Stanford-led study. Second, even low rates matter in high-stakes academic or employment contexts.

Better alternatives

The evidence suggests institutions should stop treating AI detectors as evidence of guilt. A better approach is process-based:

• Use clear course or workplace rules on permitted AI assistance.
• Ask for drafts, revision history, outlines, or oral follow-up when authenticity matters.
• Redesign assessments so students must show reasoning, source use, or personal reflection.
• Treat detector output, at most, as a weak screening signal that requires independent evidence.
• Prefer provenance methods where available, such as signed metadata or model-side watermarking, while recognizing that these only work when the generation system itself supports them.

For practical teaching guidance, see MIT Sloan’s article on why AI detectors don’t work.

Conclusion

AI detectors are not working properly because they are trying to infer authorship from unstable statistical cues in a world where writing is increasingly hybrid, edited, and diverse. The result is predictable: false positives, false negatives, bias against some writers, easy evasion, and damaging institutional mistakes. The most convincing proof is not only in research papers but in the behavior of the organizations involved: OpenAI withdrew its own detector, Vanderbilt disabled Turnitin’s detector, MIT advises against relying on such tools, Turnitin warns against sole reliance on AI scores, and real students have suffered investigations and harm.

A fair bottom line is this: AI detectors may sometimes raise suspicion, but they do not reliably establish the truth. In high-stakes settings, that makes them unfit to function as proof.

Reference links

• OpenAI: New AI classifier for indicating AI-written text
• NAACL 2025 Findings paper
• Stanford HAI: AI detectors biased against non-native English writers
• arXiv 2024 study on short-form AI detection
• Inside Higher Ed on professors and AI detection caution
• Turnitin AI Writing Report guidance
• Vanderbilt guidance on disabling Turnitin’s AI detector
• MIT Sloan: AI detectors don’t work
• ABC News on Australian Catholic University case
• Spectrum News on University at Buffalo student case
• Yale Daily News on Yale lawsuit
• Reuters on Massachusetts high school case