Teachers struggle to tell if a student used a chatbot to write an essay since AI use is often partial. Detectors tested on artificial examples may not work on real student work where AI is used voluntarily and mixed with original writing.
Researchers analyzed essays from 15-year-olds in Germany who wrote with access to a chatbot. Keystroke logs traced each word's origin, typed by student, pasted from AI, or retyped. They tested two detectors: one using linguistic features, another using embeddings from a language model.
Both detectors identified whole AI-written essays with very high accuracy. But for locating AI within a sentence, detection of AI sentences dropped from 0.92 to 0.60 when a simple rewriting prompt was used. This shows that students can easily evade sentence-level detection by asking the chatbot to sound more like them, making current detectors unreliable for partial AI use.