When a colleague pastes a news brief into an AI detector and gets back a score of 87% AI-generated, what exactly produced that number? For editors making consequential decisions about submitted work, understanding the mechanics behind that figure matters enormously. This explainer walks through the core concepts in plain language, without requiring a computer science background.

The fundamental question a detector is asking

Every AI detector is, at its core, asking a single question: does this text look like the output of a large language model, or does it look like the output of a human mind? To answer that, detectors analyse statistical patterns in the text. They are not reading for meaning the way an editor does. They are measuring how predictable each word choice is, given the words that came before it.

Large language models generate text by predicting the next most likely token (roughly, a word or word fragment) at each step. Because they are trained to produce fluent, coherent output, their choices tend to cluster around high-probability options. Human writers, by contrast, make unexpected leaps, introduce idiosyncratic phrasing, and sometimes choose the surprising word over the obvious one. Detectors try to exploit that difference.

Perplexity: how surprised is the model?

The most important concept to understand is perplexity. In this context, perplexity is a measure of how unpredictable a piece of text is to a language model. Low perplexity means the text was easy for the model to anticipate, word by word. High perplexity means the text kept making choices the model did not expect.

A detection system runs the text through its own internal language model and scores each token by how likely it was. A passage where nearly every word was the statistically obvious choice will return a low perplexity score, which the detector interprets as a signal of machine authorship. A passage full of unusual constructions, rare vocabulary, or abrupt topic shifts will score higher perplexity, suggesting a human hand. Our technical deep dive into AI content detection explores the mathematics behind this in more detail for those who want it.

Burstiness: the rhythm of human writing

Perplexity alone is not enough, which is why most detectors also measure burstiness. This concept captures the variation in sentence structure and length across a passage. Human writers tend to alternate between long, complex sentences and short punchy ones. They write in bursts of complexity followed by moments of simplicity. AI-generated text, even when it achieves low perplexity throughout, often maintains a more uniform rhythm, with sentences that are consistently medium-length and structurally similar to one another.

By combining perplexity and burstiness scores, a detector builds a more complete statistical portrait of the text. Neither metric is definitive on its own. That is precisely why the accuracy problems described in our piece on why AI detectors disagree with each other are so persistent: the two signals can point in opposite directions, and different tools weigh them differently.

Token probability distributions

Some more sophisticated detectors go beyond simple perplexity and examine the full probability distribution at each token position. Rather than just asking "was this word likely?", they ask "does the spread of possible choices here look like what a language model sees, or something else?"

This approach, sometimes called log-probability analysis, is more computationally intensive but can catch cases where a human writer happened to produce conventionally predictable prose, or where an AI was prompted in ways that pushed it toward unusual word choices. Tools like GPTZero have discussed this kind of multi-signal approach in their published documentation, and it is part of what differentiates commercial detector offerings. Our comparison of GPTZero, Originality AI, and Turnitin examines how these architectural choices translate into different results on the same content.

Training data and model specificity

A detector's accuracy is also shaped by what it was trained on. A system trained heavily on outputs from GPT-4 may perform less reliably when encountering text produced by a different model family, or by a model released after the detector's training cutoff. This is not a flaw unique to any one product; it is a structural reality of how machine learning classifiers work.

It also explains why detectors can struggle with translated text, non-native English writing, and highly technical or formulaic content such as financial disclosures or legal boilerplate. These text types share statistical properties with AI output not because they were machine-generated, but because their genre conventions constrain word choice in similar ways. Editors working in multilingual newsrooms or covering specialist beats should keep this limitation clearly in mind before acting on any score.

What this means for editorial practice

Understanding these mechanics should change how we read detection scores in practice. A high AI-probability score is not a finding; it is a signal worth investigating. It tells us that the text's statistical profile resembles what a language model tends to produce. It does not tell us the text was produced by a language model, and it certainly does not tell us anything about intent.

The right editorial response is to treat a flagged score the way we treat any other piece of uncertain sourcing: as a prompt for further conversation with the contributor, not as grounds for automatic rejection. For a broader view of the detection landscape and its limits, our pillar piece on AI content detection technology sets out the full context that individual tool reviews cannot provide.

Detectors are one instrument among many. Used alongside editorial judgment, contributor relationships, and transparent AI policies, they can be genuinely useful. Used as oracles, they will produce injustice.

Sources

  • GPTZero, technical documentation and methodology notes, gptzero.me
  • Turnitin, AI writing detection capabilities overview, turnitin.com
  • Originality AI, how it works documentation, originality.ai
  • Tian, E. et al., "GPTZero: Towards Detection of ChatGPT-Generated Text Using the Detector", referenced in GPTZero public materials
  • Mitchell, E. et al., "DetectGPT: Zero-Shot Machine-Generated Text Detection Using Probability Curvature", Stanford University, 2023