How AI Detectors Actually Work: Perplexity, Burstiness, and the Statistics Behind the Score

Ask someone how an AI detector works and you'll usually get a shrug, or worse, an answer that makes it sound like a magic trick: "it just knows." It doesn't. Detection tools apply statistical language modeling — the same math that powers the AI models they're trying to catch — to measure how predictable a piece of text is. That single idea, predictability, is the foundation almost every detector builds on, TheChecker.AI included (perplexity-based detection, including GPTZero, actually launched in January 2023, so this isn't a post-2023 invention — it's the original approach the field has been refining since). Here's what's actually happening under the hood.

The core idea: language models predict, humans surprise

Words generated by GPT, Claude, Gemini, and other large models come token by token, and at each step the model calculates a probability distribution over every possible next word. Strict argmax decoding — always picking the single highest-probability word — is only one way to sample from that distribution, and it's rarely how production chatbots actually run. Real deployed models use temperature and top-p sampling, which deliberately introduce some randomness so the output isn't robotically uniform. That distinction matters for detection: a model sampling with temperature isn't producing "exactly what the model would have written" in the strict deterministic sense, it's drawing from a distribution, which is also why, as the later section on perplexity and burstiness explains, modern generation can already narrow the statistical gap a detector is trying to measure.

A simple hypothesis guided the detection work: if a text tends to follow the highest-probability path a model would take, then feeding this text back into the model will produce a closer-than-usual match to the original sequence. A tight match is a signal of likely artificial origin, not certain proof of it. That gap between "signal" and "proof" is the whole story of why detection is probabilistic, not a yes/no scan.

Perplexity: how surprised is the model?

The first and most widely used signal is perplexity. GPTZero defines it plainly: perplexity measures how likely a language model would have chosen the exact same sequence of words that appears in the document (GPTZero). Originality.ai frames it the same way — perplexity measures how well a model can predict the next word in a sequence (Originality.ai).

When perplexity is low, the text is highly predictable for the model. Each word aligns closely with the model's own choice. This pattern marks machine-generated text. The language is smooth and statistically safe, favoring common word choices. It manifests as a sequence of sentences.

High perplexity means the model kept getting surprised — unusual word choices, idiosyncratic phrasing, the kind of specific, sometimes-clumsy detail a real person produces because they're drawing on lived experience, not a probability table.

Burstiness: the rhythm of real writing

The second classic signal is burstiness — and it's about structure, not word choice. QuillBot describes it well: burstiness reflects variation in sentence length and structure across a passage (QuillBot).

The burstiness of human writing shows itself when a tight four-word phrase precedes an extended, detailed assertion. Then a short line circles back, re-introducing the core idea. We don't chase uniformity; the flow mirrors the way we think aloud as we draft.

When a text is produced by a model, it tends to be uniform. Sentence lengths stay close together. Paragraph rhythm smooths into a steady beat. Because the model samples from a probability distribution at each step, it pulls toward an average. That process flattens the peaks and valleys that make human writing feel alive.

Why perplexity and burstiness alone aren't enough

Here's the part a lot of "how AI detection works" explainers skip: these two signals, on their own, are necessary but not sufficient — and not just because sampling narrows the gap. Pangram Labs, one of the few vendors that publishes its methodology, makes a more fundamental case against perplexity and burstiness as the sole basis for detection: because language models are trained to minimize perplexity on the text they're shown, any human-written passage that shows up often in training data gets low perplexity too, and reads as "AI" to a perplexity-based detector regardless of who wrote it. Pangram's own demonstration of this is the Declaration of Independence, so heavily reproduced across the web that it scores as machine-uniform low-perplexity text to a model that's memorized it, even though no AI wrote a word of it (Pangram Labs). That's a false-positive problem baked into the method, not just an evasion problem — and it compounds the same non-native-English-writer bias covered later in this piece, since simpler, more common phrasing scores as more "predictable" independent of who wrote it. Separately, and for that reason, Pangram has said publicly it doesn't build its detector around perplexity and burstiness at all.

That's the reason a serious detection engine layers more than two statistics. A single perplexity score can't tell you which model wrote something. It only tells you that something looks synthetic, and on its own it can misfire on genuinely human text too. At TheChecker.AI, our models cross-reference sentence-level and paragraph-level scoring. That scoring draws on patterns learned from over 40 AI systems, including Claude, GPT-5, Gemini and Llama. We publish results for that layered approach on our own accuracy page. Read those figures with one caveat in mind: they're our own published numbers, not an independent third-party audit.

The honesty problem: why detectors disagree with each other

It's worth being upfront about something the industry doesn't advertise enough: detection tools disagree with each other more than people expect, and independent academic evaluation backs that up. A 2026 study published in the International Journal for Educational Integrity directly compared detector accuracy on real academic writing samples and found meaningful gaps in both overall accuracy and recall between tools evaluated (Springer, 2026). A separate 2025 study in PMC specifically examined whether academic AI detectors can be trusted, cataloguing the accuracy limitations of the category as a whole (PMC, 2025).

This isn't a reason to distrust detection outright — it's a reason to want a detector that's transparent about how it scores, and that treats a result as a confidence signal rather than a verdict. There's also a well-documented gap that any honest "how detection works" explainer has to name directly: perplexity-based scoring is measurably biased against non-native English writers. A widely cited Stanford study found detectors misclassified 61.22% of genuine TOEFL essays written by non-native speakers as AI-generated, versus near-perfect accuracy on native-English writing from the same age group — because simpler, less varied vocabulary reads as more "predictable" to the model doing the scoring, whether or not it's a red flag for anything (Liang et al., arXiv).

Gut instinct alone fares badly here, and there's a specific reason why. The UK's National Centre for AI reports that people are generally worse than AI detectors at spotting AI-generated writing (JISC National Centre for AI, 2025). Left to their own gut feel, most people also flag more false positives than a trained detector does. But look closer at the study JISC cites, and "humans are worse" splits into two very different groups. People who use ChatGPT often for writing get good at spotting it, often beating casual readers and holding their own against dedicated detection software (Russell et al., arXiv:2501.15654). So the honest version isn't "humans can't tell." It's this: an untrained gut call is a guess wearing a confident face. A practiced eye, sharpened by frequent exposure to AI writing, is a real skill.

What this means in practice

Understanding perplexity and burstiness won't make you a detection engine, but it explains why detection works at all — and why it isn't infallible. If you'd rather eyeball a passage first, seven telltale signs of AI-generated writing walks through the patterns readers notice by eye, though a statistical read still catches what the eye alone misses. The next time you get a score back from an AI checker, you're looking at the output of a statistical comparison against how predictable, and how rhythmically uniform, a passage is relative to known human and machine writing patterns. It's not a black box; it's applied language modeling, running in reverse.

Reading about perplexity and burstiness only gets you so far. Try this instead: run a paragraph you wrote yourself next to one you know came from an AI, both through TheChecker.AI's free demo. Watching the two signals pull apart in real time is what actually makes the theory click — takes about ten seconds.