Perplexity and Burstiness: How AI Detectors Spot AI Text

If you've used an AI detection tool, you may have encountered the terms 'perplexity' and 'burstiness' without a clear explanation of what they mean. These two statistical measures are the foundation of most commercial AI detection algorithms, and understanding them helps you interpret detection results more accurately — and understand why AI detectors sometimes get it wrong.

Perplexity measures how 'surprising' a piece of text is to a language model. When a language model reads a sentence, it assigns probabilities to each possible next word. Text where the model consistently predicts the correct next word is 'low perplexity' — it follows the statistical patterns the model learned from training data. AI-generated text tends to be low-perplexity because it was literally produced by predicting probable next words. Human writing, shaped by individual style and idiosyncratic choices, tends to be higher-perplexity.

Burstiness measures the variation in sentence complexity across a text. Humans naturally write with significant variation: short punchy sentences followed by longer, more complex ones. AI-generated text tends toward more uniform sentence structure — consistent length and complexity throughout. Low burstiness alongside low perplexity is a strong statistical signal of AI generation.

Neither measure is definitive. Very formal human writing (legal, scientific, administrative) has naturally low perplexity because its genre conventions produce predictable language. Writing that has been heavily edited for clarity and consistency may have lower burstiness than typical human prose. This is why AI detectors have meaningful false positive rates and why the scores they produce should be interpreted as probabilistic indicators rather than ground truth.