How TextFlag detects AI-generated text
The models, the calibration and the measured accuracy behind every TextFlag result, including where it can go wrong.
In short
TextFlag splits your text into sentences and scores them with two transformer language models trained to tell human writing from AI writing. One model sets the document's AI percentage; the other decides which sentences to highlight. The percentage is calibrated against documents of known authorship, so it describes the share of the text that is AI-written rather than a raw model probability.
Two models, two jobs
The context model sets the percentage
The first model reads each sentence together with the sentences before and after it. Seeing context makes it the stronger judge of a whole document, so it drives the overall AI percentage.
The sentence model decides the highlights
Context can also mislead: a human sentence sitting between two AI sentences tends to “borrow” their score. So a second model scores every sentence on its own, and only that model decides what gets highlighted. You see the sentences that are AI-like by themselves, not by association.
Runs, not single sentences
A single flagged sentence is usually noise. TextFlag highlights a sentence only when it belongs to a run of consecutive AI-like sentences (three in a row in Standard mode). This one rule removes most of the isolated false flags that single-sentence detectors produce.
Calibrated against known authorship
Raw model scores are not percentages. We calibrated TextFlag against 658 documents of known authorship: pure AI text from four model families (Claude, GPT, Gemini and DeepSeek), documents mixed at 20%, 40%, 60% and 80% AI, and 440 human-written papers from before 2019, before modern AI writing tools existed.
- Nine documents in ten land within 9 percentage points of their true AI share.
- Pre-2019 human papers score about 1% or less on average.
- Weakest cases: documents that are 20% AI tend to read about 13%, and pure Claude text about 92%.
Detection modes and measured accuracy
How strict a detector should be depends on who is reading the result. A teacher cannot afford to flag a student's own work; a writer checking their own draft would rather see every passage that might read as AI. Measured on 132 held-out documents (46,862 sentences):
| Mode | Flagged sentences that are AI (precision) | AI sentences caught (recall) | Human sentences flagged by mistake |
|---|---|---|---|
| Standard (default) | 95.3% | 41.2% | 1.2% |
| Thorough | 70.5% | 80.5% | 20.5% |
The mode only changes which sentences are highlighted. The AI percentage comes from the calibrated context model either way.
How to read results under 20%
TextFlag shows the exact score and any highlighted sentences at every level. The low range, between 1% and 19%, is where false positives are most likely, so the report adds a warning there: treat those few highlights as passages to review, not as evidence of AI use.
Known limitations
- Short texts: under about 40 sentences the percentage is a rough indication; the margin widens to about ±25 points and the report says so.
- Languages: calibration is English-only. Other languages are analysed, but the percentage is less reliable.
- Edited AI text: heavy paraphrasing or rewriting makes AI text harder to recognise for every detector.
- Formulaic human writing: very formal or template-like prose can share features with AI text.
Using results responsibly
An AI detector gives you evidence to review, not a verdict. Read the highlighted passages, compare them with the writer's other work and drafts, and talk to the writer before reaching any conclusion. Never use a detection score as the only basis for an academic or employment decision.
See it on your own text
Run a free AI check and get the highlighted report in seconds.