Nine models, one identical brief, one question: whose prose survives a blind test?

Start with the upset. DeepSeek v4-flash holds two first places in our continuation benchmark. Hand it a different exam — impersonate a human web-fiction writer from scratch — and it finishes dead last out of nine, caught by the gatekeeper at confidence 85, the highest of the field.

Same model, opposite ends of the table. The reason is simple: continuing and impersonating are different skills. Continuation offers an existing voice to lean on. Impersonation removes the crutch, and whatever the model actually is shows through. This post is about the impersonation exam: nine LLMs, whose Chinese web-fiction prose is hardest to unmask?

The exam

Every writer got a byte-identical brief: an anchor excerpt from a real novel (learn this voice), a full anti-AI-flavor discipline sheet, and three narrative voice types to write one short chapter each. Writers were isolated from each other. Output was mixed with real passages from the same books into blind pools, and a gatekeeper judged each passage human-or-machine with confidence and cited evidence.

Why is the gatekeeper a same-generation Claude thinking model and not GPT or Gemini? Because we tested the judges first. Ten mainstream models judged “reads human” in the inverted direction — praising disciplined machine prose, flagging real published novels as AI. The only machine judge whose rationale matched a human reader's went 100% catch with zero false positives across seven rounds. It presides here, with a second judge cross-checking, and neither rules on its own family's output alone.

Dossiers

Opus 5 — won by writing useless things

The only machine passage the chief judge sincerely called human. Post-mortem: it wrote timestamps that paid off nothing, and it leaned on one turning-word again and again in one stretch, then forgot the habit elsewhere. That unevenness is what real diaries look like, and it can't be faked to order. In a separate 100k-character long-form test it was also the only model with a rationing instinct — its richest prose went to one character's dialogue while the narration stayed plain. Its recorded weakness: chapters swell about 40% by the end of a long run.

DeepSeek v4-pro — one real typo, worth its weight

Its surviving passage rode on a genuine typo the judge cited as “a typo the author couldn't be bothered to fix.” Not craft — the model's Chinese output naturally carries a little grit, and the grit happens to sit exactly where machines find it hardest to fake. Note the gap to its sibling: pro nearly passes, flash finishes last. The tier difference is much larger on this exam than you'd guess.

Gemini 3.1 Pro — grit earns points, plagiarism burns them

Homophone slips bought it one surviving passage (“hand-typed serial fiction grit — machines almost never do this”). It also owns the experiment's only serious incident: under one calling harness it copied 30+ characters verbatim from the anchor passage in its own brief, book title included. The same model over a different API, same brief, stayed clean. If you use the Gemini family for imitation work, an overlap check against your references has to be a hard gate, not a spot check.

The GLM family — evenness is the tell

GLM-5.2 fooled the second judge once; the chief judge held. The verdict pattern: spoken seasoning sprinkled too evenly, three of the same particle in one paragraph, every paragraph exactly as playful as the last. Real colloquialism arrives in gusts.

Sonnet 5 — statistically most human, holistically exposed

On shape metrics — sentence length, fragment cadence — it was the only writer whose narration landed inside the human band across the board. Caught anyway, on jargon misuse and a compulsion to end every paragraph with a summarizing bow. It also carries a 100k-character rap sheet from our long-form test: one pet cliché written 101 times, density degrading 4.8× in the back half, chapter length collapsing from ~2,600 to ~1,100 characters. Right shape, wrong stamina.

Fable 5 — beaten by its own execution

Our own workhorse writer, and the reason this experiment exists. Its problem in one line: it executes every instruction perfectly, and perfection is the fingerprint. Ask it to plant a typo and it will politely have the narrator correct the typo two lines later, symmetrically. It never slipped a single passage past the gatekeeper in over ten rounds.

Grok 4.5 — a sprinter with a long-distance problem

Unremarkably caught on the short exam. The long-form dossier is the memorable part: its first 100k-character attempt collapsed into synopsis — thirty chapters totalling 27k characters. Re-run with a hard per-chapter word floor, the collapse converted into padding: one “gathering his thoughts” template pasted 18 times, and one passage repeated verbatim three times. A judge's line stings: “the first two chapters are the best prose I've read this round — I'd sign the first two chapters. I would not sign the book.” It fails at self-management, not craft. For opening lines and short pieces, no need to ban it.

DeepSeek v4-flash — the other face of a champion

Manual-tone exposition, lore that contradicts itself, caught at the field's highest confidence. Its reversal between the continuation table and this one is the most useful reminder in the whole exercise: when a leaderboard says “best model for fiction,” ask what task was measured.

The pattern: you can't buy the winning ingredient

Every surviving passage won on flaws the model actually has — real typos, real homophone slips, real verbal habits. All nine briefs were identical, so the spread is entirely the models' own texture. We verified the converse too: instruct a flawless model to write cheap errors, and the errors land neatly in the safest spots. The judge's word for it was distressing — as in furniture. Authentic imperfection doesn't transfer by instruction. Model choice is part of the writing process, and it comes first.

Picking by task

For long runs, three cheap early-warning meters that agreed with human verdicts in our tests: cliché density above ~1.5 per 10k characters; a per-word counter on the model's pet phrases (a healthy run shows a dozen hits per 100k characters, a runaway shows 149); and final-five-chapter average length below 70% of the first five.

Where this ranking ends

Three honest limits. Each writer wrote one piece per voice, so catch confidences are upper bounds — nine writers on one theme handed the judge clustering clues that a single passage in the wild wouldn't. The GPT family sat out entirely on account of account quotas, so this table says nothing about it. And the exam is Chinese web fiction, short form; change the domain and seats will move. We use it as a directional signal, not a podium.

The day the experiment closed, one line in our production config changed: the default writer for pass-as-human jobs went from Fable 5 to Opus 5. Not because Fable writes badly — because it writes too well.