Field notes

Notes from the reading tavern

We are building a pocket reader and a pocket tavern. These are the potholes, trade-offs, and field tests we hit along the way — written by the people doing the work.

Ink-etched illustration on aged paper: a manuscript page pinned to a clothesline over a wash basin, dark ink letters dripping off into the water while a red architectural scaffold of story structure stays crisp on the sheet
Research notesJul 26, 202610 min

Story structure doesn't wash off

Word-list AI detection is a lost arms race — flagship models already pass slop scans clean. The StoryScope paper (arXiv:2604.03136, UMD + Google DeepMind) found something sturdier: narrative structure alone separates human from AI fiction at 93.2%, and professional-grade rewriting moves that by 1.6 points. We projected its lexically visible features into a browser tool and validated on 78 passages of our own corpus: five axes separate (direct emotion naming leads at d=1.46), every direction matches the paper — and our own show-don't-tell discipline turned out to carve a brand-new fingerprint. Also: how we killed our strongest number because it was fake.

Read the post →
Ink-etched illustration on aged paper: a handwritten character sheet on a wooden easel, red marionette threads rising from its underlined lines to a puppet control bar, then down to a small seated figure whose wrists and jaw the threads are lifting
Character cardsJul 26, 20269 min

Your character card is teaching the bot to speak for you

Impersonation — the bot writing your dialogue, your actions, your feelings — is the complaint JanitorAI's own help center calls one of the most common and frustrating. The root cause usually isn't the model: it's second-person sentences baked into the card's greeting and example dialogue. We scanned the 30 most-starred cards on Chub with a static checker: 14 clean, 11 with minor habits, 5 needing surgery — including the board's most-starred roleplay card at 19,568 stars. Here's the anatomy, with before/after rewrites, and a browser-local scanner to check your own.

Read the post →
Hand-drawn ink illustration on aged paper: a row of customs windows, a traveler passing a bundle of index cards through; small trays under each window catch fallen labels, and the bundle emerging at the far end is thinner, with one tag flipped upside down
RoleplayJul 26, 20269 min

The file made the trip. The semantics didn't.

A lorebook that hums in SillyTavern can go silent in JanitorAI and fire backwards in RisuAI — without anyone editing an entry. We read the open-source importers and matchers line by line while building a migration checkup: the 1% probability default, disabled entries that revive, NOT_ANY guards that invert into requirements, a 0% entry that becomes always-on, and keyword normalizers that eat CJK keys whole. Every behavior claim in this post is labeled source-verified, docs-backed, or honestly untested.

Read the post →
Hand-drawn ink illustration on aged paper: a wind-up gramophone with its needle stuck, identical speech bubbles spiraling out of the horn and curling back into the machine, a small pair of scissors lying on the table beside it
RoleplayJul 26, 20269 min

Your bot isn't broken. Your history is teaching it.

Three different diseases get filed under “looping”: verbatim re-paste, pet imagery, and fossilized openers. A triage guide built from our repetition detector: 12-token shingle receipts on Grok 4.5 (three straight rounds at 100% old text) and DeepSeek (one 72% slip, self-corrected), why history contamination beats every sampler knob — corrective instructions lost 9 of 10 rounds in our test — which penalty settings backfire, and the fix list in the order that actually works.

Read the post →
Warm-paper specimen-cabinet illustration: three ink book spines on display — the first curls into a closed loop, the second freezes mid-length into ice, the third quietly coils back to its starting point — with a magnifier and three specimen tags below
Research notesJul 26, 20267 min

Feels like the AI is repeating itself by round fifteen? It probably is.

Nine models, two genres, 20 consecutive rounds each — every chain archived, every repetition figure recomputed on the raw text. Long-run collapse comes in exactly three shapes: the restart loop (20 rounds, 20 copies of the same opening), the mid-chain freeze (three straight rounds of 100% recycled text), and the ending rewind — 0% to 1.8% verbatim overlap, invisible to detectors, while the plot quietly rewinds to round one. Plus the cold number behind our methodology: AI judges scored 12% on known-answer anchors, agreeing with each other 86% of the time — unanimously wrong.

Read the post →
Warm-paper woodcut of movable type blocks queuing on a conveyor belt into a printing plate; two blocks fall into a crate below, one gets a new label swapped into the queue, and a crooked extra block sticks out at the end of the plate
SillyTavernJul 26, 20268 min

Edited the preset, nothing changed? Check whether it was ever sent.

You edit a preset block and the replies don't move. A world info entry triggers in the test panel but never shows up in the request. Example dialogues have been dead since day one. All three symptoms live in the same place: the assembly step between pressing send and the request leaving. A walkthrough of SillyTavern 1.18's real assembly semantics — defined does not mean ordered, a disabled marker discards while a missing marker appends to the very end, the main prompt gets silently replaced by the card, and example blocks without name prefixes are dropped without a word.

Read the post →
Warm-paper woodcut illustration: an open manuscript folds up into a small paper stage, paper-doll actors wear tiny name plaques, and one doll without a plaque is being moved to a narrator's bench at the wing
EngineeringJul 26, 20267 min

Everyone throws an LLM at novel-to-galgame. We wrote a rule compiler.

The open-source state of the art fine-tunes an 8B model on 669 novels and still misattributes roughly one line in eight. Our web tool goes the other way: a deterministic rule compiler — five attribution rules plus a standing right to abstain — that found 16 characters with zero false names in a Dream of the Red Chamber excerpt, and turns a 5MB, 258-chapter book into a stage report in 264 milliseconds. No model, no upload, works offline.

Read the post →
Archaeology cross-section illustration on warm paper: a trench dug through strata of chat speech bubbles, the deep layers hand-drawn and varied, the upper recent layers identical and machine-stamped; a thin cinnabar boundary line with a small red flag marks the change, and a tiny archaeologist on a ladder inspects it with a brush
Research notesJul 26, 20268 min

Slop creep is a mechanism — and it has a floor number

Same model, same instruction: a fresh chat obeyed 10 rounds out of 10, a chat with polluted history disobeyed in 9. Long-chat degradation isn't the model getting dumber — it's the model imitating its own old output, with the imitation weight growing every message. The mechanism, what our 300-turn benchmark measured (image-level repetition, +2% to +4.9% reply inflation per turn, rule decay from 253 to 631 words), how the Chat Archaeology lab draws the creep as a curve with a detected inflection floor, and a recovery playbook ordered by surgery size.

Read the post →

Earlier posts (81)

Jul 2026