Prefix caching,
and what your chats really cost

The one-minute version

1. Major providers automatically discount the part of a request that is byte-identical to the previous one, from the first byte on: DeepSeek bills cache hits at 2% of full input price; Anthropic, OpenAI and Google sit around 10%; xAI, Zhipu and Qwen at 15-20% (list prices, 2026-07-24). No sign-up; the only requirement is a byte-stable beginning.
2. Hit rate is client behavior, not luck: same model, same key, byte-identical scripts. Stock SillyTavern hits 91.5% out of the box; we measured 46.5% before our fix. Same 15-round chat, 2.8× the money (July 2026 four-way benchmark; 89.9% after the fix, shipped).
3. Inside one app, card structure alone spreads hit rates threefold: a card whose lorebook is all keyword-triggered measured 22.6%; one with two always-on entries measured 74.3% (pre-fix). Your card is quietly deciding your bill.
4. Extrapolated to 100 rounds on the same formula: the 46.5% setup pays $0.20 per session, the 89.9% setup $0.054 (DeepSeek V4 Flash). The longer the history, the wider the gap: 2.8× measured at 15 rounds grows to 3.8× at 100.
5. Only one kind of action kills the cache: changing the beginning of the request. Four measured cases: lore injection drift, dynamic lines routed through a system role that gets merged to the top, plugin floods and mid-history injection, and sliding-window truncation once the context fills.
6. Output tokens never get the discount. The pricier a model's output, the less caching can save: fixing the hit rate from 46.5% to 89.9% cuts a DeepSeek bill by 74%, but a Claude Sonnet 4.6 bill by only 60%, because most of its money sits in $15/M output.

What prefix caching is, and how deep the discount runs

Prefix caching (prompt caching) is an automatic provider discount: the part of this request that is byte-identical to the previous one, starting from the very first byte, bills at the cache-read price instead of full price. DeepSeek's cache-read is 2% of full input; Anthropic, OpenAI and Google are around 10%. Nothing to apply for, nothing to code; the response's usage field reports how much was hit.

Roleplay and long-form chat are the biggest beneficiaries because every turn re-sends the entire history: by round 100, the previous 99 rounds are all “repeated prefix”, so in theory well over 90% of billed input tokens should ride the discount. A five-round Q&A barely notices it. Caching decides long-chat bills, not short ones.

Four bills for the same chat: the instrumented benchmark

In July 2026 we ran a four-way benchmark: bare SillyTavern (v1.18.0 stock), fully-modded SillyTavern (the community's most popular preset + plugin stack), Foreverse standard, and Foreverse Smart Play — same model (deepseek-v4-flash), same key, byte-identical 15-round scripts, every call metered through one gateway. The first defect it caught was ours.

What 100 rounds cost at each hit rate

Plugging the measured hit rates into the exact formula our calculator uses (a 2,000-token card-plus-system prefix, 120 tokens in and 350 out per round, DeepSeek V4 Flash at 2026-07-24 list prices) gives the following input-plus-output totals for a 100-round chat. This is a same-formula extrapolation, not a measurement; the measured data is the 15-round set above.

One bill line people miss: output tokens always bill at full price; caching only covers input. So the pricier a model's output, the less caching can rescue. The same 100 rounds on Claude Sonnet 4.6 ($15/M output): $4.97 at the 46.5% tier, $2.01 at 89.9%: the same hit-rate repair cuts the bill by only 60% (vs 74% on DeepSeek), because most of the money sits in output. On expensive-output models, trimming reply length pays off before cache repair does.

What kills your cache: four measured cases

Prefix caching has exactly one failure condition: the beginning of the request changed. All four cases below come from our own measurements; three of them happened in our own product, with numbers from before and after the fix.

Self-audit: six cache-killing habits

Each item gives the symptom, how to check, and what to do. The first four audit your own configuration; the last two audit your provider and your session length. Step zero is always getting the real hit number: it's in the response's usage field, and what you can't measure you can't fix.

1. Your lorebook is almost all keyword-triggered, with no always-on entries. Check: Chat a dozen rounds on one card and watch the per-turn hit rate; sawtooth everywhere is the tell (our measurements: 22.6% all-triggered vs 74.3% with always-on entries). Do: Make core entries always-on (constant/“blue light” in SillyTavern), or use a client that keeps once-activated entries resident.
2. Timestamps, status bars, or any per-turn content live in the system prompt or the head. Check: Export two consecutive request bodies and diff the first screen. Some chat templates merge all system messages to the top, so “written at the end” does not mean “effective at the end”. Do: Move dynamic content next to the latest user message; keep the head constant. Our two chains went from 53%/58% steady-state to 96%+ with exactly this change.
3. You run mechanisms that insert into the middle of history: depth injection, mid-depth author's notes, per-turn vector retrieval, rolling summaries. Check: Read the plugin's docs for any mention of caching (serious ones warn you); otherwise compare hit rates with it on and off (bare ST with dice config: 91.5% → 39.8%). Do: Set injection depth to the tail if the option exists; if not, accept the tax knowingly — and don't stack a second mechanism from the same family.
4. You frequently edit, delete, or reorder old messages. Check: Nothing to check; the behavior is the answer: everything after the edit point is invalidated. Do: Nothing to fix. It's a fair price for a real feature; just know the next turn after an edit will fully miss, and don't sweat single-turn dips.
5. Your provider or relay doesn't report hit numbers. Check: Look for cached_tokens, cache_read, or prompt_cache_hit_tokens in the response usage; if none exist, you're flying blind. Do: Switch to a direct connection or a channel that reports hits. We've measured a relay pool where byte-identical requests never hit across turns; when the pool isn't sticky, the discount simply never arrives.
6. Deep into a long chat, the hit rate falls off a cliff and never recovers. Check: Check whether session tokens are approaching the model's context limit; once full, the client starts sliding-window truncation and the head moves every turn. Do: Turn on summarization or compaction yourself instead of letting truncation cut history for you: truncation is a moving head every turn; seam-pinned compaction at least restores a stable prefix between seams.

Tools, and where to go next

The division of labor in one line: the tools compute your numbers; this report explains why, and what to do.

• Cache savings calculator — Plug in your turns, model and hit rate; the four measured anchors (46.5 / 73.1 / 89.9 / 91.5) are one-tap buttons
• Context lens — What's occupying your context window: card, history, injections, block by block
• LLM price table — Cache-read prices across providers, sortable, snapshotted daily
• The four-way benchmark (data source) — Full method, per-cell data and anti-cheating audit behind this page's hit rates
• Benchmarked by our own benchmark — The 46.5% → 89.9% fix, in full
• Prompt caching, explained — The mechanism itself, in plain words

Method and scope

• Four-way hit rates and costs: the 2026-07-23/24 benchmark: same model (deepseek-v4-flash), same key, byte-identical 15-round scripts; 3 cards × 4 setups × 3 independent repeats = 36 cells (pre-fix baseline) plus an 18-cell post-fix comparison; every call metered through one transparent gateway, and a cell only counts after the gateway, the ST-side log and the app-side ledger agree. About 1,530 real calls, roughly $1.07 at list prices.
• The two product-chain numbers (53%→97%, 58%→96-98%): production probes against DeepSeek direct, padded with 30 rounds of history, independent salts, two consistent re-runs (2026-07-14).
• Discount table and 100-round bills: models.dev list-price snapshot of 2026-07-24; the simulation formula is identical to our calculator's (2,000-token prefix, 120 in + 350 out per round), with parameters stated in the text.
• Cache hits are counted from provider usage fields (DeepSeek's prompt_cache_hit_tokens and friends), not estimated by us.
• Everything is recomputable: per-call metering logs, per-cell transcripts and reconciliation records are archived behind the four-way benchmark page.

Honest limits

• All measured hit rates come from a single model (deepseek-v4-flash) through one gateway. The discount table lists many providers' prices, but “how much you will hit” does not transfer across models: the failure mechanics are universal, the percentages are not.
• The benchmark scenario is a 15-round fixed-script roleplay session, not a distribution of real-world usage. Your hit rate is decided by your card structure, plugins and editing habits, which is exactly why the self-audit list exists.
• Costs are mechanical conversions at public list prices, for between-group comparison only, not a prediction of your bill. Cache-write fees and cache lifetimes (minutes to hours, provider-dependent) are not modeled.
• The 100-round section is a same-formula extrapolation, not a measurement; the measured data is the 15-round set.
• Our fix numbers come in two shapes: 89.9% measured on fresh installs, 73.1% on upgraded ones; closing that gap touches stored configs and is a separate product decision this page won't promise for.
• Prices are a 2026-07-24 snapshot and will drift; providers' pages are authoritative. CNY figures use 1 USD ≈ 7.2 CNY.

FAQ