Labs · Cache calculator
What prompt caching saves,
at real price spreads.
saved
0%
$0.365 → $0.054
Cache read $0.0028/M — 2.0% of the full $0.14/M
Cumulative cost over turns — the same chat, billed two ways
What breaks the cache (and why many setups never see the discount)
Prefix caching has exactly one rule: the part of this request that is byte-identical to the previous one, starting from the very first byte, gets the discounted rate. Everything that breaks it breaks it by changing the beginning.
One: injection drift. Lorebook and memory entries inserted at a different position each turn invalidate everything after the insertion point. We got burned by this ourselves — a benchmark measured us at a 46.5% hit rate, and the culprit was exactly this; pinning the injection position brought it back to 89.9%.
Two: a changing head. Put a timestamp or status line at the top of the prompt and the first byte differs every turn — the entire history misses.
Three: rewriting history. Editing, deleting, or reordering old messages invalidates the cache from that point on. A fair price for the feature, but now you know how the bill works.
Prices captured 2026-07-24; providers' pages are authoritative · list-price (BYOK) basis; the official channel scales both lines by +50%, the saving rate is unchanged · tokens are estimates
This injection layout ships inside the app — pinned positions, resident entries, 89.9% measured hits. Nothing for you to tune.
Get this hit rate in the appHow much does prompt caching actually save on a long chat?
At DeepSeek's price spread, a cache hit bills at 2% of the full input price — so a roleplay chat that resends its whole history every turn saves more the longer it runs. Set the model, turn count and hit rate above and read the two cost curves; the default 89.9% hit rate is a measured value from our instrumented benchmark, not a marketing number.
The benchmark behind this page (it burned us first)
In July 2026 we ran a four-way benchmark — stock SillyTavern, fully modded SillyTavern, and two Foreverse modes — with every call metered through one gateway. The first finding was at our own expense: stock SillyTavern hit 91.5% cache out of the box, we hit 46.5%, and the same 15-turn chat cost 2.8× as much. Forensics traced it to lorebook entries injected at a position that drifted every turn, invalidating everything after the insertion point. With positions pinned and entries made resident, the re-test measured 89.9% — level with stock — and costs fell by roughly 60%. The fix has shipped.
Price spreads use public list prices: DeepSeek's cache-read is 2% of full input price; Anthropic and Google sit around 10%. The four slider anchors (46.5 / 73.1 / 89.9 / 91.5) are all measured values from that benchmark.
What the simulation leaves out
The curves assume uniform turn length and a constant hit rate — real chats wobble on both, and the turn where a new lore entry first enters is a guaranteed miss (a design cost, not a bug). Some providers charge extra for cache writes, and cache lifetimes range from minutes to hours; neither is modeled. And once a conversation overflows the context window and sliding-window truncation kicks in, caching breaks permanently — that's a different bill this page doesn't compute.
FAQ
Where does the default 89.9% hit rate come from?
From our own measurements. In July 2026 we ran a four-way instrumented benchmark — same model, same key, byte-identical message scripts — and it embarrassed us first: stock SillyTavern hit 91.5% out of the box while we sat at 46.5%, because our lore entries were injected at a drifting position every turn. After pinning the injection position we re-measured 89.9% on fresh installs and 73.1% on upgraded ones. All four numbers are anchor buttons on the slider.
Why does the saving grow with more turns?
Every turn resends the whole history, and the longer the history, the larger the share that is byte-identical to the previous request. At 10 turns the reusable prefix is roughly 60% of input; at 300 turns it exceeds 99%. Short chats barely notice caching — long chats are where it decides the bill.
Do I need to apply for the discount?
No — providers grant it automatically. DeepSeek, Anthropic, Google and xAI all report hit tokens and discounted billing in the response. The one requirement is on the client: requests must keep a byte-stable prefix. Fail that and the discount silently never arrives, which is exactly what this page is about.
Can I treat the result as a bill estimate?
As an order of magnitude, yes; as an invoice, no. Tokens are estimated; your real hit rate depends on client behavior (the 45-point spread between the anchor buttons is behavior, not luck); and providers differ on cache-write fees and cache lifetimes. The methodology section spells out every assumption.