A 94.3% hit rate, and it was wrong

Some context first. If you build long-term memory for character chat, one concrete question is unavoidable: where do the retrieved memory entries go in the prompt? At the head, every memory change invalidates the prefix cache for the entire history and the bill takes off. At the tail, caching stays happy — provided the tail is actually the tail. Our first experiment (24 rounds) measured a standalone system block appended after history at a 94.3% cache-hit rate. We wrote it into the design doc as the recommended layout.

Three weeks later the number fell apart.

The 400-round rerun: 3.3% versus 96.6%

The rerun used a meaner setup: 400 rounds, 47 memory-change events, full unwindowed history — the final prompt weighed 43k tokens. Two layouts ran on identical scripts, and each session carried a random salt for isolation (that detail is the whole story; more below). Results:

Read the middle column. The moment memory updates, the “tail system” layout loses essentially all cache hits — the exact accident injection placement is supposed to prevent, in the layout our first experiment had ranked best. The user-message layout barely flinches on the same updates.

Root cause: DeepSeek merges every system message to the top

A decisive pair of runs made it plain. Warm the cache with a single system message holding persona plus memory at the head; then send the same content re-arranged as “persona at head, memory as a standalone system block after history” — 97% hits. The two layouts produce the same token stream server-side. DeepSeek's chat template collects every system-role message in the list and concatenates them at the front of the prompt. What you meant as tail injection arrives as head injection.

This is not an inference. The Jinja template in DeepSeek's open model repos (V3.1, V3.2-Exp and R1 agree) starts by iterating all messages and rendering the concatenated system contents right after the BOS token. We ran the same layout against Kimi as a control: 88% cache hits on memory-change rounds — Kimi's template preserves message order, so its tail really is a tail. The principle “volatile content goes last” survives; the trap is that whether a system role stays in place is a per-provider template detail. The portable answer is to inject on the user track: for both template families we verified — merging and order-preserving — it is a true tail.

Where the 94.3% came from

The autopsy on v1 was short. Both layouts shared a persona; group B ran after group A on the same account; nothing isolated the sessions. Group A had already warmed the provider's disk cache with prefixes containing each memory version, and group B — whose post-merge token stream nearly matched A's — collected hits for free, round after round. The 94.3% measured the previous experiment's body heat, not the layout.

Seven retrieval strategies, and a flatter answer than expected

The same rig answered a second question along the way: how much does retrieval strategy matter? At 47–82 stored memories, character-bigram BM25 injecting its top 8 hits scored 79% end-to-end — against 80% for oracle retrieval (relevant entries guaranteed present), a paired difference of 7:6 that no test would call significant. Against the no-memory control the same comparison is 59:1 (p<0.001). In plain words: get “is anything injected” right before touching “how it is retrieved” — at dozens of memories, plain BM25 is enough, and the question deserves a rerun only when stores reach the hundreds.

What actually made us sit up was the abstention probes. Ask about something never mentioned — height, birthday, salary — and under a companion persona DeepSeek almost never admits ignorance; it fabricates with conviction. Twelve probes: the no-memory control passed zero, oracle retrieval passed one. A persona line saying “if unsure, say you don't remember” changed nothing. What works is an explicit fence appended to the memory block — apart from these memories you know no other personal facts, do not invent them — plus keeping abstention probes in the regression suite. No retrieval algorithm substitutes for that sentence.

The three lines that reached the product

Foreverse's memory plugins (a fact store plus session summaries) shipped on this experiment's shape: injection rides the user track at the message-level tail, a true tail across the providers we verified and cache-friendly; retrieval is character-bigram BM25 with room to widen top-k; the injection block ends with the abstention fence. How the caching math itself works is in the earlier field test. The full design — leak detection, what remains unfinished (alias-level leak scanning, larger-store stress runs) — is documented, and the numbers will be updated as the reruns land.