The hidden cost of context: 6 billion tokens re-paid in 14 days
of history re-read through the cache in 14 days — 45 Claude Code sessions, zero spontaneous compaction observed. 96% of total volume is not new work: it is the same conversation, re-read on every API call. (Reference run of 2026-07-23, local artifact
compaction_rejeu.json, recomputable.)
The phenomenon
A coding-agent session resends its whole history with every message (stateless API).
Anthropic's cache makes that re-read ~10× cheaper, yet it still dominates the bill: in our
real data, 96% of counted tokens are cache_read. And nobody compacts:
across 14 days and 45 sessions, not one spontaneous compaction. The only sanctioned mechanism
that shrinks already-paid history is /compact — the open question is when
it pays off.
The counterfactual replay
We replayed the 45 sessions under a simple policy: "compact as soon as history weight exceeds θ" (first-order simulation: compaction cost = one re-read + summary; post-compaction context S0 = 12k, stated as an estimate).
| Threshold θ | Delta vs. actual history cost |
|---|---|
| 400,000 | −36.0% |
| 200,000 | −65.2% |
| 120,000 | −78.4% |
Published negative result: a per-session "learned" threshold does NOT beat the best fixed threshold under a pure token-cost objective (per-session oracle: +0.0% across 45/45 sessions) — the real optimum is bounded by quality, not arithmetic. We publish what does not work, too.
What Chaser shipped (measured)
- Cost-driven compaction advisor — on every prompt, YOUR session's real history weight is read; above the threshold, a quantified suggestion appears ("/compact would free ~416k per turn"). Causal effect measured through a 10% holdout (a fraction of eligible advice is silently withheld — both arms are journaled).
- Tier 0 chat — a "thanks" in a heavy session re-pays the whole history for nothing. Purely social messages (closed whitelist) are intercepted before sending and answered locally: 0 subscription tokens, the avoided turn is journaled. First production interception: 610,246 tokens avoided in a single message.
- Three-stage auto-compactor — lowered native auto-compact, a watcher that compacts dormant sessions through the official interface, and the advisor for active sessions.
- Per-request attribution — every prompt shows its cost split: fresh tokens vs. re-paid history (OpenTelemetry GenAI semantics), and the headline number is the "full-price equivalent" (cache read ×0.10, cache write ×1.25 — official weighting).
Prior art and limits
Prior art identified as of 2026-07-23: pre-send prompt compression, manual compaction
heuristics published in blogs. No public equivalent identified for the mechanisms above
at that date. Stated limits: the replay is a first-order simulation (quality loss not
modeled); how cache re-reads weigh into subscription limits is proprietary; every
token↔character conversion uses a ratio calibrated against the official
count_tokens endpoint.
Every figure in this report comes from a local, auditable journal (timestamped
JSONL) and can be recomputed with the replay script shipped with Chaser
(experiments/compaction_optimale/mesurer.py). Chaser is 100% local:
your transcripts never leave your machine.