The Cache Is the Bill

01The bill you don't see

Every time a coding agent takes a step (reads a file, runs a test, answers you) it sends the model the entire conversation so far. Not a summary: all of it. The system prompt, every tool definition, every file it has read, every reply it has written. A session that has been running for an hour can easily hold half a million tokens, and the next tool call sends all half million again.

At the normal input price that would be ruinous. Claude Opus 5.5 charges $4 per million input tokens, so resending 500k tokens costs about $2 per step, and an agent takes hundreds of steps. What makes long sessions affordable is prompt caching: the provider keeps the work it already did on that prefix and charges a small fraction to reuse it. With the cache warm, the same step costs about 12 cents.

So in a long session, the number that decides your bill isn't the headline input price. It's the cache: whether it is warm, how long it stays warm, what it costs to write, and what breaks it. This article builds that picture from the inside out, then simulates a working day so you can see which habits cost money and which don't.

02What is actually cached

A model handles a request in two phases (my book covers both in Chapter 7). In prefill it reads the whole prompt at once: every token passes through every layer, and at every layer each token leaves behind two vectors, a key and a value, that later tokens use to look back at it. In decode it generates the reply one token at a time, and each new token consults all of those stored keys and values. That store is the KV cache.

Within one request, the KV cache is simply how decoding works. Prompt caching is the provider deciding to keep it after the request ends. If the next request starts with exactly the same tokens, the provider can load the stored keys and values instead of running prefill over that prefix again. Only the new tokens at the end need computing.

How big is that store? Anthropic doesn't publish Claude's architecture unknown, but open models give the order of magnitude. Llama 3.1 405B has 126 layers and keeps 8 key-value heads of 128 numbers each, at 2 bytes per number public. Per token that is:

A single long conversation's cache can be bigger than the 80 GB of memory on one H100. Keeping it around is not free: it has to live somewhere, probably in a tier of cheaper memory or storage, and be moved back to the accelerators on a hit inferred. That explains the shape of the prices below. A write costs more than normal input, because you pay for prefill and for storing the result. A read costs much less, because prefill is skipped: for a model with N parameters it costs about 2N operations per token (Chapter 4), and a hit replaces that with moving bytes. And entries expire, because memory is finite inferred.

03The rules the API plays by

Whatever happens inside, what you're billed for follows a small set of documented rules public. They're worth knowing exactly, because every cost effect in this article follows from them.

1. It is a prefix match. The prompt is laid out as tools, then system prompt, then messages. A cached entry is keyed by the exact tokens up to a point. Change one byte anywhere and everything after that byte is a miss.
2. Entries exist only at breakpoints. A request marks up to 4 positions with cache_control, or turns on automatic caching, which puts the mark at the end of the conversation and moves it forward every turn. The cache stores the prefix up to each mark. Nothing in between is separately reusable.
3. Lookup walks back a limited distance. Each breakpoint searches backwards at most 20 content blocks for an existing entry. A turn that appends more than that (long sequences of tool calls, many images) can miss an entry that is still alive.
4. Entries live 5 minutes, or 1 hour if you ask. Every read resets the timer, for free. The timer runs from the start of the request that last touched the entry, so a 4-minute generation leaves about 1 minute before a 5-minute entry expires.
5. Each input token is billed once, at one of three prices. The response's usage reports cache_read_input_tokens, cache_creation_input_tokens (writes) and input_tokens (the uncached remainder after the last breakpoint). The three add up to the prompt size. A write replaces the input price; it doesn't add to it.
6. Caches are scoped. They are separate per model and, on the Claude API, per workspace. Switching models mid-session starts from zero.
7. A new entry is readable once its request starts streaming. Several requests fired in parallel with the same new prefix all pay to write it.
8. Short prompts don't cache. The minimum on the Opus 5 family is 512 tokens; below it, nothing is stored.

Note the read price. Most Claude models charge 0.1× input for a cache read; Opus 5.5 lists $0.20 against a $4 input price, which is 0.05×. That halves the cost of re-reading, and it doubles the ratio between a cold write and a warm read. Keep that ratio in mind: on a 5-minute cache, a cold write costs 25 times what a warm read of the same tokens costs; on a 1-hour cache, 40 times. Most of this article is consequences of those two numbers.

04Anatomy of one request

With the rules in place, one agent request splits cleanly into three parts:

• everything the previous request already sent: a cache read, if the cache is warm;
• everything new since then (the previous reply, fresh tool results, your message): a cache write, so the next request can read it;
• the reply, including the model's thinking: output.

The previous reply is billed three times over its life: once as output when it is generated, once as a write when it comes back as input, and as a read on every request after that. That isn't double billing; the token does three different jobs.

Two regimes show up. In a small context, output dominates: the reply and its thinking, at $20 per million, cost more than everything read. In a large context, re-reading dominates, even at five cents on the dollar, because there's so much of it. Agent sessions spend most of their life in the second regime.

05Every request re-reads everything

Because each request re-reads the whole prefix, the cost of a request grows in a straight line with the context. And because the context itself grows with every request, the total cost of a session grows with the square of its length:

Here g is how much the context grows per request (new input plus the previous reply) and o is the output per request. With a 25k-token start, 2.6k tokens of growth per request and a warm cache, 100 requests cost about $5.71. Doubling to 200 requests roughly triples it, to $16.48. Without caching, those 200 requests would cost $231.

The top chart shows the most important fact in this article: a cold request is more expensive than no caching at all. When the cache is cold, the harness still marks a breakpoint, so the whole prefix is billed as a write at 1.25× or 2× the input price. At 200k tokens the next request costs $0.06 warm, $0.82 with no caching, $1.02 cold on a 5-minute cache and $1.63 cold on a 1-hour cache. Caching is a bet that you'll come back soon. When you lose it, you pay a premium.

06When the cache goes cold

The cache goes cold for two kinds of reasons. The first is time: the gap from the start of one request to the start of the next exceeds the lifetime (5 minutes by default). In an agent loop that happens when you step away, when you read a long diff before replying, or when a single step (a long test run, a slow tool) takes longer than the lifetime.

The second is a change to the prefix. The docs list the usual suspects public:

• switching models: caches are per model, so everything is rewritten;
• adding, removing or reordering tools, which sit at the very start of the prompt;
• editing the system prompt, for example by putting the date or a changing setting in it;
• changing thinking or effort settings, which invalidates at least the message history;
• editing or deleting earlier messages, which is what history rewriting and some compaction schemes do.

Either way the price of a cold request is the same: every token of the prefix that isn't still cached is written again. On Opus 5.5 that is 25 times a warm read on the 5-minute cache. A cold request on a 300k-token session costs about $1.44 more than the warm one would have. That's the price of a coffee each time you take a coffee break.

07Ping or pay

If a read resets the timer, you can keep a cache alive on purpose: while you're away, resend the last request shortly before the entry would expire. The API supports this directly public. A request with max_tokens: 0 runs prefill, generates nothing, and bills only the cache read. That is a keep-alive ping.

Each ping costs one read of the context. Letting the cache expire costs one extra write of the context, minus the read you'd have paid anyway. The context size cancels out of the comparison:

So on Opus 5.5, if you'll be back within about an hour and three-quarters, keeping the 5-minute cache alive is cheaper than letting it go cold, whatever the size of the session. Beyond that, let it expire. The break-even is this generous because the read is so cheap: at the usual 0.1× read price it would be 11.5 pings, under an hour.

Two cautions. First, a keep-alive has to resend the identical last request, with the same model, tools, thinking and effort settings, and with streaming off (the API rejects max_tokens: 0 with streaming). That is a job for the harness, not for you. Second, typing "ping" into the agent is not a keep-alive. It adds a new user message, the model answers it, the answer and its thinking are billed as output at $20 per million, and both join the context forever inferred. It does keep the cache warm, but at several times the price of a real keep-alive.

08Five minutes or an hour?

The 1-hour lifetime looks like the obvious fix for gaps: pay a bit more and the cache survives your meetings. The catch is what you pay more on. The 1-hour write price applies to every write, and in an agent session every token is written once, as it first joins the context. On Opus 5.5 that's $3 more per million tokens written: a session that grows to 600k tokens pays about $1.80 extra, whether or not it ever has a gap.

The docs give a simple guide, by the start-to-start gap between requests that share the prefix public:

For models with a cheap read, the docs add that keep-alive on the 5-minute cache usually beats the 1-hour cache, and Opus 5.5's read is cheap. The simulator in section 11 bears that out on most working patterns: the 1-hour cache wins only when medium gaps are frequent and nothing sends keep-alives.

09Continue, or start fresh?

A big context is a tax on every request. At 400k tokens, each request pays 8 cents just to re-read the session before doing anything new. A fresh session that starts with the system prompt and a few thousand tokens of hand-off notes re-reads for less than a cent. Starting fresh costs one write of the notes, plus the system prompt if that has gone cold too.

So why not restart constantly? Because the dollars aren't the whole cost. A fresh session has lost everything the old one knew but the notes didn't capture. It will re-read files and re-run commands to rediscover it, which costs tokens this chart doesn't count, and it may simply do worse work. The practical rule is to restart at natural boundaries: when a task is done, when the context is full of material that no longer matters, and above all after a long break. After a long break the cache is cold anyway: continuing a 400k-token session means rewriting all 400k at the write price, $2.00, while a fresh session costs a few cents. Resuming a big session after lunch is the single most expensive moment to continue it.

Compaction, which many harnesses do automatically as the context fills up, is a restart built into the harness. One request reads everything and writes a summary as output; the session then continues from the system prompt plus the summary. It costs one full read and a long output, and resets the per-request tax.

10Forks and parallel agents

Caches belong to prefixes, not to sessions. Fork a session into two branches and they share every cache entry up to the fork point. Whenever either branch reads a shared entry, it refreshes it for both. The newer entries each branch writes after the fork are its own, and only its own requests keep them alive.

Take a session on a 1-hour cache that has finished its second turn. The latest entry, C2, covers everything up to your second message. You fork it. One minute later, branch SS1 sends U3: it reads C2 and writes C3a. At T+50, branch SS2 sends U4: it reads C2, which resets C2's timer to T+110, and writes C3b. At T+75, SS1 sends U5. Its own entry C3a expired at T+61, but C2 is still alive because SS2 read it. So SS1 reads C2 and rewrites only the part after the fork. Without SS2, C2 would have expired at T+61 too, and SS1 would have rewritten everything.

Two more rules matter when you run several agents at once. Parallel requests don't share a write in progress: an entry becomes readable only once its request starts streaming, so ten sub-agents launched at the same instant over the same new prefix all pay the write. Launch one, wait for its first token, then launch the rest. And sub-agents with a different system prompt or tool set share nothing with the parent, because their prompts diverge at the first byte. Where the sub-agent can reuse the parent's exact prefix and only append to it, it inherits the parent's cache.

11A day of agent work

Everything so far, put together. The simulator below plays out a working day: blocks of work, during which the agent sends a request every few seconds, separated by time away. It prices each request on Opus 5.5 according to the rules above, for up to five strategies at once:

• 5-minute cache: the default; gaps over 5 minutes go cold.
• 1-hour cache: survives gaps up to an hour, pays 2× on every write.
• Keep-alive, on either lifetime: pings while you're away, up to a limit you set.
• Fresh session after long breaks: after a break at least as long as you choose, start over with the system prompt and hand-off notes.

With the defaults, the "meetings in between" day shows every effect at once:

• The 5-minute cache costs $24.00. Three breaks go cold, and those three requests alone cost $3.91 more than warm ones would have.
• Keep-alive brings it to $22.16: 34 pings for $2.07 remove every cold miss.
• The 1-hour cache is the most expensive at $25.08. It pays the 2× write all day, and the 75-minute lunch break still goes cold.
• A fresh session after lunch is the cheapest by far at $16.27, because the whole afternoon runs on a small context instead of re-reading 400k tokens per request. That comes with the quality caveat from section 9.

Switch to Resume next morning, and keep-alive capped at 2 hours becomes a waste: it pays for 26 pings and then goes cold anyway. Switch to Stop and go, with short breaks only, and both alternatives beat the default: keep-alive comes in 30% cheaper, the 1-hour cache 24% cheaper. Change the settings to match how you actually work; the ranking changes more than you might expect.

12Field guide

What to do with all this, day to day:

1. Keep the prefix still. Don't switch models or change effort in the middle of a session, don't let tools come and go, and don't edit earlier messages. Each of these rewrites the session at the write price. If you need a different model for a side task, use a sub-agent and leave the main loop alone.
2. Know your gaps. Under 5 minutes, nothing needed. From 5 minutes to about 1 h 45, a keep-alive is cheaper than a cold miss on Opus 5.5, if your harness can send one. Longer than that, let it expire.
3. Treat a cold resume of a big session as a decision. It rewrites the whole context at more than the uncached price. After a long break, a fresh session with good hand-off notes is usually cheaper and often no worse.
4. Remember that a big context is a per-request tax. Start fresh at task boundaries. Compaction helps, and so do notes that let a new session skip rediscovery.
5. Choose the 1-hour cache deliberately. It's worth it when 5-to-60-minute gaps are frequent and keep-alive isn't available. Otherwise it's a 60% surcharge on every write.
6. Launch parallel agents in a staggered way, over byte-identical shared prefixes, so they read one entry instead of each writing their own.
7. Measure. The usage fields are the only ground truth. A healthy session reads nearly everything and writes only what each request added. A request whose writes are close to the whole context has gone cold, and you now know why.

13What this article can't tell you

• What your harness does. Which lifetime Claude Code, Pi or any other agent uses, where it places breakpoints, and whether it sends keep-alives varies between tools and versions unknown. The usage fields in its logs will tell you.
• Early eviction. The docs describe lifetimes and refreshes; they don't say whether an entry can be dropped before its lifetime ends under load unknown. The simulator assumes it can't.
• The exact write prices for Opus 5.5. The docs give the multipliers in general and the read price for this model. $5 and $8 apply those multipliers inferred.
• Subscriptions. On a subscription plan you don't see these dollars. How a plan counts cached tokens against its limits isn't in the documentation this article used unknown. On the API, cache reads don't count toward input-token rate limits on most models public.
• Real sessions are messier. Tool results vary from a few tokens to tens of thousands, and requests come in bursts, not every 45 seconds. The simulator is a model for comparing habits, not an invoice.

Sources

• Anthropic, Prompt caching and Pricing, Claude API documentation. Figures in this article were taken from the documentation as bundled with Claude Code (version 2.1.282) on 28 September 2026: the prompt-caching guide (breakpoints, lifetimes, refresh, 20-block lookback, keep-alive with max_tokens: 0, invalidation rules) and the model guide (Claude Opus 5.5: $4 / $20, cache reads $0.20, 1M context).
• Meta, The Llama 3 Herd of Models (2024), for the 405B model's shapes.
• Frontier LLMs, From the Inside: Chapter 7 (prefill, decode, the KV cache), Chapter 19 (serving at scale), Chapter 24 (cost in production).