Prompt caching explained: how cache reads cut LLM API costs

Most LLM apps send the same text again and again: a long system prompt, tool definitions, a knowledge-base document, the chat history. Prompt caching lets the provider keep that prefix warm and bill it at a much lower rate the next time. Done right, it is the single biggest cost saving available.

How it works

The model provider stores the processed state of your prompt's prefix. When the next request starts with exactly the same bytes, those tokens are read from cache instead of being processed again. Anything after the first difference is processed normally.

How it is billed on apillms

• Cache read — a small fraction of the input price (for example Claude Sonnet 5: about $0.046 per 1M vs $0.274 for normal input).
• Cache write (Claude) — the first time a prefix is stored, it is billed slightly above normal input.
• Fresh input and output — normal price.

On the chat and Responses endpoints apillms adds a per-user cache key when your request does not send one, so related requests are more likely to hit the cache without extra code. Hits are never guaranteed: the upstream provider decides. Cache prices for every model are on the models page.

Five rules for a high cache-hit rate

1. Stable content first. System prompt, tools, documents — then the changing user message last.
2. No timestamps or random IDs in the prefix. A single changed character early in the prompt breaks the cache for everything after it.
3. Keep tool definitions in a fixed order.
4. Append, don't rewrite, history. Summarizing old turns changes the prefix; do it rarely.
5. Reuse the session. Caches expire; bursts of related requests hit more often than scattered ones.

Cache prices by model

Prices update live from our API.

Which API format caches best?

For Claude, the Anthropic Messages format (/v1/messages, used by Claude Code and the Anthropic SDK) carries the cache markers end to end. Claude Code and Cursor add those markers for you. For other vendors the hit rate depends on the upstream provider, so measure it: the dashboard shows cached tokens on every request.

A quick example

messages = [
  {"role": "system", "content": LONG_STABLE_INSTRUCTIONS},   # cached
  {"role": "user", "content": KNOWLEDGE_BASE_DOC},            # cached
  {"role": "user", "content": question}                        # fresh
]

With a 20,000-token prefix and a 200-token question, once the prefix is cached about 99% of the input tokens of each later call are cache reads. On Claude Sonnet 5 that input then costs about $0.001 per call instead of $0.006, at today's prices.

How to check your hit rate

Every response reports cached tokens in its usage block (cache_read_input_tokens on the Messages API, prompt_tokens_details.cached_tokens on Chat Completions). Log it next to total input tokens. A low rate almost always means something early in the prompt changes between calls: a date, a request id, a shuffled tool list.

When caching does not help

• One-off prompts with no shared prefix.
• Prompts shorter than the provider's minimum cacheable length.
• Long pauses between calls, after which the cache has expired.

FAQ

Do I need to change my code to use caching on apillms?

Usually not. Keep the stable part of the prompt first; Claude Code and Cursor add cache markers themselves.

How long does the cache last?

The upstream provider decides; typically minutes. Frequent related requests keep it warm.

Is a cache hit guaranteed?

No. Check cached tokens in the usage block of each response to see your real hit rate.