Cost optimization guide
Prompt Caching Cost Calculator Guide
Prompt caching can reduce the price of repeated input, but it does not discount new context or output tokens. Estimate those parts separately before forecasting savings.
What prompt caching changes
Many AI workflows resend a stable prefix: system instructions, tool definitions, examples, reference documents, or earlier conversation history. A cache hit lets the provider reuse some of that processed prefix. The changing part of the prompt and the generated answer are still billed under their applicable rates.
Collect four usage numbers
- Repeated input tokens: the stable prefix eligible for reuse.
- New input tokens: the user question, fresh retrieval results, and other changing context.
- Output tokens: the answer generated for each request.
- Cache hit rate: the share of requests that actually reuse the prefix.
Also record requests per day and any provider-specific cache-write or storage charge. Do not assume every repeated token becomes a cache hit; minimum prefix lengths, expiration, model choice, and prompt changes can affect eligibility.
The cache-aware cost formula
Monthly planning formula
Cached input = repeated input tokens × requests × cache hit rate
Standard input = new input tokens × requests + repeated input tokens × requests × (1 − cache hit rate)
Total = cached input cost + standard input cost + output cost + cache-write or storage cost
Use the calculator to estimate the standard input and output baseline. Then apply the current cached-input rate only to the repeated tokens expected to hit the cache. This avoids the common mistake of discounting the entire request.
Example without a fixed model price
Suppose each request contains 20,000 repeated input tokens, 2,000 new input tokens, and 1,000 output tokens. At 1,000 requests per month with an 80% cache hit rate, 16 million input tokens use the cached-input rate. The remaining 6 million input tokens use the standard-input rate: 4 million from cache misses plus 2 million of new context. All 1 million output tokens use the output rate.
Improve the hit rate
Place stable content before changing content, keep the stable prefix byte-for-byte consistent when the API requires it, and avoid inserting timestamps or request IDs near the beginning. Group similar requests close enough to fit the provider's cache lifetime, but do not send artificial traffic merely to keep a cache warm.
Verify with actual usage
Run a representative test and inspect the provider's usage fields for cached tokens, cache reads, or cache writes. Compare the observed hit rate and billed cost with the estimate. Provider behavior and pricing change, so confirm details in the official OpenAI prompt caching guide, Claude prompt caching guide, and Gemini context caching guide.