Long-context cost guide
Calculate Context Window Cost
A large context window is a capacity limit, not a prepaid allowance. Estimate what every request actually sends: system instructions, tool definitions, documents, retrieved chunks, conversation history, the current message, and room for the answer.
Separate context capacity from token cost
First check whether the request fits: system and tools + documents + history + current message + reserved output must be no more than the model's context window. Reserving output space matters because a prompt that nearly fills the window can leave too little room for the answer.
Then estimate cost from the tokens processed on each API call, not from the model's maximum window. An application using 20,000 tokens in a one-million-token window normally pays for the request's measured token usage, subject to the provider's current pricing rules. Some providers or models apply different rates to long inputs, cached tokens, or other token classes, so confirm the tier before launch.
Count every input component
Include hidden-looking but billable context: the system prompt, tool names and schemas, retrieved RAG passages, images or documents after provider tokenization, earlier user and assistant messages, tool results, and any summary carried forward. Count against the exact model when possible because tokenization and media accounting differ.
For a single call, use: input cost = input tokens ÷ 1,000,000 × input price. Add output tokens ÷ 1,000,000 × output price. For a conversation or agent run, repeat that calculation for every call and sum the results.
Eight-turn conversation example
Assume each turn sends 3,000 fixed instruction and tool tokens, 8,000 retrieved tokens, a new 200-token user message, all earlier messages, and produces a 600-token answer. Across eight turns, the growing inputs total 112,000 input tokens and the answers total 4,800 output tokens.
At illustrative rates of $2 per million input tokens and $8 per million output tokens, one conversation costs $0.2624: $0.2240 for input plus $0.0384 for output. At 10,000 conversations per month, that is $2,624.00. These are illustrative rates, not a quote for any model.
Pricing only the eighth request would count 16,800 input tokens and 600 output tokens, or $0.0384. That misses the first seven calls and understates this conversation's model cost by about 85%. Multi-turn budgets must sum every request.
Choose what stays in the window
Longer context is useful when the model needs exact source material, earlier decisions, or recent tool evidence. It is wasteful when every request resends irrelevant history, entire documents for a narrow question, verbose tool payloads, or duplicated instructions.
Set a token budget for each bucket. Retrieve only relevant passages, trim tool results to required fields, summarize completed phases, and measure whether removing context changes answer quality. For stable repeated prefixes, use the prompt caching guide; caching can change price and latency, but cached tokens may still occupy context capacity.
Compare full context, retrieval, and compaction
Test at least three representative cases: the normal request, a high-percentile long request, and the largest request you will accept. Compare full-context prompting with retrieval-augmented generation and conversation summarization or provider-supported compaction. Record accuracy, latency, retries, input tokens, and output tokens rather than choosing only by maximum window size.
Verify model limits and measured usage
Context sizes, output limits, tokenizers, and pricing rules change. Check the current model documentation and usage response before relying on a forecast. See the official OpenAI model guidance, Claude context window guide, and Gemini long-context guide.