RPM and TPM capacity calculator
AI API Rate Limit Calculator: RPM, TPM, and Safe Throughput
A workload can be affordable but still fail at launch. Calculate how many requests per minute fit your request, input-token, and output-token limits, then reserve headroom for bursts and retries.
Collect the limits that apply to your workload
Copy the active limits for the exact model, project, organization, and service tier you will use. Providers can enforce requests per minute (RPM), input tokens per minute (input TPM), output tokens per minute (output TPM), daily quotas, concurrent requests, or spend limits. A short classification job may hit RPM first, while a document summarizer, RAG answer, or code review may hit TPM first.
Calculate the narrowest throughput limit
Keep input and output tokens separate when the provider exposes separate buckets. For one model call, calculate each ceiling and use the smallest result:
RPM capacity = RPM limit
Input capacity = input TPM limit ÷ input tokens per request
Output capacity = output TPM limit ÷ expected output tokens per request
Safe requests per minute = smallest capacity × target utilization
A target utilization such as 80% leaves 20% headroom; it does not guarantee zero throttling. Use a lower target for spiky traffic or high-variance document sizes. If a provider uses one combined token bucket, divide that TPM limit by the token amount that its documentation says counts toward the limit.
Worked capacity example
Suppose a deployment allows 60 RPM, 300,000 input TPM, and 30,000 output TPM. One request uses 12,000 input tokens and 800 output tokens.
- RPM capacity: 60 requests per minute.
- Input capacity: 300,000 ÷ 12,000 = 25 requests per minute.
- Output capacity: 30,000 ÷ 800 = 37.5 requests per minute.
Input TPM is the bottleneck. At an 80% utilization target, plan for about 20 requests per minute, not 60. If one user action makes two model calls, this supports about 10 completed user actions per minute before retry overhead.
Work backward from forecast traffic
To size the limit you need, multiply peak model calls per minute by the tokens in each call, then divide by your utilization target. Eight calls per minute at 12,000 input tokens and 800 output tokens need at least 120,000 input TPM, 8,000 output TPM, and 10 RPM at an 80% target. Use p90 or p95 request sizes as a second scenario so a few large files do not consume the entire minute.
Include every model call and retry
Count model calls, not just visible user actions. An agent may plan, call a tool, inspect the result, and answer in four separate requests. A chunked PDF job may make ten map calls and one reduce call. Estimate peak-minute capacity for each model pool separately, then add a bounded retry allowance based on load-test evidence.
Do not immediately replay every 429 response. First distinguish a temporary rate limit from a credit, spend, or account-usage error. For temporary throttling, honor a valid Retry-After header or use capped exponential backoff with jitter. OpenAI notes that unsuccessful requests can still contribute to per-minute limits, so an unbounded retry loop can extend the incident.
Why monthly averages miss burst failures
A workload averaging 600 requests per hour may arrive as 100 requests in one minute and almost none in the next nine. Providers may also enforce a displayed minute limit over shorter intervals. Test a realistic burst, cap worker concurrency, and queue excess work instead of assuming the hourly average will pass.
Separate user traffic from background jobs
Batch imports, nightly summaries, embedding jobs, and re-indexing can compete with user-facing requests when they share a limit pool. Schedule background work outside peak usage, cap concurrency, and reserve capacity for interactive actions. Some providers give batch processing a separate enqueued-token or concurrency limit; verify the current rules before moving work. Compare eligible asynchronous jobs with the Batch API cost and capacity guide.
Reduce bursts before upgrading limits
Trim oversized prompts, split large documents into queued chunks, cache stable prefixes, deduplicate repeated context, and use smaller models for routing or extraction. These changes reduce both cost and rate-limit pressure. For repeated prefixes, compare the result with the prompt caching cost guide.
Validate the estimate with real responses
After a representative load test, compare predicted RPM and TPM with request timestamps, token usage, 429 error details, retries, queue depth, and p95 latency. Where available, record provider rate-limit response headers and reset times. Keep model and project identifiers with the observation because limits can differ across pools.
Limits and enforcement can change. Confirm them in the provider console and current official documentation: OpenAI rate limits, OpenAI 429 troubleshooting, Claude rate limits, and Gemini rate limits (checked August 30, 2026).