Prompt Caching Guide: OpenAI, Anthropic, and DashScope Cost Optimization Strategies

Prompt caching lets you reuse previously processed input tokens so the provider skips redundant computation on repeated prefixes. The result is lower latency and cheaper input costs — up to 90% cheaper on some providers. Every major LLM API now supports some form of prompt caching, but the mechanisms, pricing multipliers, TTLs, and minimum token thresholds differ enough that choosing the wrong strategy can cost more than skipping caching entirely.

We wrote this guide because we route requests across OpenAI, Anthropic, and Alibaba Cloud Model Studio (DashScope) daily, and caching economics directly affect which provider-model combination delivers the best cost per successful request. This is the cross-provider caching reference we wanted when we first started optimizing input costs across our routing configuration.

Sources: OpenAI Prompt Caching docs, retrieved 2026-06-17; Anthropic Prompt Caching docs, retrieved 2026-06-17; DashScope Context Cache docs, retrieved 2026-06-17.

---

How prompt caching works in 30 seconds

When you send a prompt to an LLM API, the model tokenizes and processes every input token. If a large portion of your prompt — a system instruction, a code repository, a document — is identical across requests, the provider can cache the intermediate computation (key/value tensors from the attention layers) and reuse it. Subsequent requests that share the same prefix skip that computation, cutting both wall-clock latency and the effective token price.

The critical detail: caching is prefix-based. Only the beginning of your prompt can be cached. If the variable part (e.g., a user question) comes before the static part (e.g., a system prompt), caching will not trigger. Structure your prompts with static content first, dynamic content last.

OpenAI-compatible means a provider exposes a chat-completions endpoint whose request and response shape matches the OpenAI API contract closely enough that an unmodified OpenAI SDK call works against it after swapping three values: API key, base URL, and model name. The minimum surface in practice is POST /v1/chat/completions with messages, model, and an OpenAI-shaped streaming response.

---

OpenAI: automatic caching with extended retention

OpenAI's prompt cache is fully automatic — no code changes required. Any prompt of 1,024 tokens or longer is eligible. The system routes requests to servers that recently processed the same prefix, and matching prefixes are served from cache.

Key parameters:

• Minimum tokens: 1,024
• Mechanism: Automatic prefix matching, hash-based routing
• Retention policies:
  • In-memory: 5–10 minutes of inactivity, up to 1 hour max. Available on GPT-4.1 and newer.
  • Extended (24h): Up to 24 hours. Available on GPT-4.1, GPT-5, GPT-5.1 series, GPT-5.2, GPT-5.4, GPT-5.5. Default for non-ZDR organizations on supported models.
• Pricing discount: Cached input tokens are billed at the cached rate shown in OpenAI's pricing. For GPT-5.5: $0.50/MTok cached vs $5.00/MTok base input (90% discount). For GPT-5.4: $0.25/MTok cached vs $2.50/MTok base (90% discount).
• prompt_cache_key parameter: Allows you to influence routing by combining a user-provided key with the prefix hash, improving hit rates when many requests share long common prefixes. Keep each unique combination below ~15 RPM to avoid cache overflow.

Source: OpenAI Prompt Caching docs, retrieved 2026-06-17; OpenAI API Pricing, retrieved 2026-06-17.

Code example: OpenAI automatic caching

from openai import OpenAI

client = OpenAI()  # uses OPENAI_API_KEY env var

# Static system prompt — place it first for cache hits
SYSTEM_PROMPT = """You are an expert code reviewer. Analyze the following
repository for security vulnerabilities, performance issues, and code
style violations. The repository contents follow:

{repository_contents}
"""  # This would be >1024 tokens with real repo content

response = client.chat.completions.create(
    model="gpt-5.4",
    prompt_cache_retention="24h",  # request extended retention
    messages=[
        {"role": "system", "content": SYSTEM_PROMPT},
        {"role": "user", "content": "Focus on the authentication module."},
    ],
)

# Check cache performance
usage = response.usage
cached = usage.prompt_tokens_details.cached_tokens
total_input = usage.prompt_tokens
print(f"Cached: {cached}/{total_input} tokens ({cached/total_input*100:.0f}%)")

No special annotation required. On the second request with the same system prompt prefix, cached_tokens will be non-zero.

---

Anthropic: explicit cache_control with two TTL tiers

Anthropic offers developer-controlled caching through the cache_control field. You can choose between automatic caching (the system picks the optimal breakpoint) and explicit breakpoints (you mark exactly which content blocks to cache). There is a cost for creating the cache, but reads are heavily discounted.

Key parameters:

• Minimum tokens: 1,024 for explicit tool/message blocks; varies per content type
• Mechanism: Explicit cache_control annotations on content blocks, or top-level automatic mode
• TTL options:
  • 5-minute (ephemeral): Default. Each cache hit resets the 5-minute timer. Cache write costs 1.25× the base input price.
  • 1-hour: Opt-in. Cache write costs 2× the base input price. Available for Claude models on the Anthropic API and Amazon Bedrock.
• Cache read pricing: 10% of base input price (90% discount). For Claude Opus 4.8: $0.50/MTok read vs $5.00/MTok base.
• Maximum breakpoints: Up to 4 explicit cache_control markers per request.

Source: Anthropic Prompt Caching docs, retrieved 2026-06-17; Anthropic Pricing, retrieved 2026-06-17.

Code example: Anthropic explicit caching

import anthropic

client = anthropic.Anthropic()

# Automatic caching — system applies breakpoint to last cacheable block
response = client.messages.create(
    model="claude-sonnet-4-6",
    max_tokens=1024,
    cache_control={"type": "ephemeral"},  # top-level: auto mode
    system="You are an expert code reviewer. Analyze the following "
           "repository for security vulnerabilities, performance issues, "
           "and code style violations. [large repository contents here]",
    messages=[
        {"role": "user", "content": "Focus on the authentication module."},
    ],
)

print(response.usage)
# Shows: cache_creation_input_tokens, cache_read_input_tokens, input_tokens

For fine-grained control, place cache_control on individual content blocks:

response = client.messages.create(
    model="claude-sonnet-4-6",
    max_tokens=1024,
    system=[
        {
            "type": "text",
            "text": "[large static content — repository, docs, etc.]",
            "cache_control": {"type": "ephemeral"},  # explicit breakpoint
        }
    ],
    messages=[
        {"role": "user", "content": "What security issues exist?"},
    ],
)

When Anthropic caching costs more than it saves

Cache writes are not free. With the 5-minute TTL, you pay 1.25× the base input rate on cache creation. If your requests do not repeat within 5 minutes, you pay the write premium without getting any read discount. The break-even point: if fewer than ~15% of your requests hit the cache, you are paying more than the no-cache baseline. Monitor cache_creation_input_tokens vs cache_read_input_tokens to verify savings.

---

DashScope: explicit and implicit context caching

Alibaba Cloud Model Studio (DashScope) offers two caching modes. Both use the familiar cache_control annotation via the OpenAI-compatible API:

Explicit caching:

• Minimum tokens: 1,024
• TTL: 5 minutes (resets on hit)
• Cache write cost: 125% of standard input price
• Cache read cost: 10% of standard input price (90% discount)
• Max breakpoints: 4 per request
• Supported models: qwen3.7-max, qwen3.6-max-preview, qwen3-max, qwen3.7-plus, qwen3.6-plus, qwen3.5-plus, qwen-plus, qwen3.6-flash, qwen3.5-flash, qwen3-coder-plus, qwen3-vl-plus, deepseek-v3.2, kimi-k2.6, kimi-k2.5, glm-5.1, and more.

Implicit caching (automatic):

• Minimum tokens: 256
• TTL: Not guaranteed — system clears unused caches periodically
• Cache write cost: 100% of standard input price (no premium)
• Cache read cost: 20% of standard input price (80% discount)
• Always on: Cannot be disabled. Applied automatically to all requests.

Explicit and implicit caching are mutually exclusive per request — if you set cache_control, only explicit mode applies.

Source: DashScope Context Cache docs, retrieved 2026-06-17; DashScope Pricing, retrieved 2026-06-17.

Code example: DashScope explicit caching (OpenAI-compatible)

from openai import OpenAI

client = OpenAI(
    api_key="sk-xxx",  # DashScope API key
    base_url="https://dashscope.aliyuncs.com/compatible-mode/v1",
)

LONG_SYSTEM_CONTENT = "[large document, code, or context — must be ≥1024 tokens]"

response = client.chat.completions.create(
    model="qwen3.7-max",
    messages=[
        {
            "role": "system",
            "content": [
                {
                    "type": "text",
                    "text": LONG_SYSTEM_CONTENT,
                    "cache_control": {"type": "ephemeral"},
                }
            ],
        },
        {"role": "user", "content": "Summarize the key findings."},
    ],
)

# Check cache tokens
details = response.usage.prompt_tokens_details
print(f"Cache created: {details.cache_creation_input_tokens}")
print(f"Cache hit: {details.cached_tokens}")

The DashScope OpenAI-compatible endpoint returns cache_creation_input_tokens and cached_tokens in the same field structure as OpenAI — so switching between providers requires no response-parsing changes.

---

Cross-provider comparison

Key takeaways:

• OpenAI is the simplest: no code changes, automatic prefix detection, and the extended 24h retention on GPT-5+ models means cache hits persist across long development sessions.
• Anthropic gives the most control: explicit breakpoints let you target exactly what to cache, but the write premium means you need consistent request patterns to break even.
• DashScope is the most cost-efficient for high-hit-rate scenarios: the 10% read cost on explicit mode matches Anthropic, and the implicit 20% read cost applies with zero effort.

---

Routing strategies to maximize cache hits

When routing across multiple providers, cache affinity is critical. A request that hits provider A's cache but gets routed to provider B wastes the entire cache investment. Consider these strategies:

1. Provider-sticky routing for cacheable workloads

Pin conversations or workload types to a single provider. If your system prompt is 10K tokens, every provider switch forces a cold cache miss on the new provider. The cost of one cache miss can exceed the savings from cheaper per-token pricing on the alternative provider.

2. Prompt prefix ordering discipline

Place static content first, variable content last. This applies universally:

• System instructions → cached
• Few-shot examples → cached
• Tool definitions → cached
• Conversation history → partially cached (grows each turn)
• Current user message → not cached

3. Cache-aware model selection

When your application uses long system prompts (>10K tokens), models with extended cache retention become disproportionately valuable. A model at $5/MTok input with 24h cache retention can be cheaper per effective token than a model at $2/MTok with 5-minute cache that expires between requests.

4. Monitoring cache performance

Track these metrics per provider:

• Cache hit rate: cached_tokens / total_input_tokens
• Effective input cost: (uncached_tokens × base_rate + cached_tokens × cached_rate + write_tokens × write_rate) / total_tokens
• Cache ROI: Compare effective input cost against the no-cache baseline

---

Cost modeling: when caching saves money

Caching is not free. On Anthropic and DashScope (explicit mode), the cache write premium means every cache creation costs more than a standard input. You only save money if subsequent requests hit the cache often enough to amortize the write cost.

Break-even formula (Anthropic 5-minute ephemeral):

• Write cost: 1.25× base input
• Read cost: 0.10× base input
• Base cost (no cache): 1.00× base input

For N requests over the same prefix, total cost with caching = 1.25 + 0.10 × (N - 1). Total cost without caching = 1.00 × N. Caching breaks even when:

1.25 + 0.10(N - 1) < N → N > 1.28

So with as few as 2 requests hitting the cache, Anthropic explicit caching saves money. The constraint is the 5-minute TTL — if your requests are spaced more than 5 minutes apart, the cache expires and you pay the write premium again.

OpenAI automatic caching has no write premium, so every cache hit is pure savings. The only risk is low hit rates due to prefix divergence or cache eviction (in-memory: 5–10 min idle timeout).

DashScope implicit caching also has no write premium, so any cache hit saves 80%. It is the safest default mode.

---

TheRouter integration: routing considerations for cache-aware workloads

TheRouter routes OpenAI-compatible requests through configured providers and supports provider/model routing and fallback where the live product path supports it. For cache-sensitive workloads, keep these points in mind:

• Provider affinity: When a workload benefits from prompt caching, configure routing to prefer a consistent provider rather than round-robin across providers. Each provider switch resets the cache.
• Fallback cost awareness: If a primary provider is down and TheRouter falls back to an alternative, the first request on the fallback will be a cache miss. Factor this into your cost projections for high-availability configurations.
• Model aliasing: When DashScope and OpenAI both serve comparable models with caching support, the caching economics (write premium, TTL, read discount) may differ. Evaluate effective cost per request, not just per-token sticker price.

For more on fallback routing architecture, see our LLM API Fallback Routing Guide.

---

Production checklist

Before deploying prompt caching in production:

• Measure baseline costs — log prompt_tokens and completion_tokens for at least 100 representative requests without caching optimization
• Restructure prompts — move static content (system instructions, examples, tool definitions) to the front; keep variable content at the end
• Choose explicit vs automatic — use OpenAI automatic for simplicity; use Anthropic/DashScope explicit when you need guaranteed breakpoints
• Set retention policy — on OpenAI, opt into prompt_cache_retention: "24h" for long-lived sessions; on Anthropic, consider 1-hour TTL for batch workloads
• Monitor cache metrics — track cached_tokens, cache_creation_input_tokens, and effective cost per request
• Test cache invalidation — verify that prompt changes (system instructions, tool definitions) correctly invalidate stale caches
• Calculate ROI — confirm that your hit rate exceeds the break-even threshold for your chosen provider's write premium
• Review provider routing — ensure your routing configuration maintains provider affinity for cacheable workloads

---

FAQ

Q: Does prompt caching affect output quality? No. Caching only reuses the intermediate computation (key/value tensors) from input processing. The model generates new output for every request. Cached and uncached requests produce identical results.

Q: Can I cache tool definitions? Yes, on all three providers. Tool definitions are part of the prompt prefix and can be cached. On OpenAI, they are cached automatically. On Anthropic and DashScope, place them before the cache_control breakpoint.

Q: Is cached data shared between organizations? No. On OpenAI, caches are isolated per organization. On Anthropic, caches are scoped per API key. On DashScope, caches are scoped per workspace.

Q: What happens if I change one token in a cached prefix? The cache misses from that point forward. Prompt caching is exact prefix match — any change to the cached portion invalidates the match. The unchanged portion before the change point may still match on some providers.

Q: Which provider has the best caching for agentic workloads? Agentic workloads (multi-turn, tool-use-heavy) benefit most from extended TTLs because tool calls create round-trip delays. OpenAI's 24h extended retention is the strongest here. Anthropic's 5-minute default works if tool calls complete within that window. DashScope's 5-minute explicit cache also resets on hit, which suits rapid multi-turn exchanges.

For provider-specific setup, see our guides: Aliyun Bailian API Guide, DeepSeek API Guide, OpenAI-Compatible API Providers, and OpenAI to TheRouter Migration Guide.