Prompt caching lets a model provider reuse an unchanged prompt prefix (system/developer instructions, tool definitions, other stable context) across turns instead of reprocessing it every request. This cuts token cost and latency on long-running sessions with repeated context.

Apus normalizes provider usage into cacheRead and cacheWrite wherever the upstream API exposes those counters. Usage summaries (/status and similar) fall back to the last transcript usage entry when the live session snapshot lacks cache counters; a nonzero live value always wins over the fallback.

Provider references:

Keep model settings stable

Prompt-cache reuse depends on provider request configuration as well as prompt text. Changing the model always starts a different cache lineage. Changing the thinking or reasoning level can also invalidate reuse even when the prompt and model stay the same. Supported native OpenAI Responses requests preserve the original effort and append turn-scoped configuration controls, including after transport expiry or a Gateway restart when saved replay metadata and history still match. See OpenAI reasoning changes. Other OpenAI models or incompatible modes can still reprocess the full prefix. Anthropic likewise documents cache invalidation when its thinking budget, effort, or mode changes.

If cache continuity matters, choose the model and thinking level when creating the session and keep both stable. Start a new session for a planned change. Invalidating reuse means the next request misses that cached state; it does not necessarily delete the provider's older cache entry before its normal expiry.

Primary knobs

Worker turns

Gateway-proxied worker inference uses the same OpenAI cache-key derivation as local turns: an explicit Gateway key takes precedence; otherwise the key combines the session ID with the authoritative transcript's reset and compaction boundary count. The Gateway retains these facts with the admitted turn, rather than accepting a worker-provided cache key or deriving one from trimmed replay history.

Delivered worker skills use stable session-scoped, content-addressed paths and a deterministic catalog order. Unchanged skills therefore keep the system prompt prefix stable between turns. Skill refreshes still deliver current verified bytes; a content change intentionally changes that skill's path. Worker and local prompts remain different in scope: workers load bounded workspace AGENTS.md and the Gateway's supplied instructions with their restricted tool set. Moving a session between runtimes or workspaces can still invalidate its cached prefix.

cacheRetention

Values: "none" | "short" | "long". Configurable as a global default, per model, and per agent. "standard" is not an alias; use "short" for the provider's default cache window. Invalid values are ignored with a warning.

agents:
  defaults:
    params:
      cacheRetention: "long" # none | short | long
    models:
      "anthropic/claude-opus-4-6":
        params:
          cacheRetention: "short" # overrides the global default for this model
  list:
    - id: "alerts"
      params:
        cacheRetention: "none" # overrides both defaults for this agent

Merge order (later wins):

  1. agents.defaults.params - global default for all models
  2. agents.defaults.models["provider/model"].params - per-model override
  3. agents.entries.*.models["provider/model"].params - agent-specific per-model override
  4. agents.entries.*.params - agent-wide override, matched by agent id

Source: src/agent/agents/embedded-agent-runner/extra-params.ts (resolveExtraParams).

contextPruning.mode: "cache-ttl"

Prunes old tool-result context after the cache TTL window elapses, so a post-idle request does not re-cache oversized history.

agents:
  defaults:
    contextPruning:
      mode: "cache-ttl"
      ttl: "1h"

See Session pruning for full behavior.

Heartbeat keep-warm

Heartbeat can keep cache windows warm and reduce repeated cache writes after idle gaps. Configurable globally (agents.defaults.heartbeat) or per agent (agents.entries.*.heartbeat).

agents:
  defaults:
    heartbeat:
      every: "55m"

Provider behavior

Anthropic (direct API and Vertex AI)

Source: packages/ai/src/transports/anthropic-payload-policy.ts (resolveAnthropicEphemeralCacheControl, isLongTtlEligibleEndpoint).

DeepInfra

For anthropic/* models, the managed and SDK Chat Completions paths use the shared marker layout. cacheRetention: "none" disables these markers. By default, both "short" and "long" use ephemeral markers without a TTL override; Apus does not assume one-hour support for this route.

Model Studio / DashScope (Qwen)

Both Chat Completions builders enable the shared marker layout on native or default Model Studio / DashScope routes through compat.cacheControlFormat: "anthropic". Explicit model compat settings take precedence over detected defaults. Custom proxy endpoints receive no automatic format default; set compat.cacheControlFormat: "anthropic" explicitly only when the proxy supports these markers.

Model Studio includes tool definitions in the system cache and ignores markers on tools themselves, so Apus omits that marker on detected native routes. Qwen3.5 and later support message-level checkpoints only: splitting the stable and volatile system content into blocks does not guarantee independent reuse of the stable block. See Model Studio explicit cache guidance.

Explicit cacheRetention values reach this transport without enabling compat.supportsPromptCacheKey; leave that flag unset because this route does not need OpenAI's prompt_cache_key or prompt_cache_retention fields. With no explicit retention, the transport keeps its "short" default. Model Studio uses a five-minute explicit cache window, so "long" keeps ephemeral markers without requesting a one-hour TTL.

To disable Apus's explicit markers, set cacheRetention: "none". The current compat.cacheControlFormat schema accepts only "anthropic", not a disable value; omitting it uses the detected default. Alibaba's automatic implicit caching is separate and cannot be disabled. Supported models, minimum prompt lengths, and cache billing are described in Model Studio context caching.

OpenAI (direct API)

Amazon Bedrock

AWS's prompt caching guide and model cards for Micro, Lite, Pro, Premier, and Nova 2 Lite list these limits: a 1K-token minimum, four checkpoints, and at most 20K cached tokens for Nova. Provider token limits still determine whether a checkpoint is cached.

Nova explicit caching has not been live-verified against AWS by Apus maintainers yet. Live AWS acceptance proof remains a gap until a maintainer with Bedrock access runs it.

OpenRouter

For openrouter/anthropic/* model refs, both Chat Completions builders apply the shared marker layout, but only when the request still targets a verified OpenRouter route (openrouter on its default endpoint, or any provider/base URL that resolves to openrouter.ai). Repointing the model at an arbitrary OpenAI-compatible proxy URL stops automatic marker injection. cacheRetention: "long" requests ttl: "1h" on these verified routes; "none" disables markers. See OpenRouter prompt caching.

contextPruning.mode: "cache-ttl" is allowed for openrouter/anthropic/*, openrouter/deepseek/*, openrouter/moonshot/*, openrouter/moonshotai/*, openrouter/x-ai/*, openrouter/zai/*, openrouter/google/gemini-2.5*, and openrouter/google/gemini-3* model refs, because these routes handle provider-side prompt caching without needing Apus's injected markers. Older Gemini models and other google/* models (for example Gemma) are excluded because OpenRouter caches them only with explicit cache_control markers, if at all.

Source: extensions/openrouter/index.ts (OPENROUTER_CACHE_TTL_MODEL_FAMILY).

DeepSeek cache construction on OpenRouter is best-effort and can take a few seconds; an immediate follow-up request may still show cached_tokens: 0. Verify with a repeated same-prefix request after a short delay, using usage.prompt_tokens_details.cached_tokens as the cache-hit signal.

Google Gemini (direct API)

Source: src/agent/agents/embedded-agent-runner/google-prompt-cache.ts.

CLI-harness providers (Claude Code, Gemini CLI)

CLI backends that emit JSONL usage events (jsonlDialect: "claude-stream-json" or "gemini-stream-json") go through a shared usage parser that recognizes several field-name variants, including a plain cached counter mapped to cacheRead. When the CLI's JSON payload omits a direct input-token field, Apus derives it as input_tokens - cached. This is usage normalization only - it does not create Anthropic/OpenAI-style prompt-cache markers for these CLI-driven models.

Claude Code has no Apus-controlled cache_control breakpoint on --append-system-prompt-file, so Apus keeps its complete system prompt in that transport. When the bounded version check finds Claude Code 2.1.98 or newer, bundled claude-cli also passes --exclude-dynamic-system-prompt-sections. The first CLI execution or direct Anthropic OAuth request starts the shared check; concurrent executions reuse it, and API catalog discovery does not start it. That Claude Code flag moves only Claude's own per-machine cwd, environment, memory-path, and Git-status sections out of its native system prompt; an older, unknown, or failed check keeps the established argv. cacheRetention still has no effect on this path.

One-shot helper runs dispatched through a CLI backend, such as Active Memory recall on claude-cli, get a new session key on every run. Those runs carry the Runtime facts line (agent, session, model, channel) in their only user turn instead of the system prompt, so repeated recalls of one agent send a byte-identical system prompt and can reuse Claude's prompt cache. Normal CLI turns keep the Runtime line in the system prompt.

Source: src/agent/agents/cli-output.ts (toCliUsage).

Other providers

If a provider does not support any of the above cache modes, cacheRetention has no effect.

Chat Completions cache markers

OpenAI-compatible routes with compat.cacheControlFormat: "anthropic" share one marker policy across the managed transport, SDK builder, and provider wrappers:

The layout normally uses three markers and stays within the four-breakpoint budget. cacheRetention: "none" emits none. A conversation checkpoint covers all preceding tools, system content, and messages, so changing the volatile system suffix still invalidates that later checkpoint; the earlier stable-system checkpoint remains reusable where the backend supports block-level caching. Backend token minimums and cache lifetimes still apply.

With compat.requiresStringContent: true, managed requests keep message content as strings and omit message-block markers, including through provider wrappers. The tool-definition marker remains where supported.

Detected defaults request ttl: "1h" only on verified OpenRouter routes. For a custom endpoint that supports one-hour Anthropic caching, explicitly set compat.cacheControlFormat: "anthropic", compat.supportsLongCacheRetention: true, and cacheRetention: "long" to send that TTL. Omitting the capability override keeps custom-endpoint markers without a TTL; setting it to false disables the one-hour TTL even on OpenRouter.

System-prompt cache boundary

Apus splits the system prompt into a stable prefix and a volatile suffix at an internal cache-prefix boundary. Content above the boundary (tool definitions, skills metadata, workspace files) is ordered to stay byte-identical across turns. Content below the boundary (for example runtime timestamps and other per-turn metadata) can change without invalidating the cached prefix.

Key design choices:

If you see unexpected cacheWrite spikes after a config or workspace change, check whether the change lands above or below the cache boundary. Moving volatile content below the boundary (or stabilizing it) usually resolves the issue.

Chat Completions routes without an explicit message-cache breakpoint move the bounded Runtime facts line to the first emitted user message. This keeps session identifiers behind the system-and-tools prefix on compatible local servers. The line stays on that first message during follow-ups. Behavioral instructions, including hook additions, permission notices, and Git coauthor guidance, retain their system/developer role. Routes with explicit message breakpoints keep their existing system layout. Current-turn Runtime Context snapshots still use their separate transient carrier; they do not become permanent first-message context.

Apus cache-stability guards

Tuning patterns

Mixed traffic (recommended default)

Keep a long-lived baseline on your main agent, disable caching on bursty notifier agents:

agents:
  defaults:
    model:
      primary: "anthropic/claude-opus-4-6"
    models:
      "anthropic/claude-opus-4-6":
        params:
          cacheRetention: "long"
  list:
    - id: "research"
      default: true
      heartbeat:
        every: "55m"
    - id: "alerts"
      params:
        cacheRetention: "none"

Cost-first baseline

Live regression tests

Apus runs one combined live cache regression gate covering repeated prefixes, tool turns, image turns, MCP-style tool transcripts, and an Anthropic no-cache control.

Run it with:

APUS_LIVE_TEST=1 APUS_LIVE_CACHE_TEST=1 pnpm test:live:cache

This command selects the existing cache regression test through the shared live runner. Set APUS_VITEST_RUNTIME=bun to run its provider calls on Bun; the default is Node. Provider usage summaries and advisory warnings remain visible, and hard regression floors fail the test.

The baseline file stores the most recently observed live numbers plus the provider-specific regression floors the test checks against. Each run uses fresh per-run session IDs and prompt namespaces so previous cache state does not pollute the current sample. Anthropic and OpenAI use different enforcement: an Anthropic floor miss is a hard regression (test fails), while an OpenAI floor miss is watch-only (recorded as a warning, does not fail the run). They do not share a single cross-provider threshold.

Claude CLI prompt reuse has a separate Docker lane because it exercises Claude Code's native session transport rather than the direct Anthropic API. After a fresh turn and tool-bearing warmup resume, it allows a no-tool settlement resume to run hot or cold, dirties the workspace, and requires at least 90% reuse on the following resume without rotating the settled live-session generation. It also verifies that a thinking-level change rotates the generation and that the next steady resume restores at least 90% reuse:

pnpm test:docker:live-cli-backend:claude:cache

Anthropic live expectations

OpenAI live expectations

Scenario cacheRead floor Hit-rate floor
Stable prefix 4,608 0.90
Tool transcript 4,096 0.85
Image transcript 3,840 0.82
MCP-style transcript 4,096 0.85

The most recently observed baseline numbers (from live-cache-regression-baseline.ts, recorded 2026-04-04) landed at: stable prefix cacheRead=4864, hit rate 0.966; tool transcript cacheRead=4608, hit rate 0.896; image transcript cacheRead=4864, hit rate 0.954; MCP-style transcript cacheRead=4608, hit rate 0.891.

Why the assertions differ: Anthropic exposes explicit cache breakpoints, while OpenAI's reported reuse depends on available matching cached prefixes. The dated measurements below showed stepwise growth, not a universal hit-rate guarantee. Comparing the two providers against a single cross-provider percentage threshold produces false regressions.

Observed on 2026-09-16 with openai/gpt-5.6-luna through the ChatGPT-backed Responses route (transport: auto, default cacheRetention), using both apus agent --local and an isolated Gateway:

diagnostics.cacheTrace config

diagnostics:
  cacheTrace:
    enabled: true

enabled defaults to false. Cache traces otherwise write to $APUS_STATE_DIR/logs/cache-trace.jsonl and include messages, prompt text, and the system prompt by default. Output-path and payload-inclusion overrides are environment-only controls for one-off debugging.

Env toggles (one-off debugging)

Variable Effect
APUS_CACHE_TRACE=1 Enables cache tracing
APUS_CACHE_TRACE_FILE=path Overrides output path
APUS_CACHE_TRACE_MESSAGES=0|1 Toggles full message payload capture
APUS_CACHE_TRACE_PROMPT=0|1 Toggles prompt text capture
APUS_CACHE_TRACE_SYSTEM=0|1 Toggles system prompt capture

What to inspect

Prompt-cache observations record input, cacheRead, and cacheWrite per completed foreground model request alongside its stable system-prefix, volatile-suffix, and tools fingerprints, and flag cache-read drops from the previous request, including reported zero reads; billing totals remain separate. A flagged drop lists the tracked changes since the last request (model, cacheRetention, transport, streamStrategy, systemPrompt, systemPromptSuffix, tools, aggregateToolResultTruncation). Trace results require cache tracing (diagnostics.cacheTrace.enabled or APUS_CACHE_TRACE=1) and identify each request within its attempt.

The comparison baseline resets when the session ID changes, even if an isolated cron job reuses its provider cache key. A fresh transcript can legitimately reuse fewer tokens than the previous run's final request. Within a session, no tracked cache input change means the tracked fingerprints stayed stable; it does not prove identical final provider payloads or diagnose cache expiry. Compare request timing and final payloads before attributing a drop to provider caching.

Apus also checks that each converted request history extends the previous request in the same session. An undeclared edit, removal, or reorder records historyRewrite and warns once for the session. The diagnostic names bounded changed fields (such as timestamp, __apus, or content[0]) without logging their values; fields outside the diagnostic limit are grouped. Set APUS_PROMPT_CACHE_ASSERT=1 to throw at the first differing message during development or tests. Compaction, pruning, transient runtime-context removal, and image cleanup declare their rewrites as compaction, pruning, runtimeContextCarrier, and imageCleanup; model, transport, or retention changes start a new history series. Content-block fingerprints reuse hashes only while every primitive property still matches; unchanged large text and image data are not hashed again. Nested block values and message envelopes are checked on every observation, so in-place edits are detected even when wrappers or content arrays are reused. String-content hashes live with the bounded history baseline. Provider-owned tool schema declarations are fingerprinted once per object identity.

Quick troubleshooting

Related