Live: OpenAI long context

• Goal: validate exact-model embedded Apus execution through a process-owned isolated Gateway, cross the long-context pricing boundary, observe a first-class OpenAI Responses compaction item, and prove opaque replay plus prefix pruning on the next request.
• Test: src/server/gateway/gateway-openai-long-context.live.test.ts
• Enable: APUS_LIVE_OPENAI_LONG_CONTEXT=1
• Profiles: APUS_LIVE_OPENAI_LONG_CONTEXT_PROFILE=full selects exact openai/gpt-5.6-luna with a 1050000 total window, 922000 safe active input, 128000 maximum output, and 700000 compaction threshold. reduced reaches the same transport and persistence path with a smaller budget.
• Metrics: APUS_LIVE_OPENAI_LONG_CONTEXT_METRICS=1 emits phase timing and token observations. These measurements are informational, not pass/fail latency targets.
• Long output: APUS_LIVE_OPENAI_LONG_CONTEXT_OUTPUT=1 requires a deterministic response between 4000 and 8000 output tokens.
• Optional raw read-tool stress: APUS_LIVE_OPENAI_LONG_CONTEXT_TOOL_OUTPUT=1. It is not part of the default recipe because the effective tool surface may use Code Mode instead of exposing the raw read tool.

Full 922000 input-budget recipe:

APUS_LIVE_OPENAI_LONG_CONTEXT=1 \
  APUS_LIVE_OPENAI_LONG_CONTEXT_PROFILE=full \
  APUS_LIVE_OPENAI_LONG_CONTEXT_METRICS=1 \
  APUS_LIVE_OPENAI_LONG_CONTEXT_OUTPUT=1 \
  node --import tsx scripts/test-live.mts --quiet src/server/gateway/gateway-openai-long-context.live.test.ts

Reduced-budget recipe:

APUS_LIVE_OPENAI_LONG_CONTEXT=1 \
  APUS_LIVE_OPENAI_LONG_CONTEXT_PROFILE=reduced \
  APUS_LIVE_OPENAI_LONG_CONTEXT_METRICS=1 \
  APUS_LIVE_OPENAI_LONG_CONTEXT_OUTPUT=1 \
  node --import tsx scripts/test-live.mts --quiet src/server/gateway/gateway-openai-long-context.live.test.ts

Long-context hard oracles

The full embedded and native recipes are proof runs, not throughput benchmarks. They fail unless the following runtime contracts hold:

• Runtime and model identity are exact: embedded Apus or native Codex as requested, both on openai/gpt-5.6-luna.
• At least one provider request crosses 272000 input tokens and every call reports priority service.
• Embedded Apus receives and persists a first-class encrypted Responses compaction item, replays the exact opaque item on the next request, and prunes the earlier input prefix. The encrypted content must never appear in display or diagnostics.
• Native Codex reports an effective window of 875900, grows beyond the 700000 total-scope threshold without a manual compact, and automatically compacts on the next turn.
• Each runtime produces a deterministic long response between 4000 and 8000 output tokens and preserves a durable marker through compaction and a Gateway restart.

Compaction duration, restart latency, turn latency, and total suite duration are emitted as informational metrics only.

Bounded compaction replay

The bounded replay cases put a verification marker only in synthetic tool output, reopen the SQLite session, and require the model to recall that marker. Configure the corresponding provider API key before running either case; explicit provider failures are test failures.

Anthropic exercises streamed native compaction and the next request's replay. Its native threshold has a 50,000-token minimum:

APUS_LIVE_TEST=1 ANTHROPIC_LIVE_TEST=1 \
  node scripts/run-vitest.mjs --config test/vitest/vitest.live.config.ts \
  src/agent/agents/anthropic-transport-stream.live.test.ts \
  -t 'replays streamed native compaction from SQLite'

OpenAI exercises automatic client summarization through the managed runner with a smaller configured context budget, then replays its persisted summary:

APUS_LIVE_TEST=1 APUS_LIVE_OPENAI_COMPACTION=1 \
  node scripts/run-vitest.mjs --config test/vitest/vitest.live.config.ts \
  src/agent/agents/sessions/agent-session.openai-compaction.live.test.ts \
  -t 'automatically compacts tool history and resumes from its SQLite checkpoint'

Fresh OpenAI API-key default:

APUS_LIVE_GATEWAY_OPENAI_API_DEFAULT=1 \
  APUS_LIVE_GATEWAY_PROVIDERS=openai \
  APUS_LIVE_GATEWAY_THINKING=off \
  pnpm test:live -- src/server/gateway/gateway-models.profiles.live.test.ts

This proof leaves APUS_LIVE_GATEWAY_MODELS unset, resolves the model through the fresh onboarding inference-selection seam, asserts openai/gpt-6-astra, and then runs a real gateway turn with that resolved model.

GPT-5.6 embedded Apus matrix:

APUS_LIVE_GATEWAY_THINKING=ultra \
  APUS_LIVE_GATEWAY_PROVIDERS=openai \
  APUS_LIVE_GATEWAY_MODELS='openai/gpt-5.6-terra,openai/gpt-5.6-luna' \
  pnpm test:live -- src/server/gateway/gateway-models.profiles.live.test.ts

Docker notes:

• The Docker runner lives at scripts/test-live-codex-harness-docker.sh.
• It passes OPENAI_API_KEY, copies Codex CLI auth files when present, installs @openai/codex into a writable mounted npm prefix, stages the source tree, then runs only the Codex-harness live test.
• Docker enables the image, MCP/tool, and Guardian checks by default. Set APUS_LIVE_CODEX_HARNESS_IMAGE_PROBE=0 or APUS_LIVE_CODEX_HARNESS_MCP_PROBE=0 or APUS_LIVE_CODEX_HARNESS_GUARDIAN_PROBE=0 when you need a narrower debug run.
• Docker uses the same explicit Codex runtime config, so legacy aliases or Apus fallback cannot hide a Codex harness regression.
• Matrix targets run sequentially in one container. The Docker script scales its default 35-minute timeout by target count; any outer shell or CI timeout must allow the same total. Canonical CI keeps each GPT-5.6 target in a separate shard.

Recommended live recipes

Narrow, explicit allowlists are fastest and least flaky:

• Single model, direct (no gateway):

  • APUS_LIVE_MODELS="openai/gpt-5.6-luna" pnpm test:live src/agent/agents/models.profiles.live.test.ts

• Small-model direct profile:

  • APUS_LIVE_MODELS=small pnpm test:live src/agent/agents/models.profiles.live.test.ts

• Small-model gateway profile:

  • APUS_LIVE_GATEWAY_MODELS=small pnpm test:live src/server/gateway/gateway-models.profiles.live.test.ts

• Ollama Cloud API smoke:

  • APUS_LIVE_TEST=1 APUS_LIVE_OLLAMA=1 APUS_LIVE_OLLAMA_BASE_URL=https://ollama.com APUS_LIVE_OLLAMA_MODEL=glm-5.1:cloud APUS_LIVE_OLLAMA_WEB_SEARCH=0 pnpm test:live -- extensions/ollama/ollama.live.test.ts

• Single model, gateway smoke:

  • APUS_LIVE_GATEWAY_MODELS="openai/gpt-5.6-luna" pnpm test:live src/server/gateway/gateway-models.profiles.live.test.ts

• Tool calling across several providers:

  • APUS_LIVE_GATEWAY_MODELS="openai/gpt-5.6-luna,anthropic/claude-opus-4-6,google/gemini-3.5-flash,deepseek/deepseek-v4-flash,zai/glm-5.1,minimax/MiniMax-M3" pnpm test:live src/server/gateway/gateway-models.profiles.live.test.ts

• Z.AI Coding Plan GLM-5.3 direct smoke:

  • ZAI_CODING_LIVE_TEST=1 pnpm test:live src/agent/agents/zai.live.test.ts

• Google focus:

  • Gemini (API key): APUS_LIVE_GATEWAY_MODELS="google/gemini-3.5-flash" pnpm test:live src/server/gateway/gateway-models.profiles.live.test.ts

• Google adaptive thinking smoke (qa manual from the private QA CLI - requires APUS_ENABLE_PRIVATE_QA_CLI=1 and a source checkout; see QA overview):

  • Gemini 3 dynamic default: APUS_ENABLE_PRIVATE_QA_CLI=1 pnpm apus qa manual --provider-mode live-frontier --model google/gemini-3.1-pro-preview --alt-model google/gemini-3.1-pro-preview --message '/think adaptive Reply exactly: GEMINI_ADAPTIVE_OK' --timeout-ms 180000
  • Gemini 2.5 dynamic budget: APUS_ENABLE_PRIVATE_QA_CLI=1 pnpm apus qa manual --provider-mode live-frontier --model google/gemini-2.5-flash --alt-model google/gemini-2.5-flash --message '/think adaptive Reply exactly: GEMINI25_ADAPTIVE_OK' --timeout-ms 180000

Notes:

• google/... uses the Gemini API (API key).
• google-gemini-cli/... uses the local Gemini CLI on your machine (separate auth + tooling quirks).
• google-antigravity/... is not a registered provider or supported setup path. Do not add it to live-test allowlists.
• Gemini API vs Gemini CLI:
  • API: Apus calls Google's hosted Gemini API over HTTP (API key / profile auth); this is what most users mean by "Gemini".
  • CLI: Apus shells out to a local gemini binary; it has its own auth and can behave differently (streaming/tool support/version skew).

Live: model matrix (what we cover)

Live is opt-in, so there is no fixed "CI model list." APUS_LIVE_MODELS=modern / APUS_LIVE_GATEWAY_MODELS=modern (and their all alias) run the curated priority list from HIGH_SIGNAL_LIVE_MODEL_PRIORITY in src/agent/agents/test-helpers/live-model-dynamic-candidates.ts, in this priority order:

The curated small-model list (APUS_LIVE_MODELS=small / APUS_LIVE_GATEWAY_MODELS=small), from SMALL_LIVE_MODEL_PRIORITY:

Notes on the modern list:

• codex and codex-cli providers are excluded from the default modern sweep (they cover CLI-backend/ACP behavior, tested separately on CLI backend and APNs lanes and ACP bind and Codex app-server lanes). openai/gpt-5.6 itself routes through the Codex app-server harness by default; see Live: Codex app-server harness smoke.
• fireworks, google, openrouter, and xai only run their explicitly curated model ids in the modern sweep (no automatic "every model from this provider" expansion).
• Include at least one image-capable model (Claude/Gemini/OpenAI-family vision variants, etc.) in APUS_LIVE_GATEWAY_MODELS to exercise the image check.

Run gateway smoke with tools + image across a hand-picked cross-provider set:

APUS_LIVE_GATEWAY_MODELS="openai/gpt-5.6-luna,anthropic/claude-opus-4-6,google/gemini-3.1-pro-preview,google/gemini-3.5-flash,deepseek/deepseek-v4-flash,zai/glm-5.1,minimax/MiniMax-M3" pnpm test:live src/server/gateway/gateway-models.profiles.live.test.ts

Optional additional coverage outside the curated lists (nice to have, pick a "tools"-capable model you have enabled):

• Mistral: mistral/...
• Cerebras: cerebras/... (if you have access)
• LM Studio: lmstudio/... (local; tool calling depends on API mode)

Aggregators / alternate gateways

If you have keys enabled, you can also test via:

• OpenRouter: openrouter/... (hundreds of models; use apus models scan to find tool+image capable candidates)
• OpenCode: opencode/... for Zen and opencode-go/... for Go (auth via OPENCODE_API_KEY / OPENCODE_ZEN_API_KEY)

More providers you can include in the live matrix (if you have creds/config):

• First-party provider plugins: anthropic, cerebras, github-copilot, google, google-gemini-cli, google-vertex, groq, mistral, openai, openrouter, opencode, opencode-go, xai, zai
• Via models.providers (custom endpoints): minimax (cloud/API), plus any OpenAI/Anthropic-compatible proxy (LM Studio, vLLM, LiteLLM, etc.)