Testing: live suites

For quick start, QA runners, unit/integration suites, and Docker flows, see Testing. This page covers live (network-touching) tests: model matrix, CLI backends, ACP, media providers, and credential handling.

This page is an index. The live testing kit is documented on six pages, one per reader job. Open the page that matches your task.

Page	Read it when
Quick live smokes and the Android node sweep	You want a fast ad hoc smoke or an Android node sweep.
Live model smoke (profile keys)	You are smoking a provider or model through the direct and gateway layers.
CLI backend and APNs lanes	You are driving a local CLI backend, or checking APNs proxy reachability.
ACP bind and Codex app-server lanes	You are debugging an ACP bind or the Codex app-server harness.
OpenAI long context and the live model matrix	You need the long-context proof runs, the recipes, or the curated model lists.
Media provider live lanes	You are running an image, music, video, or other media provider sweep.

Live tests vs your real gateway

Live suites and ad hoc smokes must never disturb a gateway that is already serving real traffic (yours or another operator's):

Bring your own gateway: use the in-process gateway (Layer 2 on Live model smoke) or start a dev instance with an isolated state dir (APUS_STATE_DIR=<scratch>) and a free port. Do not bind the default gateway port (18789) while a real gateway is running on it.
Do not apus gateway stop/restart (or launchctl/systemctl/tmux equivalents) a service you did not start in this session — that is the operator's live instance. Get explicit approval first.
Need realistic data? Copy the live state/DB into your dev state dir and test against the copy. In-place migrations of a live gateway's state also require explicit approval.

Credentials (never commit)

Live tests discover credentials the same way the CLI does. Practical implications:

If the CLI works, live tests should find the same keys.
If a live test says "no creds", debug the same way you'd debug apus models list / model selection.
An Apus live suite that cannot resolve its credentials must skip visibly in the reporter with Vitest test-context skip(reason) or fail; it must never pass green without reaching the provider, because a green run that did not reach the provider is not live evidence.
Per-agent auth profiles: SQLite credential rows in ~/.apus/agents/<agentId>/agent/apus-agent.sqlite (this is what "profile keys" means in the live tests)
Config: ~/.apus/apus.json (or APUS_CONFIG_PATH)
Legacy OAuth dir: ~/.apus/credentials/ (copied into the staged live home when present, but not the main profile-key store)
Local live runs copy the active config (with agents.*.workspace / agentDir overrides stripped) and stage each agent's canonical SQLite auth credential/state rows through the auth-store reader/writer APIs, not by copying its database or the rest of its directory. Agent sessions, workspace/, and sandboxes/ data are not staged. The runner also copies the legacy credentials/ dir and supported external CLI auth files/dirs (.claude.json, .claude/.credentials.json, .claude/settings*.json, .claude/backups, .codex/auth.json, .codex/config.toml, .gemini, .minimax) into a temp test home.

If you want to rely on env keys, export them before local tests or use the Docker runners on the lane pages listed above with an explicit APUS_PROFILE_FILE.

Where each section moved

Every section heading from the previous single-page version keeps its anchor here, so an existing link such as /help/testing-live#live-codex-app-server-harness-smoke still resolves. Each entry points at the page that now holds the content.

Testing - unit, integration, QA, and Docker suites
Tests - index of the testing reference, one page per reader job

Live tests vs your real gateway

Credentials (never commit)

Where each section moved

Related