For setup, configuration, and troubleshooting, see Browser. This page is the reference for the local control HTTP API, the apus browser CLI, and scripting patterns (snapshots, refs, waits, debug flows).

Control API (optional)

For local integrations only, the Gateway exposes a small loopback HTTP API. This standalone server is opt-in — set the environment variable APUS_EAGER_BROWSER_CONTROL_SERVER=1 in the gateway service environment and restart the gateway before the HTTP endpoints become available. Without this variable the browser control runtime still works through the CLI and agent tools, but nothing listens on the loopback control port.

POST /tabs/action is the batched form the CLI uses internally for browser tab subcommands ({"action":"new"|"label"|"select"|"close"|"list", ...}). Prefer the single-purpose tab routes above when scripting directly.

All endpoints accept ?profile=<name>. POST /start?headless=true requests a one-shot headless launch for local managed profiles without changing persisted browser config. Attach-only, remote CDP, and existing-session profiles reject that override because Apus does not launch those browser processes.

For tab endpoints, targetId is the compatibility field name. Prefer passing suggestedTargetId from GET /tabs or POST /tabs/open. Labels and tabId handles such as t1 are also accepted. Raw CDP target ids and unique raw target-id prefixes still work, but they are volatile diagnostic handles. Tab handles are scoped to a browser host or node and profile. Keep that route with the handle when making follow-up requests.

For profiles configured with driver: "extension", GET /tabs and the browser tool can also return webExtensionTabId, the runtime-scoped numeric Chrome WebExtensions tab ID for the same tab. This field is omitted for other drivers or when extension metadata is unavailable. Use it only when calling a WebExtensions API; continue to use suggestedTargetId or tabId for Apus browser actions because a webExtensionTabId can change after the browser or extension reconnects.

The Control UI's browser.request Gateway method accepts target: "host" to pin the Gateway host or target: "node" with node: "<node-id>" to pin a browser node. Pass the profile in query.profile. Explicit routes do not fall back to another host. Omitting them keeps the configured automatic routing. These routing fields do not grant access or change browser policy.

Browser previews require a result from the browser tool with a known route. Browser-shaped metadata from other tools does not trigger screenshots or change the panel's selection. Those results remain ordinary tool output.

When URL validation fails during tab listing, the tab keeps its identity and title but returns url: "" and urlUnavailableReason:

An empty URL alone does not indicate a policy denial. Navigation-policy errors also carry reason: "navigation_blocked". Raw blocked URLs and DNS details are not included in that metadata. Tab listings are observations, not authorization: every subsequent content read or action still enforces its own checks.

If shared-secret gateway auth is configured, browser HTTP routes require auth too:

Notes:

Screencast stream

POST /screencast mints a single-use token for a live view of the selected tab. Pass an optional targetId, maxWidth, maxHeight, and quality in the JSON body. Dimensions default to 1280 and are clamped to integers from 320 to 2000. JPEG quality defaults to 70 and is clamped from 30 to 90.

The response contains token, wsPath, expiresAtMs, targetId, and url. Resolve wsPath against the Gateway URL and open a WebSocket there: /browser/screencast?token=<token>. The 48-character hexadecimal token expires after 60 seconds and can be consumed once. Invalid, expired, and reused tokens are rejected with HTTP 401 before upgrade. Viewers send no application messages. Binary viewer messages close the connection.

Tickets and viewers minted through the Gateway are bound to the requesting Gateway connection. Ending that connection revokes unused tickets and closes its viewers. Revoked (invalidated) connections are fenced immediately, before the Gateway socket finishes closing: their tickets cannot upgrade, and their viewers receive no further frames or metadata. The loopback HTTP control API has no Gateway connection to bind, so its tickets remain TTL-only.

The plugin shares one CDP screencast per profile and tab. Chrome sends JPEG frames on repaint, paced to approximately 20 frames per second. Slow viewers skip frames instead of building a queue. Navigation immediately retires the capture session. A new CDP session starts only after the address is allowed, so delayed frames from the previous document cannot enter the new stream. A rejected navigation stops the stream.

Text messages are JSON with type in ready, meta, or error. A ready message includes targetId, url, and title. meta updates url and title after allowed navigation and page load. Binary messages contain:

  1. A four-byte unsigned big-endian JSON header length.
  2. That many bytes of UTF-8 JSON: { "url", "cssWidth", "cssHeight", "scrollX", "scrollY", "ts" }.
  3. The JPEG bytes.

cssWidth and cssHeight come from CDP's deviceWidth and deviceHeight metadata and describe the layout viewport in CSS pixels. scrollX and scrollY come from scrollOffsetX and scrollOffsetY. ts is the CDP frame timestamp.

Close code Meaning
4001 Token invalid or expired (normally rejected before upgrade with HTTP 401)
4003 navigation_blocked
4004 target_closed, including a profile lifecycle change
4005 Unsupported streaming
4006 authority_revoked (the requesting Gateway connection ended or was invalidated, e.g. device token revoked)
1012 Gateway shutting down

Chrome MCP existing-session profiles and missing Playwright return HTTP 501 with code: "SCREENCAST_UNSUPPORTED" and reason: "existing-session" or "playwright". Node-routed requests fail before proxying with INVALID_REQUEST and details { "code": "SCREENCAST_UNSUPPORTED", "reason": "node" }. The Control UI falls back to the existing screenshot route when streaming is unavailable. Navigation metadata updates the tab and address bar. The displayed image keeps its own URL and metrics until a replacement frame arrives. While Annotate or Inspect is active, the Control UI pins the captured image and its URL, then displays the latest held frame when capture mode ends.

/act error contract

POST /act uses a structured error response for validation, policy, and recognized interaction failures:

{ "error": "<message>", "code": "ACT_*" }

Current code values:

Other runtime failures may still return { "error": "<message>" } without a code field.

Playwright requirement

Some features (navigate/act/AI snapshot/role snapshot, element screenshots, PDF) require Playwright. If Playwright isn't installed, those endpoints return a clear 501 error.

What still works without Playwright:

What still needs Playwright:

Element screenshots also reject --full-page. The route returns fullPage is not supported for element screenshots.

If you see Playwright is not available in this gateway build, the packaged Gateway is missing the core browser runtime dependency. Reinstall or update Apus, then restart the gateway. For Docker, also install the Chromium browser binaries as shown below.

Docker Playwright install

If your Gateway runs in Docker, avoid npx playwright (npm override conflicts). For custom images, bake Chromium into the image:

APUS_INSTALL_BROWSER=1 ./scripts/docker/setup.sh

The browser also needs system libraries, so installing Chromium in a one-off Compose container is not durable. Rebuild the image with APUS_INSTALL_BROWSER=1 instead. To persist browser downloads and other caches, persist /home/node with APUS_HOME_VOLUME or a bind mount. See Docker.

How it works (internal)

A small loopback control server accepts HTTP requests and connects to Chromium-based browsers via CDP. Advanced actions (click/type/snapshot/PDF) go through Playwright on top of CDP. When Playwright is missing, only non-Playwright operations are available. The agent sees one stable interface while local/remote browsers and profiles swap freely underneath.

CLI quick reference

All commands accept --browser-profile <name> to target a specific profile, and --json for machine-readable output.

Basics: status, tabs, open/focus/close 
apus browser status
apus browser doctor
apus browser doctor --deep    # add a live snapshot check
apus browser start
apus browser start --headless # one-shot local managed headless launch
apus browser stop            # also clears emulation on attach-only/remote CDP
apus browser reset-profile   # moves the profile's browser data to Trash
apus browser tabs
apus browser tab             # shortcut for current tab
apus browser tab new
apus browser tab new --label research
apus browser tab label abcd1234 research
apus browser tab select 2
apus browser tab close 2
apus browser open https://example.com
apus browser focus abcd1234
apus browser close abcd1234
Profiles: list, create, delete 
apus browser profiles
apus browser create-profile --name research --color "#0066CC"
apus browser create-profile --name attach --driver existing-session --cdp-url http://127.0.0.1:9222
apus browser delete-profile --name research
Inspection: screenshot, snapshot, console, errors, requests 
apus browser screenshot
apus browser screenshot --full-page
apus browser screenshot --ref 12        # or --ref e12
apus browser screenshot --labels
apus browser snapshot
apus browser snapshot --format aria --limit 200
apus browser snapshot --interactive --compact --depth 6
apus browser snapshot --efficient
apus browser snapshot --labels
apus browser snapshot --urls
apus browser snapshot --selector "#main" --interactive
apus browser snapshot --frame "iframe#main" --interactive
apus browser snapshot --out snapshot.txt
apus browser console --level error
apus browser errors --clear
apus browser requests --filter api --clear
apus browser pdf
apus browser responsebody "**/api" --max-chars 5000
Actions: navigate, click, type, drag, wait, evaluate 
apus browser navigate https://example.com
apus browser resize 1280 720
apus browser click 12 --double           # or e12 for role refs
apus browser click-coords 120 340        # viewport coordinates
apus browser type 23 "hello" --submit
apus browser press Enter
apus browser hover 44
apus browser scrollintoview e12
apus browser drag 10 11
apus browser select 9 OptionA OptionB
apus browser download e12 report.pdf
apus browser waitfordownload report.pdf
apus browser upload /tmp/apus/uploads/file.pdf
apus browser upload /tmp/apus/uploads/file.pdf --ref e12
apus browser upload media://inbound/file.pdf
apus browser fill --fields '[{"ref":"1","type":"text","value":"Ada"}]'
apus browser dialog --accept
apus browser dialog --dismiss --dialog-id d1
apus browser wait --text "Done"
apus browser wait "#main" --url "**/dash" --load networkidle --fn "window.ready===true"
apus browser evaluate --fn '(el) => el.textContent' --ref 7
apus browser evaluate --fn 'const title = document.title; return title;'
apus browser evaluate --timeout-ms 30000 --fn 'async () => { await window.ready; return true; }'
apus browser highlight e12
apus browser trace start
apus browser trace stop
State: cookies, storage, offline, headers, geo, device 
apus browser cookies
apus browser cookies set session abc123 --url "https://example.com"
apus browser cookies clear
apus browser storage local get
apus browser storage local set theme dark
apus browser storage session clear
apus browser set offline on
apus browser set headers --headers-json '{"X-Debug":"1"}'
apus browser set credentials user pass            # --clear to remove
apus browser set geo 37.7749 -122.4194 --origin "https://example.com"
apus browser set media dark
apus browser set timezone America/New_York
apus browser set locale en-US
apus browser set device "iPhone 14"

Notes:

Stable tab ids and labels survive Chromium raw-target replacement when Apus can prove the replacement tab, such as a unique old/new pair for the same URL or a single old tab becoming a single new tab after form submission. Ambiguous duplicate-URL replacements receive fresh handles. Raw target ids are still volatile. Prefer suggestedTargetId from tabs in scripts.

Snapshot flags at a glance:

Snapshots and refs

Apus supports three "snapshot" styles:

Ref behavior:

Browser batch CLI

apus browser batch runs an array of nested /act actions in one /act call (the same kind="batch" runtime reached through the agent tool), so CLI users and scripts can combine actions like wait, click, type, and evaluate into a single replayable plan without per-action round trips. Each entry in actions[] is a BrowserActRequest — the closed union the /act route accepts (click, clickCoords, type, press, hover, scrollIntoView, drag, select, fill, resize, wait, evaluate, close, batch) — not arbitrary apus browser subcommands. batch is not supported on profile="user" and other existing-session (chrome-mcp) profiles. Send actions individually there.

Wait power-ups

You can wait on more than just time/text:

These can be combined:

apus browser wait "#main" \
  --url "**/dash" \
  --load networkidle \
  --fn "window.ready===true" \
  --timeout-ms 15000

Debug workflows

When an action fails (e.g. "not visible", "strict mode violation", "covered"):

  1. apus browser snapshot --interactive
  2. Use click <ref> / type <ref> (prefer role refs in interactive mode)
  3. If it still fails: apus browser highlight <ref> to see what Playwright is targeting
  4. If the page behaves oddly:
    • apus browser errors --clear
    • apus browser requests --filter api --clear
  5. For deep debugging: record a trace:
    • apus browser trace start
    • reproduce the issue
    • apus browser trace stop (prints TRACE:<path>)

JSON output

--json is for scripting and structured tooling.

Examples:

apus browser --json status
apus browser --json snapshot --interactive
apus browser --json requests --filter api
apus browser --json cookies

Role snapshots in JSON include refs plus a small stats block (lines/chars/refs/interactive) so tools can reason about payload size and density.

State and environment knobs

These are useful for "make the site behave like X" workflows:

Security and privacy

Strict-mode example (block private/internal destinations by default):

{
  browser: {
    ssrfPolicy: {
      dangerouslyAllowPrivateNetwork: false,
      allowedHostnames: ["*.example.com", "example.com", "localhost"],
    },
  },
}

Related