Gateway service not running

Use when the service is installed but the process does not stay up.

apus gateway status
apus status
apus logs --follow
apus doctor
apus gateway status --deep   # also scan system-level services

Look for:

Common signatures 
- `Gateway start blocked: set gateway.mode=local` or `existing config is missing gateway.mode` → local gateway mode is not enabled, or the config file was clobbered and lost `gateway.mode`. Fix: set `gateway.mode="local"` in your config, or re-run `apus onboard --mode local` / `apus setup` to restamp the expected local-mode config. If you are running Apus via Podman, the default config path is `~/.apus/apus.json`.
- `refusing to bind gateway ... without auth` → non-loopback bind without a valid gateway auth path (token/password, or trusted-proxy where configured).
- `another gateway instance is already listening` / `EADDRINUSE` → port conflict.
- `Other gateway-like services detected (best effort)` → stale or parallel launchd/systemd/schtasks units exist. Most setups should keep one gateway per machine; if you do need more than one, isolate ports + config/state/workspace. See [/gateway#multiple-gateways-same-host](/gateway#multiple-gateways-same-host).
- `System-level Apus gateway service detected` from doctor → a systemd system unit exists while the user-level service is missing. Remove or disable the duplicate before allowing doctor to install a user service, or set `APUS_SERVICE_REPAIR_POLICY=external` if the system unit is the intended supervisor.
- `Gateway service port does not match current gateway config` → the installed supervisor still pins the old `--port`. Run `apus doctor --fix` or `apus gateway install --force`, then restart the gateway service.

Related:

macOS gateway silently stops responding, then resumes when you touch the dashboard

Use when channels (Telegram, WhatsApp, etc.) on a macOS host go quiet for minutes to hours at a time, and the gateway appears to come back the moment you open the Control UI, SSH in, or otherwise interact with the host. There is usually no obvious symptom in apus status because by the time you look the gateway is alive again.

ls ~/.apus/logs/stability/ | tail -5
apus gateway stability --bundle latest
pmset -g log | grep -iE "sleep|wake|maintenance" | tail -50
launchctl print gui/$UID/ai.apus.gateway | grep -E "state|last exit|runs"

Look for:

Common signatures:

What to do:

  1. Upgrade the gateway if you are running a release before 2026.5.26. After upgrading, future ENETDOWN errors are logged as warnings instead of terminating the process.

  2. Reduce maintenance sleep activity on Mac mini / desktop hosts meant to run as always-on servers:

    sudo pmset -a sleep 0 disksleep 0 standby 0 powernap 0

    This significantly reduces, but does not entirely eliminate, the underlying driver flap. The system can still perform some maintenance sleeps for TCP keepalive and mDNS upkeep regardless of these flags.

  3. Add a liveness watchdog so a future crash burst that gets parked by launchd is caught quickly:

    # Example launchd-aware liveness check, suitable for a 5-minute cron or LaunchAgent
state=$(launchctl print gui/$UID/ai.apus.gateway 2>/dev/null | awk -F'= ' '/state =/ {print $2; exit}')
if [ "$state" != "running" ]; then
  launchctl kickstart -k gui/$UID/ai.apus.gateway
fi

    The point is to externally re-arm the respawn gate; KeepAlive=true alone is not sufficient on macOS after a crash burst.

Related:

macOS launchd supervisor loop with duplicate gateway/node LaunchAgents

Use this when a macOS install keeps restarting every few seconds, apus health checks flap between healthy and unavailable, and channel dispatch stalls even though the service appears to be running.

This happens when both ai.apus.gateway and ai.apus.node LaunchAgents are active and each injects APUS_LAUNCHD_LABEL. In that state Apus can detect launchd supervision, try to hand restart back to launchd, and fall into a fast EADDRINUSE/respawn loop instead of one stable gateway process.

for i in 1 2 3 4; do
  ps aux | grep 'apus.*index.js' | grep -v grep | awk '{print $2}'
  sleep 10
done

apus gateway status --deep
apus node status
launchctl print gui/$UID/ai.apus.gateway | grep -E 'state|last exit|runs'
tail -n 80 ~/Library/Logs/apus/gateway.log

Look for:

What to do:

  1. If this host should only run the Gateway service, remove the managed node service through Apus. Skip this step if you actively rely on the node service for remote node features; uninstalling it stops those features on this host:

    apus node uninstall

  2. Install a persistent Gateway wrapper that clears the inherited launchd markers before starting Apus. Use the supported --wrapper option; do not edit the generated file under ~/.apus/service-env/, because service reinstall, update, and doctor repair regenerate that file:

    mkdir -p ~/.local/bin
cat >~/.local/bin/apus-launchd-workaround <<'EOF'
#!/bin/sh
set -eu
unset APUS_LAUNCHD_LABEL LAUNCH_JOB_LABEL LAUNCH_JOB_NAME XPC_SERVICE_NAME || true
exec apus "$@"
EOF
chmod 700 ~/.local/bin/apus-launchd-workaround

apus gateway install \
  --wrapper ~/.local/bin/apus-launchd-workaround \
  --force

    gateway install persists the wrapper path across forced reinstalls, updates, and doctor repairs.

  3. Verify that the Gateway is stable and serving RPC, not merely listening:

    apus gateway status --deep --require-rpc

for i in 1 2 3 4; do
  ps aux | grep 'apus.*index.js' | grep -v grep | awk '{print $2}'
  sleep 10
done

    The PID sample should show one stable process instead of a rotating set of PIDs, and inbound channel dispatch should resume.

  4. After upgrading to a release where the underlying dual-LaunchAgent loop is fixed, remove the workaround and reinstall the normal managed service:

    APUS_WRAPPER= apus gateway install --force
rm ~/.local/bin/apus-launchd-workaround

Related:

Native aborts on Linux (SIGABRT)

malloc(): invalid next->prev_inuse (unsorted) followed by systemd status=6/ABRT indicates detected native heap corruption. It does not identify the corrupting code, and it is different from a kernel OOM kill. A JavaScript signal handler or stability bundle cannot reliably capture a native abort(). Arrange OS core capture before another failure.

Startup logs include native runtime (PID, platform, architecture, Node, V8, libuv, OpenSSL and SQLite versions) and worker startup state (tracked Worker count, starts/retirements by script, and shared compute admission counters). These are startup facts, not a snapshot of the moment of failure. Retain them with the crash timestamp, journal, exact Apus build and Node executable.

For a systemd system service, substitute the installed unit name below. For a user service, use systemctl --user without sudo; its hard core limit cannot exceed the user manager's inherited limit.

sudo systemctl edit apus-gateway.service
# Add this drop-in:
# [Service]
# LimitCORE=infinity

sudo systemctl daemon-reload
sudo systemctl show apus-gateway.service -p LimitCORE -p MainPID
sysctl kernel.core_pattern

Apply the limit at the next operator-coordinated service restart. Then check /proc/<gateway-pid>/limits for the running process's Max core file size; changing the unit does not change an already running process's limit.

Choose the capture backend indicated by kernel.core_pattern:

Validate the chosen backend with a disposable process under equivalent service limits and identity, never by aborting the live Gateway. Open a captured core with the matching Node binary and debug symbols (gdb /path/to/node /path/to/core for direct files), then run thread apply all bt. Preserve all thread stacks: the aborting thread can be detecting damage caused elsewhere. Core files contain process memory, including credentials and message content; keep them private and share only reviewed, redacted evidence.

Gateway exits during high memory use

Use when the Gateway disappears under load, the supervisor reports an OOM-style restart, or logs show memory pressure: level=critical.

apus gateway status --deep
apus logs --follow
apus gateway stability --bundle latest
apus gateway diagnostics export

Look for:

Common signatures:

An administrator can also sample allocations with apus gateway call diagnostics.heapProfile --timeout 30000 on Node or Apus's Bun runtime. This captures current allocation activity, not a past spike or all native memory. Older bundles remain readable with apus gateway stability --bundle latest.

Review the sanitized diagnostics export before attaching it to a bug report; avoid copying raw logs.

Node's automatic heap ceiling can be roughly 4 GiB on a large host. That is a default sizing decision, not a general 64-bit address-space ceiling. --max-old-space-size controls V8 old space; the measured total V8 heap ceiling also includes other heap spaces. RSS additionally includes native allocations, buffers, and other process memory. A higher heap ceiling does not preallocate the ceiling, but it still needs enough real capacity and headroom under sustained load.

For a foreground Node Gateway, set a native heap flag before Node starts, for example on a host with sufficient capacity:

NODE_OPTIONS="--max-old-space-size=16384" apus gateway run

For a custom supervisor or Docker runtime command, place --max-old-space-size=16384 immediately after node, before the Apus entry script, or set NODE_OPTIONS in that process or container's launch environment. Docker image build-time heap options do not configure the runtime Gateway. An Apus config or dotenv value loaded after Node starts cannot resize its heap. NODE_OPTIONS can also reach spawned Node children, so prefer a direct Node argument when only the Gateway should receive the budget.

For managed Node services, use the managed Gateway heap policy and inspect both managed launch arguments and operator-owned environment overrides before changing them. Native argv overrides the same option in NODE_OPTIONS; percentage old-space sizing takes precedence over absolute old-space sizing. Regeneration preserves stored argv but does not add an automatic heap flag when an operator override owns NODE_OPTIONS. Installer-shell NODE_OPTIONS does not become a service override. Runtime pressure diagnostics use the effective V8 heap ceiling and physical/reported constraint headroom; an oversized explicit heap setting does not raise the RSS alert threshold above physical capacity. Pressure warnings are diagnostic evidence, not heap limits or automatic restart triggers.

RSS growth detection compares minimum RSS values from completed five-minute windows and requires two consecutive increases. Growth accumulates while these floors keep rising; a flat or falling floor, a sampling gap over ten minutes, or a clock rollback resets the trend. This filters ordinary GC peaks while detecting smaller sustained increases. The logged rssGrowth and windowMs describe the accumulated floor increase and elapsed time between those minima, which can exceed ten minutes.

On Node, growth warnings and critical events use 4% and 8% of the smaller of the measured V8 heap ceiling and available process capacity, with minimum thresholds of 512 MiB and 1 GiB. Process capacity uses the reported constraint bounded by physical RAM, or physical RAM when no constraint is reported. With a 16 GiB heap and sufficient RAM, those thresholds are about 655 MiB and 1.28 GiB. Unknown heap limits and Bun retain the 512 MiB/1 GiB growth thresholds; Bun's existing absolute memory caps are unchanged. Absolute RSS and heap pressure checks still run on every sample.

Related: