Slack QA

pnpm apus qa slack

Targets one real private Slack channel with two distinct bots: a driver bot controlled by the harness and a SUT bot started by the child Apus gateway through the bundled Slack plugin.

Once the Slack QA adapter loads, it requires async proxy capture support so write evidence can be read without running SQLite on the Gateway thread. If that capability is missing, adapter creation asks you to upgrade before acquiring credentials or contacting Slack. This does not guarantee that the current QA CLI loads on older hosts: use a matching current source checkout and see the older-host import limitation.

Agent E2E recipes

Use .agents/skills/slack-e2e/SKILL.md from the checkout under test for reusable native fixtures, Gateway replies, and runtime/config experiments. With an existing authenticated Convex CLI that can access the QA broker:

pnpm apus qa slack --doctor
pnpm apus qa slack \
  --scenario-file qa/scenarios/channels/slack-e2e-lifecycle.yaml

These opt-in modes discover credentials in memory and default to Convex, the CI role, and mock-openai. Explicit flags or credential environment settings take precedence; ordinary qa slack defaults remain unchanged. Repeat --scenario-file for complete custom YAML flows declaring execution.channel: slack and execution.config.agentE2e: true. Native writes run once; positive retryCount values are rejected.

Readiness checks the actual lease's identities, shared workspace, channel access, Gateway connection, and advertised scopes. The full lifecycle additionally needs driver reactions:read, reactions:write, files:read, and files:write. The driver manifest below includes those capabilities. Existing installations need the app owner to add missing scopes and reinstall the app before rerunning the lifecycle. A missing scope blocks its operation and full lifecycle proof, not basic text flows. The runner does not grant permissions or rotate credentials to find a passing result. Bot OAuth scopes, not workspace-admin privileges, authorize these fixture operations.

The driver uses Web API calls; the SUT alone owns Socket Mode. Stored message/file/reaction readback proves native state, while a correlated SUT reply proves Gateway ingress and delivery. Neither proves client rendering, human slash commands, button clicks, Agent View, or human typing. Use the manual client workflow for those claims.

Cleanup captures final Gateway write receipts after process shutdown and before temporary capture state is removed, then removes only owned fixtures before releasing the lease. Private <scenario-id>-slack-e2e.json receipts distinguish accepted writes, stored state, and incomplete cleanup.

With agentE2e: true, message scenarios in the leased channel also record their driver roots through the native receipt owner. Cleanup includes those roots and their captured SUT replies and standalone progress messages even when a scenario assertion fails. Capture reads acquire current database admission so Gateway restarts do not invalidate final cleanup.

Direct credential setup

Required env when --credential-source env:

Optional:

Canonical YAML scenarios exposed through the Slack live adapter:

Slack YAML module scenarios (qa/scenarios/channels/slack-*.yaml):

The Codex approval scenarios require an openai/* or codex/* --model, the normal live model credentials, and Codex auth or API-key auth accepted by the Codex plugin. The scenario details include the Codex app-server method, selected Codex model key, final Codex turn status, and operation-marker verification alongside the redacted Slack approval metadata.

Output artifacts:

Setting up the Slack workspace

The lane needs two distinct Slack apps in one workspace, plus a channel both bots are members of:

Prefer a Slack workspace dedicated to QA over reusing a production workspace.

The SUT manifest below intentionally narrows the bundled Slack plugin's production install (extensions/slack/src/setup-shared.ts:12) to the permissions and events covered by the live Slack QA suite. For the production-channel setup as users see it, see Slack channel quick setup; the QA Driver/SUT pair is intentionally separate because the lane needs two distinct bot user ids in one workspace.

1. Create the Driver app

Go to api.slack.com/apps → Create New App → From a manifest → pick the QA workspace, paste the following manifest, then Install to Workspace:

{
  "display_information": {
    "name": "Apus QA Driver",
    "description": "Test driver bot for Apus QA Slack live lane"
  },
  "features": {
    "bot_user": {
      "display_name": "Apus QA Driver",
      "always_online": true
    }
  },
  "oauth_config": {
    "scopes": {
      "bot": [
        "channels:history",
        "chat:write",
        "files:read",
        "files:write",
        "groups:history",
        "reactions:read",
        "reactions:write",
        "users:read"
      ]
    }
  },
  "settings": {
    "socket_mode_enabled": false
  }
}

Copy the Bot User OAuth Token (xoxb-...) - that becomes driverBotToken. The driver can read/write fixture messages, reactions, and files; it needs neither event subscriptions nor Socket Mode.

2. Create the SUT app

Repeat Create New App → From a manifest in the same workspace. This QA app intentionally uses a narrower version of the bundled Slack plugin's production manifest (extensions/slack/src/setup-shared.ts:12): reaction scopes and events are omitted because the live Slack QA suite does not cover reaction handling yet.

{
  "display_information": {
    "name": "Apus QA SUT",
    "description": "Apus QA SUT connector for Apus"
  },
  "features": {
    "bot_user": {
      "display_name": "Apus QA SUT",
      "always_online": true
    },
    "app_home": {
      "home_tab_enabled": true,
      "messages_tab_enabled": true,
      "messages_tab_read_only_enabled": false
    }
  },
  "oauth_config": {
    "scopes": {
      "bot": [
        "app_mentions:read",
        "assistant:write",
        "channels:history",
        "channels:read",
        "chat:write",
        "commands",
        "emoji:read",
        "files:read",
        "files:write",
        "groups:history",
        "groups:read",
        "im:history",
        "im:read",
        "im:write",
        "mpim:history",
        "mpim:read",
        "mpim:write",
        "pins:read",
        "pins:write",
        "usergroups:read",
        "users:read"
      ]
    }
  },
  "settings": {
    "socket_mode_enabled": true,
    "event_subscriptions": {
      "bot_events": [
        "app_home_opened",
        "app_mention",
        "channel_rename",
        "member_joined_channel",
        "member_left_channel",
        "message.channels",
        "message.groups",
        "message.im",
        "message.mpim",
        "pin_added",
        "pin_removed"
      ]
    }
  }
}

After Slack creates the app, do two things on its settings page:

Verify the two bots have distinct user ids by calling auth.test on each token. The runtime distinguishes driver and SUT by user id; reusing one app for both will fail mention-gating immediately.

3. Create the channel

In the QA workspace, create a channel (e.g. #apus-qa) and invite both bots from inside the channel:

/invite @Apus QA Driver
/invite @Apus QA SUT

Copy the Cxxxxxxxxxx id from channel info → About → Channel ID - that becomes channelId. A public channel works; if you use a private channel both apps already have groups:history so the harness's history reads will still succeed.

4. Register the credentials

Two options. Use env vars for single-machine debugging (set the four APUS_QA_SLACK_* variables and pass --credential-source env), or seed the shared Convex pool so CI and other maintainers can lease them.

For the Convex pool, write the four fields to a JSON file:

{
  "channelId": "Cxxxxxxxxxx",
  "driverBotToken": "xoxb-...",
  "sutBotToken": "xoxb-...",
  "sutAppToken": "xapp-..."
}

With APUS_QA_CONVEX_SITE_URL and APUS_QA_CONVEX_SECRET_MAINTAINER exported in your shell, register and verify:

pnpm apus qa credentials add \
  --kind slack \
  --payload-file slack-creds.json \
  --note "QA Slack pool seed"

pnpm apus qa credentials list --kind slack --status all --json

Expect count: 1, status: "active", no lease field.

5. Verify end to end

Run the lane locally to confirm both bots can talk to each other through the broker:

pnpm apus qa slack \
  --credential-source convex \
  --credential-role maintainer \
  --output-dir .artifacts/qa-e2e/slack-local

A green run completes in well under 30 seconds and qa-suite-report.md shows both slack-canary and slack-mention-gating at status pass. If the lane hangs for ~90 seconds and exits with Convex credential pool exhausted for kind "slack", either the pool is empty or every row is leased - qa credentials list --kind slack --status all --json will tell you which.