Prompt injection

An attacker crafts a message that manipulates the model into unsafe action ("ignore your instructions", "dump your filesystem", "follow this link and run commands").

Model choice now carries real weight here. Frontier models have become substantially more resistant: in a 2026 crowdsourced arena of 272K attacks across 41 agent scenarios - scored only when the agent both executed the harmful action and hid it from the user - success rates were 0.5% for Claude Opus 4.5, 1.0% for Sonnet 4.5, 1.3% for Haiku 4.5, and 8.5% for Gemini 2.5 Pro. Robustness tracked capability within a model family, so the models we recommend are a meaningful mitigation in their own right, not just a soft guardrail.

Two caveats keep this from being a solved problem. Adaptive human attackers still break models that score well on static benchmarks, with published success rates above 80% against state-of-the-art defenses once the attacker adapts. And smaller or older models remain markedly easier to steer. So treat model choice as your first and cheapest layer, then keep hard enforcement - tool policy, exec approvals, sandboxing, and channel allowlists - for anything whose blast radius you would not accept on a bad day.

Prompt injection does not require public DMs: even if only you can message the bot, any untrusted content it reads (web search/fetch results, browser pages, emails, docs, attachments, pasted logs/code) can carry adversarial instructions. The content itself is a threat surface, not just the sender.

Red flags to treat as untrusted:

What helps in practice:

Model choice matters. Prompt-injection resistance is not uniform across model tiers - smaller/cheaper models are more susceptible to tool misuse and instruction hijacking under adversarial prompts.

For tool-enabled agents or agents that read untrusted content, prompt-injection risk with older/smaller models is often too high. Do not run those workloads on weak model tiers.

External content and untrusted-input wrapping

OpenResponses input_file text is still injected as untrusted external content even though the Gateway decodes it locally - the block carries <<<EXTERNAL_UNTRUSTED_CONTENT ...>>> boundary markers plus Source: External metadata (this path omits the one-line data-boundary note used elsewhere). The same marker-based wrapping applies when media-understanding extracts text from attached documents before appending it to the media prompt.

Apus also strips common self-hosted LLM chat-template special-token literals (Qwen/ChatML, Llama, Gemma, Mistral, Phi, GPT-OSS role/turn tokens) from wrapped external content and metadata before they reach the model. Self-hosted OpenAI-compatible backends (vLLM, SGLang, TGI, LM Studio, custom Hugging Face tokenizer stacks) sometimes tokenize literal strings like <|im_start|> or <|start_header_id|> as structural chat-template tokens inside user content; without this sanitization, untrusted text in a fetched page, email body, or file-contents tool output could forge a synthetic assistant/system role boundary. Sanitization happens at the external-content wrapping layer, so it applies uniformly across fetch/read tools and inbound channel content. Hosted providers (OpenAI, Anthropic) already apply their own request-side sanitization; keep external-content wrapping enabled and prefer backend settings that split/escape special tokens when available.

Outbound model responses have a separate sanitizer that strips leaked <tool_call>, <function_calls>, <system-reminder>, <previous_response>, and similar internal scaffolding from user-visible replies at the final channel delivery boundary.

This does not replace dmPolicy, allowlists, exec approvals, sandboxing, or contextVisibility - it closes one specific tokenizer-layer bypass.

Bypass flags (keep off in production)

Only enable temporarily for tightly scoped debugging; if enabled, isolate that agent (sandbox + minimal tools + dedicated session namespace).

Hook payloads are untrusted content even when delivery comes from systems you control (mail/docs/web content can carry prompt injection). Weak model tiers increase this risk - for hook-driven automation, prefer strong modern model tiers and keep tool policy tight (tools.profile: "messaging" or stricter), plus sandboxing where possible.

Reasoning and verbose output in groups

/reasoning, /verbose, and /trace can expose internal reasoning, tool output, or plugin diagnostics not meant for a public channel - they can include tool args, URLs, plugin diagnostics, and data the model saw. Keep them disabled in public rooms; enable only in trusted DMs or tightly controlled rooms.