Threat model

This page enumerates what the Security Model defends: the assets worth protecting, the adversaries who threaten them, and the concrete threats each control answers. It is specific to Agent Assembly — the system that governs AI agents at the three interception layers and a central gateway.

Assets

Adversaries

The model assumes the agent’s own process is not trusted. The relevant adversaries are:

• The prompt-injected agent. A legitimate, SDK-adopting agent whose model has been steered by adversarial input into making tool calls, network requests, or data accesses its operator never intended. It is cooperative at the binary level but hostile at the behavioral level.
• The compromised / modified agent. An agent whose code has been altered — including one running a patched, outdated, or stubbed SDK — that deliberately tries to suppress, weaken, or skip the in-process checks.
• The bypassing agent. An agent that ignores the SDK entirely and makes raw syscalls, opens its own sockets, or links TLS directly to avoid the fastest interception layer.
• The careless integrator. Not malicious, but ships secrets in tool arguments or LLM payloads, or wires an agent to an over-broad capability set.

Trust posture. Because the SDK runs inside an attacker-controllable process, any guarantee anchored in the SDK is not a guarantee. Security must hold even when the SDK does nothing — see Trust boundaries and ADR 0002.

Threat scenarios

1. Unauthorized tool call — a prompt-injected agent invokes a tool outside its policy (e.g. terminal_exec, or read_file on /etc/...). Caught by the gateway tool allow/deny stage and, for MCP tools/call, by argument-level matching in the proxy (aa-proxy/src/intercept/mcp.rs).
2. Credential / data exfiltration — the agent embeds an API key in an LLM prompt or a tool argument and sends it upstream. The aa-security scanner detects the secret (via Aho-Corasick literal patterns) and redacts it before forward and before audit, on every path.
3. Network egress to an arbitrary host — the agent attempts to POST data to an attacker-controlled domain. The gateway’s network allowlist denies the request; the proxy enforces it at the wire even with no agent code change; eBPF SSL uprobes observe the plaintext if the agent bypasses both.
4. Policy evasion / SDK bypass — the agent disables, stubs, or skips the SDK scanner, or asserts an “already clean” marker. Defeated structurally: there is no trust marker on the wire, and aa-runtime re-scans unconditionally. This is proven by the bypass-resistance suite (aa-runtime/tests/aaasm_2568_gate_verification.rs).
5. Runaway spend — a looping agent issues unbounded LLM calls. The budget tracker denies further requests once the daily/monthly limit is met, or suspends the agent, per action_on_exceed.
6. Audit poisoning / secret leakage into logs — a sender emits raw prompts, full payloads, or per-beat heartbeat noise into the audit stream. The write-boundary sanitizer strips banned keys recursively and collapses heartbeats before anything reaches audit_logs.

STRIDE mapping

Each control is detailed in Protection and enforcement.

Last updated: 2026-06-11 by Chisanan232