Infrastructure overview — end-to-end design & dataflow

This page is the operator’s deployment map: it follows a single agent action all the way across the running infrastructure, from the agent process to the dashboard, and lists the real config knobs for each hop. Where the other architecture pages explain crates (System architecture), decisions (Key workflows), and data shape (Data flows), this page is the wiring diagram you reach for when you have to actually stand the system up and know which environment variable controls which boundary.

The product governs AI agents through three independently-deployable interception layers, ordered by latency cost (lowest first) and detection authority (highest first). All three converge on one central gateway, which decides, records, and persists every action it receives before serving it back to the dashboard via the read API:

1. L1 — in-process SDK shim (aa-sdk-client, behind the per-language FFI). Fastest path; requires SDK adoption. Emits events to aa-runtime over a Unix domain socket.
2. L2 — sidecar proxy (aa-proxy). MitM of outbound HTTPS using per-host certificates minted from a local root CA; enforces network-egress policy with no agent code change. Requires the process to honour HTTP_PROXY / HTTPS_PROXY and to trust the CA, and under the default llm_only only the built-in LLM hosts are decrypted — everything else is tunnelled uninspected.
3. L3 — eBPF (aa-ebpf). Kernel uprobes on OpenSSL plus exec/file syscall hooks; observe-only — it reports, it does not block. Linux only (file-I/O kprobes are x86_64-only), and it fails open if it cannot attach.

aa-runtime is the per-agent chokepoint that re-scans every event (the SDK is untrusted) and forwards it to aa-gateway over gRPC. The gateway holds the registry, the policy engine, and per-team budgets, writes an audit record, persists through the aa-storage facade, and exposes its read surfaces over HTTP/OpenAPI through aa-api for the dashboard.

This page gives you two complementary views of the same system:

1. Architecture at a glance — a static map of the components, the layers/planes they live in, and the relations (and transports/ports) between them. Read this first for the whole picture.
2. Request flow over time — a dynamic trace of one agent action moving through those components, top to bottom, with the enforcement decision returning before execution.

Architecture at a glance — components, layers & relations

The system is organised into four planes. An action is observed in the agent host, decided in the control plane, recorded in the persistence plane, and surfaced to humans through the presentation plane. Solid arrows are the in-band enforcement path; dashed arrows are out-of-band observation, async persistence, or the enforcement decision returning to the agent. Edge labels name the transport and port.

flowchart TB
    subgraph HOST["🖥️ Agent host — one per governed agent"]
        AGENT["AI agent process"]
        subgraph LAYERS["Interception layers · independently deployable (AA_LAYERS)"]
            direction LR
            L1["L1 · in-process SDK shim<br/>aa-sdk-client + per-lang FFI<br/><i>lowest latency · needs adoption</i>"]
            L2["L2 · sidecar proxy<br/>aa-proxy · HTTPS MitM<br/><i>needs proxy routing + CA trust</i>"]
            L3["L3 · eBPF<br/>aa-ebpf · kernel uprobes<br/><i>highest authority · Linux-only</i>"]
        end
        RT["aa-runtime<br/>per-agent chokepoint<br/>re-scan · redact · enforce"]
    end

    subgraph CTRL["🧠 Control plane · aa-gateway (the brain)"]
        GRPC["gRPC services :50051<br/>Policy · Audit · AgentLifecycle · Topology<br/>Approval · Secrets · Invalidation"]
        subgraph BRAIN["Decision core"]
            direction LR
            REG["Agent<br/>registry"]
            POL["Policy<br/>engine"]
            BUD["Team<br/>budgets"]
            AUD["Audit<br/>writer"]
        end
    end

    subgraph DATA["💾 Persistence"]
        CACHE["aa-cache · L1"]
        STORE["aa-storage facade<br/>driver registry"]
        DB[("Postgres /<br/>TimescaleDB")]
        JSONL[/"tamper-evident JSONL<br/>hash-chained · sync"/]
        NATSQ["NATS → audit_consumer<br/>async"]
    end

    subgraph PRES["📊 Presentation plane"]
        API["aa-api<br/>HTTP / OpenAPI :7700"]
        DASH["Dashboard<br/>React / Vite"]
        CLI["aasm CLI"]
    end

    AGENT -->|in-process| L1
    AGENT -.->|outbound HTTPS| L2
    AGENT -.->|SSL uprobe · syscalls| L3
    L1 -->|IpcFrame over UDS| RT
    L2 -->|forwarded event| RT
    L3 -->|ring-buffer event| RT

    RT -->|"gRPC CheckAction :50051"| GRPC
    GRPC -->|"Allow / Deny / RequireApproval"| RT
    RT -.->|block before execution| AGENT
    GRPC --> BRAIN

    BRAIN -->|via facade| CACHE
    CACHE --> STORE
    STORE --> DB
    AUD -->|sync write| JSONL
    AUD -.->|async| NATSQ
    NATSQ --> DB

    DASH -->|HTTP / WS :7700| API
    CLI -->|gRPC :50051| GRPC
    API -->|in-process read| BRAIN

How to read it quickly:

• Layers stack by trade-off, not sequence. L1→L2→L3 go from lowest latency to highest detection authority; you deploy the subset you need (AA_LAYERS), and whichever fire all converge on the one aa-runtime chokepoint.
• One brain, many services. Every gRPC service is a façade onto the same decision core (registry · policy · budgets · audit). aa-api reads that core in-process — it is not a second source of truth.
• The control plane is the only writer of record. Agents and the dashboard never touch persistence directly; all reads and writes funnel through the gateway and its aa-cache/aa-storage facade.

Request flow over time: a single agent action

sequenceDiagram
    autonumber
    participant Agent as AI agent process
    participant L1 as L1 SDK shim<br/>(aa-sdk-client)
    participant L2 as L2 proxy<br/>(aa-proxy)
    participant L3 as L3 eBPF<br/>(aa-ebpf, kernel)
    participant RT as aa-runtime<br/>per-agent chokepoint
    participant GW as aa-gateway<br/>registry · policy · budget · audit
    participant Store as aa-storage<br/>(memory / postgres / redis)
    participant API as aa-api<br/>HTTP / OpenAPI
    participant Dash as Dashboard / operators

    Note over Agent,L3: An agent action is observed by whichever<br/>subset of layers is deployed (AA_LAYERS)
    Agent->>L1: tool / network action (in-process)
    Agent-->>L2: outbound HTTPS (MitM)
    Agent-->>L3: SSL uprobe + exec/file syscalls

    L1->>RT: IpcFrame event over UDS<br/>(/tmp/aa-runtime-<agent_id>.sock)
    L2->>RT: forwarded event
    L3->>RT: ring-buffer event

    RT->>RT: enrich + re-scan + redact<br/>(fail-closed, AA_ENFORCEMENT_MODE)
    RT->>GW: gRPC PolicyService.CheckAction :50051

    GW->>GW: registry lookup · policy eval · budget check
    GW-->>RT: decision (Allow / Deny / RequireApproval)
    RT-->>Agent: enforce decision (block before execution)

    GW->>Store: append audit record + budget rollup
    Note over GW,Store: sync JSONL (tamper-evident) +<br/>async NATS → audit_consumer → Postgres

    Dash->>API: HTTP / OpenAPI :7700
    API->>GW: in-process read (registry, topology,<br/>audit, costs, alerts, traces)
    GW->>Store: read
    Store-->>GW: rows
    GW-->>API: read model
    API-->>Dash: JSON over HTTP / WS

Three properties this diagram encodes that matter operationally:

• Pre-execution enforcement. The decision returns to aa-runtime before the agent’s action runs, so a Deny blocks the action rather than recording it after the fact.
• The runtime never trusts the SDK. aa-runtime re-scans and redacts every event in its enforcement stage (aa-runtime/src/pipeline/enforcement.rs) regardless of which layer produced it — the SDK is an optimisation, not a trust boundary.
• Two audit sinks, neither a single point of failure. The synchronous, hash-chained JSONL write is the tamper-evident primary record; the asynchronous NATS → audit_consumer → Postgres path is the queryable store the dashboard reads. See Data flows for the full audit write path.

Per-component configuration notes

Each hop below lists the real environment variables / config knobs that control it, the file where the knob is read, and its default where one exists. All variable names are verified against the source; defaults are quoted from the code.

L1 — in-process SDK shim (aa-sdk-client)

The SDK client resolves where to reach the runtime/gateway and how the agent identifies itself. (The per-language FFI shims pin aa-sdk-client by git SHA and expose these through their own SDK config too.)

Layer selection (which layers run)

L2 — sidecar proxy (aa-proxy)

All read in aa-proxy/src/config.rs:

The sink is a bounded ring whose rotation deletes earlier evidence, and every deletion is counted in a completeness sidecar beside it. See Proxy Prevention-Evidence Retention.

L3 — eBPF (aa-ebpf, Linux-only)

eBPF probe activation is driven by env vars read by aa-runtime: the eBPF layer is selected via AA_LAYERS (aa-runtime/src/layer.rs), and the loader is tuned by AA_EBPF_INPROCESS_LOAD, AA_EBPF_CONFINE_PID, AA_EBPF_POLICY_PATH, and AA_EBPF_LOADERD_SOCK (aa-runtime/src/ebpf_control.rs). Events surface to aa-runtime over the kernel ring buffer. (AA_TLS_BPF / AA_EXEC_BPF / AA_FILE_IO_BPF are not runtime env vars — they are the compiled-in BPF objects in aa-ebpf, verified against the build-time AA_*_BPF_SHA256 digests.) eBPF is Linux-only; on other platforms cargo check -p aa-ebpf is the supported path and the layer is unavailable at runtime.

aa-runtime — the per-agent chokepoint

Read in aa-runtime/src/config.rs:

aa-gateway — registry, policy engine, budgets, audit

The default gRPC listen address is 127.0.0.1:50051; the seven gRPC services (PolicyService, AuditService, AgentLifecycleService, TopologyService, ApprovalService, SecretsService, InvalidationService) are registered together in aa-gateway/src/server.rs and can be served over TCP or UDS.

Persistence — aa-storage drivers

The gateway never talks to a concrete database directly; it goes through the aa-storage trait facade fronted by the aa-cache L1 cache, and the active driver decides where bytes land.

Driver selection is resolved at boot by aa-storage’s Registry + register_builtin_drivers; aasm config validate / aasm config boot exercise this loader. See Data flows → Storage data flow.

aa-api — HTTP / OpenAPI read surface

Dashboard / aasm CLI

The dashboard speaks HTTP/OpenAPI (and WS) to aa-api on :7700; the aasm CLI speaks gRPC to the gateway on :50051.

Where to go next

• System architecture — the crate map and transport topology behind this deployment.
• Key workflows — policy evaluation, agent registration, and the enforcement path as sequence diagrams.
• Data flows — the full audit write path and the write-boundary sanitizer.
• Security Model — the same system viewed through trust boundaries and defense-in-depth.

Last updated: 2026-08-07 by Chisanan232