Sandboxes, Credentials, and Human Boundaries

Model-layer defenses never reach 100 percent. The dependable floor is the environment: filesystem limits, egress control, per-tool credentials, external content as data.

Model-layer defenses never reach 100 percent: on the Gray Swan agent red-teaming benchmark, Opus 4.7 succumbed to a single attack about 0.1 percent of the time and to 5–6 percent after a hundred adaptive attempts, and auto mode still misses 17 percent of real overreach. The dependable floor is the environment — filesystem boundaries, egress control, per-tool credentials, and one hard rule: external content is data.

The floor is the environment

An internal red team phished an employee into launching Claude Code with a malicious prompt disguised as an ordinary task; one step asked Claude to read ~/.aws/credentials, encode it, and POST it out. Of 25 attempts, 24 exfiltrated successfully; this is direct prompt injection, since defenses are anchored to user intent and nothing looks anomalous while the user types anthropic-contain. Egress control blocks the POST regardless of intent; filesystem boundaries make ~/.aws unreachable.

Allowlists get punched through. In one incident a malicious file carried the attacker's own API key and encoded stolen data into requests to api.anthropic.com, on the allowlist and therefore legitimate-looking: an allowlist governs where traffic goes, not who sends it. The fix is a MITM proxy inside the VM that forwards only requests carrying the session token.

Three layers, ranked by how hard they are to bypass

The book splits guardrails into three layers: the context layer governs what the model can see, the execution layer governs what it can do, and the data layer governs what the world can end up looking like. The order is not request flow but the difficulty of bypassing each layer — the lower it sits, the less it relies on the model's own judgment, so a single successful injection cannot carry it away ai-agent-book.

The context layer can only lower probability; it can promise nothing, for a structural reason: an agent inside the same context has great difficulty telling whether it has already been injected. What this layer achieves in industry is putting the cheap test first. Anthropic's Constitutional Classifiers train input and output classifiers on synthetic data generated from natural-language rules, judge the question and the answer together (some replies look harmless alone and only read as a code word against the question they answer), and screen in two stages — a near-zero-cost probe that reads the model's own activations over every conversation, escalating to a stronger classifier only when it flags something. False positives at stage one then cost the user nothing ai-agent-book.

For the execution layer to count, its review must happen outside the context: a separate reviewer process, least-privilege credentials, sandbox isolation, a human in the loop. Otherwise it falls along with the injected agent, which is the same arrangement as letting the suspect investigate whether a crime was committed. The data layer is the last cell and the only one that does not depend on the two above being right: hand "who may do what to which row" to row-level security policies, constraints and validators, controlled views and stored procedures, and an access context bound by a trusted runtime so it cannot be forged. Even when the injection succeeded and the generated code left out every permission check, the overreaching write is still refused at the data layer ai-agent-book.

Guardrails fail in one other direction: over-refusal. Pushing the rate of dangerous requests getting through lower makes the model refuse legitimate but sensitive-looking work too, authorized security testing among them. So guardrail evaluation has to run both ways — testing only what should be blocked tells you nothing about whether what is explicitly allowed still completes.

Sandboxes set capability, approvals set timing

The sandbox decides what is technically possible; approval decides when consent is required. You need both, in that order — narrow the reachable surface first, then decide which residual risks deserve a human. Invert it and you get constant prompts the user rubber-stamps; one yes is not open-ended consent.

Then there is the moment before trust exists: a developer clones a repo to review a PR, and its .claude/settings.json hook runs before the "Do you trust this folder?" prompt. Three related vulnerabilities shared one fix — defer project-local config until the user accepts anthropic-contain.

Split brain, hands, and session

Brain — model, runtime, context engine, planner — scales statelessly. Hands — tool executor, browser, shell, filesystem, external APIs, credentials — runs sandboxed with narrowed privileges. Session — events, checkpoints, approvals, artifacts — is the recoverable source of truth.

The reason to split is single: a reachable credential is a risk, and one injection only has to convince the agent to read its own environment. Narrow scopes help, but the model outgrows them. Keep the token unreachable instead — bind credentials to resources at setup, or hold them in a vault outside the sandbox.

Pair it with a data rule: web pages, email, PDF and Office files, tool output, retrieved snippets, MCP resources, other agents' messages, uploaded code — all data. An audited connector is not audited data: a GitHub connector that passed malware checks still loads a poisoned README. Check where a high-risk tool's arguments came from.

Memory poisoning is not prompt injection

An attacker writes malicious content into long-term memory in one session so it fires later, in an unrelated session, triggering payments, config changes, or exfiltration. It persists, and the attacker need not be present.

Content inspection and trust scoring do not hold: across eight frontier models, attack success reached 68 percent, and Gemini-Flash's trust scoring gave all 54 poisoned entries a perfect 1.0 — a confidence filter, not a safety filter. Poisoned content looks benign until it activates, and three laundering channels help: the agent paraphrasing it into its own notes, a trusted tool echoing attacker content, and fabricated duplicates manufacturing consensus arxiv-memory-poisoning.

Three things do work. Bind an immutable origin label at write time and propagate it through derivation. Run an act-gate before sensitive actions — a deterministic check of the memory origin chain, so untrusted memory cannot trigger payments, exfiltration, or config changes, at roughly 1.3 microseconds. Require independent attestation for privilege escalation.

Approval fatigue, AI-BOM, and delegation chains

Telemetry showed users approving roughly 93 percent of prompts, and attention drops with each one. Two layers of auto mode defense still leave a 17 percent miss rate, and most misses are not failures to spot danger but misjudgments about whether consent covers this action: "clean up the PR" does not authorize a force-push anthropic-auto-mode.

So tier the approvals: gate what is hard to undo, crosses a boundary, or moves money and permissions, and let the sandbox carry the rest. A human should appear where judgment adds something.

AI-BOM: models, tools, MCP endpoints, and datasets are supply-chain surface, and what you cannot see you cannot govern — inventory them before launch, and keep scanning in CI. Non-human identity: tie every agent to a human principal so you can answer which agent acted, for whom, and under whose authority, with delegation chains that attenuate at each hop.

The cost is engineering effort and some autonomy. Pick your most dangerous tool and ask how much damage a fully compromised model could do — that is your sandbox boundary, not your approval dialog.