Sandboxes, Credentials, and Human Boundaries

Model-layer defenses never reach 100 percent. The dependable floor is the environment: filesystem limits, egress control, per-tool credentials, external content as data.

Model-layer defenses never reach 100 percent: on the Gray Swan agent red-teaming benchmark, Opus 4.7 succumbed to a single attack about 0.1 percent of the time and to 5–6 percent after a hundred adaptive attempts, and auto mode still misses 17 percent of real overreach. The dependable floor is the environment — filesystem boundaries, egress control, per-tool credentials, and one hard rule: external content is data.

The floor is the environment

An internal red team phished an employee into launching Claude Code with a malicious prompt disguised as an ordinary task; one step asked Claude to read ~/.aws/credentials, encode it, and POST it out. Of 25 attempts, 24 exfiltrated successfully; this is direct prompt injection, since defenses are anchored to user intent and nothing looks anomalous while the user types anthropic-contain. Egress control blocks the POST regardless of intent; filesystem boundaries make ~/.aws unreachable.

Allowlists get punched through. In one incident a malicious file carried the attacker's own API key and encoded stolen data into requests to api.anthropic.com, on the allowlist and therefore legitimate-looking: an allowlist governs where traffic goes, not who sends it. The fix is a MITM proxy inside the VM that forwards only requests carrying the session token.

Three layers, ranked by how hard they are to bypass

The book splits guardrails into three layers: the context layer governs what the model can see, the execution layer governs what it can do, and the data layer governs what the world can end up looking like. The order is not request flow but the difficulty of bypassing each layer — the lower it sits, the less it relies on the model's own judgment, so a single successful injection cannot carry it away ai-agent-book.

The context layer can only lower probability; it can promise nothing, for a structural reason: an agent inside the same context has great difficulty telling whether it has already been injected. What this layer achieves in industry is putting the cheap test first. Anthropic's Constitutional Classifiers train input and output classifiers on synthetic data generated from natural-language rules, judge the question and the answer together (some replies look harmless alone and only read as a code word against the question they answer), and screen in two stages — a near-zero-cost probe that reads the model's own activations over every conversation, escalating to a stronger classifier only when it flags something. False positives at stage one then cost the user nothing ai-agent-book.

For the execution layer to count, its review must happen outside the context: a separate reviewer process, least-privilege credentials, sandbox isolation, a human in the loop. Otherwise it falls along with the injected agent, which is the same arrangement as letting the suspect investigate whether a crime was committed. The data layer is the last cell and the only one that does not depend on the two above being right: hand "who may do what to which row" to row-level security policies, constraints and validators, controlled views and stored procedures, and an access context bound by a trusted runtime so it cannot be forged. Even when the injection succeeded and the generated code left out every permission check, the overreaching write is still refused at the data layer ai-agent-book.

Guardrails fail in one other direction: over-refusal. Pushing the rate of dangerous requests getting through lower makes the model refuse legitimate but sensitive-looking work too, authorized security testing among them. So guardrail evaluation has to run both ways — testing only what should be blocked tells you nothing about whether what is explicitly allowed still completes.

Sandboxes set capability, approvals set timing

The sandbox decides what is technically possible; approval decides when consent is required. You need both, in that order — narrow the reachable surface first, then decide which residual risks deserve a human. Invert it and you get constant prompts the user rubber-stamps; one yes is not open-ended consent.

Then there is the moment before trust exists: a developer clones a repo to review a PR, and its .claude/settings.json hook runs before the "Do you trust this folder?" prompt. Three related vulnerabilities shared one fix — defer project-local config until the user accepts anthropic-contain.

Split brain, hands, and session

Brain — model, runtime, context engine, planner — scales statelessly. Hands — tool executor, browser, shell, filesystem, external APIs, credentials — runs sandboxed with narrowed privileges. Session — events, checkpoints, approvals, artifacts — is the recoverable source of truth.

The reason to split is single: a reachable credential is a risk, and one injection only has to convince the agent to read its own environment. Narrow scopes help, but the model outgrows them. Keep the token unreachable instead — bind credentials to resources at setup, or hold them in a vault outside the sandbox.

Pair it with a data rule: web pages, email, PDF and Office files, tool output, retrieved snippets, MCP resources, other agents' messages, uploaded code — all data. An audited connector is not audited data: a GitHub connector that passed malware checks still loads a poisoned README. Check where a high-risk tool's arguments came from.

From passive defense to a full container lifecycle

Many teams treat the sandbox as a passive shell, or as a command blacklist — block anything containing a delete verb, and call it contained. String blacklists fall to an alias or an encoding change.

On modern cloud-native infrastructure the sandbox's security properties come from managing the whole lifecycle of a run environment:

Embedding defense in the lifecycle is what A Process-Oriented Runtime builds its execution host on. It converts a passive security review into the physical determinism of immutable infrastructure and one-shot destruction.

But the "final teardown" row rests on an assumption that is usually taken for granted: blocks are zeroed before the next tenant gets them. Cloudflare's September 2026 cross-tenant disclosure showed that assumption failing — the flaw was not in the VM boundary but in the storage allocator beneath it. Container root disks are provisioned through dm-thin; the affected pool used 64 KiB chunks with the "skip zeroing on allocation" option enabled. Writing a single 4 KiB block caused the system to allocate a 64 KiB physical chunk, of which only 4 KiB was overwritten; the remaining 60 KiB still held the previous container's data, so a raw disk read returned bytes the new container never wrote. Researchers tested 5,614 directory blocks across six production deployments; all of them came from other users, and they recovered directory structures, database pages, and structurally intact SQLite databases cloudflare-tenant.

The fix went beyond flipping one switch. Removing the option only restores default behavior for newly allocated chunks — blocks already mapped into existing thin devices (including running container disks and the host's image-layer snapshot cache) are not scrubbed, so new containers can inherit those mappings. That required deactivating running container disks, clearing host data off-peak, restarting VMs, and purging each host's image cache.

The criterion: verify isolation layer by layer instead of declaring it on an architecture diagram. A sound VM boundary does not make the allocator, volume reuse, and snapshot caches beneath it sound. Residual data on shared storage belongs at the top of the isolation-control list, and "zero on allocation" or "erase on release" must be explicitly configured and tested on its own, never assumed from a default.

Memory poisoning is not prompt injection

An attacker writes malicious content into long-term memory in one session so it fires later, in an unrelated session, triggering payments, config changes, or exfiltration. It persists, and the attacker need not be present.

Content inspection and trust scoring do not hold: across eight frontier models, attack success reached 68 percent, and Gemini-Flash's trust scoring gave all 54 poisoned entries a perfect 1.0 — a confidence filter, not a safety filter. Poisoned content looks benign until it activates, and three laundering channels help: the agent paraphrasing it into its own notes, a trusted tool echoing attacker content, and fabricated duplicates manufacturing consensus arxiv-memory-poisoning.

Three things do work. Bind an immutable origin label at write time and propagate it through derivation. Run an act-gate before sensitive actions — a deterministic check of the memory origin chain, so untrusted memory cannot trigger payments, exfiltration, or config changes, at roughly 1.3 microseconds. Require independent attestation for privilege escalation.

Approval fatigue, AI-BOM, and delegation chains

Telemetry showed users approving roughly 93 percent of prompts, and attention drops with each one. Two layers of auto mode defense still leave a 17 percent miss rate, and most misses are not failures to spot danger but misjudgments about whether consent covers this action: "clean up the PR" does not authorize a force-push anthropic-auto-mode.

So tier the approvals: gate what is hard to undo, crosses a boundary, or moves money and permissions, and let the sandbox carry the rest. A human should appear where judgment adds something.

One step beyond per-action approval is the plan is the permission: the agent investigates inside a read-only baseline, then submits a plan stating what it will do and what access it needs; one approval covers the whole plan and unlocks everything beyond the read-only baseline. Inside the plan, four links stay explicit: intent, work items, required scopes, and the enforcement point. Two boundaries hold: approval cannot grant access the user does not already have — the agent acts on the user's behalf, it does not expand it — and read-only operations can still leak sensitive information, so automatic approval is not the same as harmless vercel-plan-permissions. The failure shape is the per-action prompt: users approve each step before knowing how the agent intends to solve the problem, and repeated prompts turn review into a habit of clicking approve.

Identity has a split that is easy to miss: split by action, not by person. In one group chat different people will have the agent read documents, commit code, run deployments, and query controlled resources; one shared identity either falls short or gets silently amplified. The principle: whoever consumes data uses their own permission, whoever produces a result carries their own attribution. Reading documents uses the current sender's identity, platform actions like builds and deployments use a service identity, commits are attributed to the sender, and controlled resources go through explicit delegation confirmed by the owner's own login. The rule that matters most: the model cannot name an arbitrary identity — identity-scoped calls accept only controlled roles (current sender, service identity, confirmed owner), the role-to-identity mapping is injected by the trusted environment, and a model-generated "query as so-and-so" is rejected at the control layer (production practice from Tmall's digital employee).

AI-BOM: models, tools, MCP endpoints, and datasets are supply-chain surface, and what you cannot see you cannot govern — inventory them before launch, and keep scanning in CI. Non-human identity: tie every agent to a human principal so you can answer which agent acted, for whom, and under whose authority, with delegation chains that attenuate at each hop.

The cost is engineering effort and some autonomy. Pick your most dangerous tool and ask how much damage a fully compromised model could do — that is your sandbox boundary, not your approval dialog.