Permissions and Human Checkpoints

Which steps to gate so it can't go and delete the database.

Giving an agent permissions isn't a binary choice — all or nothing. The right approach is graded: let read-only actions through, allow low-risk writes, block high-risk writes, and route boundary actions to a human. anthropic-agents

Symptoms you'll recognize:

• The demo shows it deleting a database, and you assume it probably wouldn't really do it.
• You granted full permissions so it could be "fully automated."
• Every action happens with no human checkpoint at all.

Why grade permissions

Everything an agent does can be wrong, and the mistakes are distributed: goals drift, tools get misused, external content deceives it. A single malicious line in a webpage can make a fully privileged agent execute something it should never touch.

Grading moves risk assessment from "the model must get everything right every time" to "the system only lets it be right inside a safe envelope." A misread can be re-read; a deletion cannot be undone. The tolerance cost of the first is far lower.

It's the same logic as granting access to a person: you wouldn't hand an intern delete rights on the production database just because you trust them. Agents are no different.

Four levels

• Read-only. Read docs, read data, read code. Hand these to the agent freely — if they go wrong, you can simply redo them.
• Low-risk writes. Drafts, temp files, its own draft folder. Auto-approve, but log what it wrote and changed.
• High-risk writes. Deletes, overwrites, sends, config changes, anything involving money. Denied by default, executed only with explicit authorization — which usually means a human approving it.
• Human checkpoint. Any action beyond the granted scope, or matching a high-risk list, stops and waits for a person. This is the top tier: a human makes the final call.

The boundaries between levels can shift with your scenario, but the skeleton stays the same: the more irreversible the impact, the more it depends on a person.

Where to put checkpoints

More checkpoints aren't better — if every step asks a human, the agent stops being useful. Runaway permissions are among the most common ways LLM applications fail, and OWASP keeps it on its own risk list. owasp-llm Ask three questions:

• Is the action reversible? Reversible (edit a draft, save a temp file) → let it through. Irreversible (delete, send, pay) → checkpoint.
• How wide is the blast radius? One user's email → maybe automatic. A blast to every user, or a batch operation on production data → checkpoint, without debate.
• Does it have complete information? Agents usually see only local context, not global constraints. Anything that needs a global judgment to confirm should keep a human gate.

A pragmatic design: put the checkpoint before the action, not after. Have the agent request permission before a high-risk action and only execute once a person approves. Reporting afterward turns the checkpoint into a log entry, not a protection.

Setting defaults

For agent permissions, the default matters more than the list. Two principles:

• Least privilege. Nothing is allowed by default; open things up per task and revoke when the task ends. "Grant everything first, tighten later" is a disaster recipe — you usually never get around to tightening.
• Bind permissions to identity. The agent's rights should match who it is acting for. User A's agent must not have access to user B's data. That is both security and compliance.

Finally, treat permissions as part of your evaluation: periodically check which high-risk actions it actually called, whether it overstepped, and what your checkpoint interception rate is. A permission system is judged by its run records, not by how clean the design looks.

Keep this in mind: a human checkpoint isn't a slow process — it's the only line of defense that doesn't require trusting the model. Models err, drift, and get deceived; a human gate doesn't.