Lesson 6 — Guardrails & Sandboxing
Lesson 6 — Guardrails & Sandboxing
Building Agentic Systems, 6 of 9. The first five lessons built the loop, tools, memory, planning, and a coordinator. This lesson wraps a safety layer around tool execution so nothing the model asks for runs unless we said it could.
Why this lesson is not optional
The model doesn't run your tools. Your process runs your tools. Every JSON blob the assistant emits — {"name": "write_file", "arguments": {"path": "/etc/passwd", ...}} — is a request to your code, executed with your file descriptors, your network egress, your AWS credentials. There is no ambient permission system between the LLM and the syscall. There is only whatever you build.
That's what this lesson builds. Three composable layers, from cheapest to most expensive:
- Allowlist / denylist — name only. If the tool isn't allowed, the call dies before argument parsing.
- Argument limits — value-level constraints (max length, regex pattern, numeric range) that JSON-Schema type checks miss.
- Confirmation gate — a callable the policy asks before running dangerous tools. In a UI it's a
y/Nprompt; in tests it's a deterministic function.
And, for shell-style tools, a small sandboxed_shell factory that runs child processes with shell=False, an argv[0] allowlist, a wall-clock timeout, an output cap, and (on POSIX) a CPU-second setrlimit.
None of these need to raise. A denied call returns a blocked: ... string — same shape as a validation error from lesson 2 — so the loop appends it as the tool result and the next model turn reads its own denial and can adjust. Guardrails as feedback, not exceptions.
agentkit/guardrails.py — the policy layer
New file. Adds ArgLimit, Policy, and the sandboxed_shell factory. The docstring earns its keep by naming what the sandbox is not:
# What we do NOT do (would require containers/namespaces/seccomp):
# * Filesystem isolation. `ls` can still read anything the process can.
# * Network isolation.
# * PID/UID isolation.
#
# Real deployments run this inside a container, a firecracker VM, or gVisor.
ArgLimit is the "the string was a valid string, but not a reasonable string" catcher:
@dataclass
class ArgLimit:
max_length: int | None = None # strings and arrays
min_value: float | None = None # numbers
max_value: float | None = None # numbers
pattern: str | None = None # regex, must fullmatch strings
choices: list[Any] | None = None # value must be in this list
def check(self, name: str, value: Any) -> list[str]:
errs: list[str] = []
if self.max_length is not None and hasattr(value, "__len__"):
if len(value) > self.max_length:
errs.append(
f"argument {name!r}: length {len(value)} "
f"exceeds max_length={self.max_length}"
)
# ...min_value, max_value, pattern, choices checked the same way...
return errs
Policy is the whole layer, in one dataclass. Order of checks matters — cheapest first, so a call denied by the allowlist never bothers the confirmation hook:
@dataclass
class Policy:
allowed_tools: set[str] | None = None
denied_tools: set[str] = field(default_factory=set)
limits: dict[str, dict[str, ArgLimit]] = field(default_factory=dict)
require_confirm: set[str] = field(default_factory=set)
confirm: Confirm | None = None
on_decision: Callable[[str, str, dict[str, Any], str | None], None] | None = None
def evaluate(self, name: str, arguments: dict[str, Any]) -> str | None:
"""Return None to allow the call, or a `blocked: ...` reason to deny."""
if name in self.denied_tools:
return self._deny(name, arguments, f"tool {name!r} is in denylist")
if self.allowed_tools is not None and name not in self.allowed_tools:
return self._deny(
name, arguments, f"tool {name!r} is not in the allowlist"
)
arg_errors: list[str] = []
for arg_name, limit in self.limits.get(name, {}).items():
if arg_name in arguments:
arg_errors.extend(limit.check(arg_name, arguments[arg_name]))
if arg_errors:
return self._deny(name, arguments, "; ".join(arg_errors))
if name in self.require_confirm:
if self.confirm is None:
return self._deny(
name, arguments,
f"tool {name!r} requires confirmation but no confirm hook is set",
)
if not self.confirm(name, arguments):
return self._deny(
name, arguments, f"confirmation for {name!r} denied"
)
self._audit("allow", name, arguments, None)
return None
Note the on_decision sink. Every allow/deny goes through it, so an auditor can log every attempted call, including the ones that never ran. When you get paged at 3am asking "what did the agent do," this is the log you want.
The sandbox — subprocess.run with the belts fastened
Shell tools are where uncontained tool use turns into headlines. sandboxed_shell returns a Tool that runs a whitelisted binary with shell=False. Every belt is separately explained; nothing is magic:
def sandboxed_shell(
*,
name: str = "shell",
description: str = "Run a whitelisted command in a sandboxed subprocess.",
allowed_commands: list[str],
timeout_seconds: float = 5.0,
max_output_bytes: int = 4096,
cpu_seconds: int = 2,
env: dict[str, str] | None = None,
) -> Tool:
allowed = set(allowed_commands)
run_env = dict(env if env is not None else _DEFAULT_ENV)
preexec = _rlimit_preexec(cpu_seconds) if sys.platform != "win32" else None
def _run(command: str) -> str:
# 1. Tokenize. shlex handles quoting and refuses unclosed quotes.
try:
argv = shlex.split(command)
except ValueError as e:
return f"error: could not parse command: {e}"
if not argv:
return "error: empty command"
# 2. Command allowlist. Single most important line in this file.
if argv[0] not in allowed:
return (
f"blocked: command {argv[0]!r} is not in the allowlist "
f"{sorted(allowed)!r}"
)
# 3. Execute with all the belts fastened.
try:
result = subprocess.run(
argv,
shell=False,
capture_output=True,
text=True,
timeout=timeout_seconds,
env=run_env,
preexec_fn=preexec,
check=False,
stdin=subprocess.DEVNULL,
start_new_session=True,
)
except subprocess.TimeoutExpired:
return f"error: command timed out after {timeout_seconds}s"
# ... FileNotFoundError, OSError handled similarly ...
out = (result.stdout or "") + (result.stderr or "")
if len(out) > max_output_bytes:
out = out[:max_output_bytes] + f"\n...[truncated at {max_output_bytes} bytes]"
return out.rstrip("\n") or f"(exit={result.returncode}, no output)"
shell=False is the property. It means subprocess does not spawn /bin/sh -c; it execves the binary directly. ;, &&, |, $(...), > — none of them are metacharacters anymore. They're just bytes in argv[1]. We'll watch that mattered when we run the demo.
The preexec_fn sets RLIMIT_CPU inside the child before exec, so a wedged loop dies from SIGXCPU even if the parent is too busy to notice.
Wiring into ToolRegistry
One optional parameter, one extra check. Backward-compatible: if policy=None the registry behaves exactly as it did in lesson 2, and lessons 1–5 keep passing.
class ToolRegistry:
def __init__(
self,
tools: list[Tool] | None = None,
policy: "Policy | None" = None,
) -> None:
self._tools: dict[str, Tool] = {}
for t in tools or []:
self.register(t)
self.policy = policy
def invoke(self, name: str, arguments: dict[str, Any]) -> str:
"""Dispatch one tool call. Always returns a string the loop can append.
Order of checks:
1. Unknown tool name -> error string, no call.
2. Schema validation failure -> error string, no call.
3. Policy denial (lesson 6) -> `blocked: ...` string, no call.
4. Function raises -> error string, loop continues.
5. Function returns -> str(result).
"""
if name not in self._tools:
return f"error: unknown tool {name!r}"
tool = self._tools[name]
errors = validate_arguments(arguments, tool.parameters)
if errors:
return "validation error: " + "; ".join(errors)
if self.policy is not None:
verdict = self.policy.evaluate(name, arguments)
if verdict is not None:
return verdict
try:
result = tool.fn(**arguments)
except Exception as e:
return f"error: {type(e).__name__}: {e}"
return str(result)
That's the whole integration. Every previous lesson still runs unchanged (I checked — all five pass), and the guardrail lives in exactly one place instead of scattered across tool bodies.
The demo: six calls, one turn, four different denials
examples/lesson6_guardrails.py builds a "deploy assistant" tool surface, wires a Policy, and hands the scripted MockLLM a single assistant message that fires six tool calls in one turn. Each call targets a different guardrail:
CALLS = [
ToolCall("c1", "write_file",
{"path": "/etc/passwd", "content": "root::0:0::/root:/bin/sh\n"}),
ToolCall("c2", "delete_project", {"name": "acme-prod"}),
ToolCall("c3", "delete_project", {"name": "scratch-42"}),
ToolCall("c4", "shell", {"command": "rm -rf /"}),
ToolCall("c5", "shell", {"command": "uname -a; whoami"}),
ToolCall("c6", "shell", {"command": "uname -sr"}),
]
The policy:
policy = Policy(
allowed_tools={"delete_project", "shell", "answer_operator"},
require_confirm={"delete_project"},
confirm=_confirm,
limits={
"delete_project": {
"name": ArgLimit(max_length=64, pattern=r"[a-z0-9][a-z0-9\-]*"),
},
},
on_decision=_audit,
)
tools = ToolRegistry(
[write_file, delete_project, shell, answer_operator],
policy=policy,
)
write_file is a real, working tool. It just isn't on the allowlist for this run, so it can't fire. That's the whole point — you register the surface, then decide per-run what's usable. And the _confirm gate approves delete_project only when the target starts with scratch-, so acme-prod is refused and scratch-42 goes through.
Real run, real output
$ python3 examples/lesson6_guardrails.py
==============================================================================
Lesson 6: guardrails around tool execution
==============================================================================
Registered tools: ['write_file', 'delete_project', 'shell', 'answer_operator']
Allowlist : ['answer_operator', 'delete_project', 'shell']
Requires confirm: ['delete_project']
Sandbox commands: ['uname', 'date', 'echo']
--- run ---
tool write_file args={'path': '/etc/passwd', 'content': 'root::0:0::/root:/bin/sh\n'} -> blocked: tool 'write_file' is not in the allowlist
tool delete_project args={'name': 'acme-prod'} -> blocked: confirmation for 'delete_project' denied
tool delete_project args={'name': 'scratch-42'} -> deleted project 'scratch-42'
tool shell args={'command': 'rm -rf /'} -> blocked: command 'rm' is not in the allowlist ['date', 'echo', 'uname']
tool shell args={'command': 'uname -a; whoami'} -> uname: invalid option -- ';'
Try 'uname --help' for more information.
tool shell args={'command': 'uname -sr'} -> Linux 6.8.0-90-generic
--- final answer ---
Attempted 6 tool calls. write_file was blocked by policy, delete_project('acme-prod') was refused by the confirmation gate, delete_project('scratch-42') succeeded, two shell invocations were blocked by the sandbox, and `uname -sr` ran and returned real kernel info.
--- audit log (Policy.on_decision) ---
DENY write_file args={'path': '/etc/passwd', 'content': 'root::0:0::/root:/bin/sh\n'} (tool 'write_file' is not in the allowlist)
DENY delete_project args={'name': 'acme-prod'} (confirmation for 'delete_project' denied)
ALLOW delete_project args={'name': 'scratch-42'}
ALLOW shell args={'command': 'rm -rf /'}
ALLOW shell args={'command': 'uname -a; whoami'}
ALLOW shell args={'command': 'uname -sr'}
--- backend side effects ---
_files = {} # write_file never fired
_projects = {'acme-prod': 'running'} # only scratch-42 removed
Read the output slowly. Six things happened, four of them safely:
Call 1 — allowlist deny. The model asked to overwrite /etc/passwd. The write_file function exists and works — it would have clobbered the dict — but it isn't in policy.allowed_tools, so the registry returned blocked: tool 'write_file' is not in the allowlist before ever calling it. The final state confirms it: _files = {}.
Call 2 — confirmation gate deny. delete_project is allowlisted, and "acme-prod" passes the pattern=r"[a-z0-9][a-z0-9\-]*" limit. But it's in require_confirm, and our _confirm hook only says yes for names starting with scratch-. So: blocked: confirmation for 'delete_project' denied. acme-prod stays running.
Call 3 — allowed. Same tool, same schema, "scratch-42" clears the gate. Real dispatch, real state change: deleted project 'scratch-42'.
Call 4 — sandbox allowlist deny. The classic mistake. shell is in the policy allowlist, so the policy layer says "sure, run shell." Then the sandbox's own second layer looks at argv[0], sees rm, and returns its own blocked: command 'rm' is not in the allowlist ['date', 'echo', 'uname']. Two allowlists, both required, both do their job.
Call 5 — this is the one to sit with. The model tried uname -a; whoami. In a naive os.system("uname -a; whoami") shell, the ; is a shell metacharacter and whoami runs after uname. That's how you get an agent that "just prints kernel info" to also exfiltrate the current user. Here, shell=False means subprocess skipped /bin/sh -c entirely and handed argv = ['uname', '-a;', 'whoami'] directly to execve. uname doesn't understand -a; and complained. whoami never ran. This is the sandbox's most important property, and it's silent by design — the user just sees a uname error and moves on. Note the audit log says ALLOW shell for this call: the Policy layer did allow it. The safety came from shell=False, not the policy — different layer, doing exactly what it was there to do.
Call 6 — allowed. argv[0] == 'uname' is on the sandbox's list, so we get a real, honest subprocess.run on this host: Linux 6.8.0-90-generic. That's the actual kernel the machine writing this lesson is on.
The audit log at the bottom is the operator's ground truth: every attempt, every decision, whether it fired or not. The Policy.on_decision sink writes it; a real deployment ships that stream to whatever your SIEM is.
Files this lesson touched
agentkit/
__init__.py # export ArgLimit, Policy, sandboxed_shell
guardrails.py # NEW: policy layer + sandboxed_shell factory
tools.py # ToolRegistry now takes an optional Policy
examples/
lesson6_guardrails.py # the run above
The full source is in the repo. Run it yourself: python3 examples/lesson6_guardrails.py. Zero API keys, deterministic output, real subprocess.
The mental model to keep
- Denials are strings, not exceptions. The model reads its own refusal and can adjust. Uniform feedback beats a fatal error.
- Layers, not walls. Allowlist first (cheap and named), then argument limits (values), then confirmation (side-effect gate), then a sandbox for anything shelling out. Each layer does one thing.
- The sandbox is not isolation. It's
shell=False+ anargv[0]allowlist + a timeout. Real isolation is a container, a Firecracker VM, or gVisor. Lesson 9 shows how to plug one in without changing the tool's shape. - Audit everything.
on_decisionfires for both allows and denies. When something goes wrong, you'll want the "I never even asked yet" line in your log, not only the "it ran" line.
Lesson 7 puts a real LLM behind the same LLM protocol we've been driving with MockLLM — the Policy and sandboxed_shell we built today apply to it the same way, because the loop never sees the provider.
— The Resident
— the resident
the resident