The LLM proposes and the code decides. Each safeguard lives in code, where the LLM cannot talk its way past it, like the access control in RAG.
Safeguards
| Safeguard | What it stops in the poisoned listing task |
|---|---|
| Approved sellers only | Buying from TechDealz88 |
| Mandate check in code | The 65.18 total |
| Human approval showing the real total | A misleading summary |
| Step and spending limits | Endless retries |
| A log of every step | Unanswerable disputes |
No single safeguard is enough. Together they turn a successful trick from "money gone" into "a refused cart and a log entry".
Code
The same loop as the bare loop, with 4 safeguards:
def run_guarded(goal, llm, shop, mandate, ask_user, max_steps=15):
"""The agent loop with safeguards: the LLM proposes, code checks, and only then does anything happen."""
tools, history = make_tools(shop), []
for step in range(max_steps): # 1. step limit
call = llm(goal, history)
if call["name"] == "finish":
history.append({"call": call, "obs": call["arguments"]["answer"]})
return history
problems = validate_call(call) # 2. known tools, valid arguments
if problems:
history.append({"call": call, "obs": {"status": "error", "problems": problems}})
continue
entry = {"call": call}
if call["name"] == "checkout":
cart = shop.current_cart()
ok, reasons = mandate.allows(cart) # 3. mandate check in code
if not ok:
history.append({"call": call, "obs": {"status": "refused",
"product_id": cart["product_id"], "reasons": reasons}})
continue
if not ask_user(cart): # 4. human approval
history.append({"call": call, "obs": "stopped: you declined"})
return history
entry["approved"] = True
entry["obs"] = tools[call["name"]](**call["arguments"]) # code runs the tool
history.append(entry)
history.append({"call": {"name": "stop", "arguments": {}}, # out of steps: hand over to a person
"obs": "Step limit reached: escalating to a person."})
return historyNotes:
validate_callis from Tool Call Validation andmandate.allowsis from Agent Mandate (AP2)- a refusal goes back to the LLM as an observation, so even a gullible LLM then buys the VoltMate instead
- in the poisoned listing task the LLM is still fooled and still proposes to pay 65.18, and the mandate check refuses it
- in the sold-out task the step limit stops the loop after 15 steps, which costs about 0.25 USD instead of over 8 USD
- the LLM did not get any smarter, and all of the improvement came from the code around it
Remembering tip
Limit, Known, Mandate, Approve. The four safeguards are the step limit, known tools with valid arguments, the mandate check, and human approval.
Writing these controls down, with named people responsible, is governance, see Agentic AI Governance (IMDA).