A runaway agent never stops. If you send a friend to buy a charger and every shop is sold out, a sensible friend phones you after the third shop. A runaway agent keeps searching, rephrasing and retrying, and every step costs money.
Stop rules
| Rule | Example in the shopping task |
|---|---|
| A step limit | Stop after 15 steps |
| A budget | Stop after spending, say, 0.50 USD on the LLM |
| A "going in circles" check | Stop if the same search returns the same results twice |
| Ask a person | "Nothing under 50 dollars arrives by Friday, raise the limit?" |
Cost of a loop
Every step re-reads everything so far: a fixed part of tokens, plus new tokens at each step. Over steps the total tokens read is:
Worked example with , and a price of 3 USD per million tokens:
- a single call with one search result reads tokens, which costs 0.0075 USD
- 10 steps read tokens, which costs about 0.13 USD
- the ratio is , so 10 steps cost 17 times one call
- the total grows with , because each step re-reads a growing history
A sold-out shop with no stop rule, stopped only by a safety cap of 100 steps, reads tokens, or over 8 USD to buy nothing.
A missing stop rule is expensive
At 100,000 shoppers a day, a missing stop rule is expensive. Summarising old steps and caching repeated text keep the cost down.
Code
S, D = 2_000, 500 # fixed tokens, new tokens per step
PRICE = 3 / 1_000_000 # USD per input token
def tokens_read(n_steps):
"""Total tokens read when every step re-reads the whole history: n*S + d*n(n-1)/2."""
return sum(S + D * t for t in range(n_steps))
single = S + D
print(f"10 steps : {tokens_read(10):,} tokens = USD {tokens_read(10) * PRICE:.4f}") # 42,500, 0.1275
print(f"One call : {single:,} tokens = USD {single * PRICE:.4f}") # 2,500, 0.0075
print(f"Ratio : {tokens_read(10) / single:.0f}x") # 17x
print(f"100 steps: USD {tokens_read(100) * PRICE:.2f}") # 8.03A stop rule against going in circles can be added to the loop like this:
seen = set()
for step in range(max_steps):
call = llm(goal, history)
key = (call["name"], str(call["arguments"]))
if key in seen: # the same call twice: going in circles
history.append({"call": call, "obs": "stopped: going in circles, asking a person"})
break
seen.add(key)
# ... run the tool as beforeNotes:
tokens_read(n)is the same as written as a sum- a repeated call is a sign that the agent is going in circles
Cost is one side of reliability, and the other side is how often the agent succeeds, see Pass At K and Pass Hat K.