What changed this month
If you followed tech news in October, you saw the word “agent” everywhere — in model launches, developer tools and enterprise announcements. The pitch is the same each time: give a model tools, a goal and some autonomy, and let it finish the job.
The gap between that pitch and production is still wide. Agents that look magical in a two-minute demo loop forever, call the wrong tool, or confidently do the wrong thing when real users show up. The teams shipping agents successfully aren’t using a secret model. They’re using four unglamorous patterns.
1. Constrain the tools
An agent is only as safe as the tools it can reach. Start with a short allowlist of narrow, well-described tools — “search the help center”, not “run any SQL”. Read-only tools first; anything that writes, sends or spends money gets its own review step.
ALLOWLIST = {"search_docs", "read_ticket", "draft_reply"}
def run_agent(task, llm, tools, max_steps=8, budget_usd=0.50):
spent, history = 0.0, []
for step in range(max_steps):
action = llm.decide(task, history, tools=ALLOWLIST)
if action.name not in ALLOWLIST:
return handoff(task, reason="tool not allowed")
if action.confidence < 0.6:
return handoff(task, reason="low confidence")
result = tools[action.name](**action.args)
spent += action.cost
history.append((action, result))
log_step(task.id, step, action, result, spent)
if action.is_final or spent > budget_usd:
break
# never fail silently: no result means a human takes over
return history[-1][1] if history else handoff(task, reason="no progress")
2. Budget every run
Agents fail in loops. Put hard limits on steps, wall-clock time and cost for every single run, and treat hitting a limit as a normal outcome, not an exception. A run that stops at step eight with a clear “I couldn’t finish” is far better than one that burns through your API budget overnight.
3. Hand off when unsure
The most valuable feature of a production agent is knowing when to stop. Design the handoff first: what the human sees, what context is attached, and how their decision flows back. Users forgive “I’ve passed this to a teammate” far more easily than a confident mistake.
Autonomy is a dial, not a switch. Start an agent at “suggest”, earn “act with approval”, and only then “act alone” — task by task.
4. Log every step
You can’t improve what you can’t replay. Trace every run end to end — it’s also how you build the evaluation set that makes the next version better.
| LOG THIS | SO YOU CAN |
|---|---|
| Prompt, model and version | Reproduce any run exactly |
| Each tool call and its arguments | Audit what the agent actually did |
| Tokens, latency and cost per step | Catch runaway loops early |
| Outcome and human feedback | Turn real runs into test cases |
Where agents still fall over
- Long, open-ended tasks. Errors compound with every step; ten mostly-right steps can still add up to a wrong result.
- Ambiguous goals. “Clean up the CRM” means something different to every person who says it.
- Untrusted inputs. Web pages, emails and documents can carry instructions the agent shouldn’t follow — treat them as data, never as commands.
- Irreversible actions. Payments, deletions and outbound messages still need a human in the loop.
What I’d build this quarter
Pick one narrow, repetitive workflow your team already does by hand — triaging support tickets, drafting first replies, summarising pull requests. Give the agent three read-only tools, a step budget and a clear handoff. Measure it against the manual process for two weeks. If it wins, widen the scope one tool at a time.
That’s less exciting than a fully autonomous assistant. It’s also the version that will still be running next year.