Building Agents: The Decisions That Actually Matter
The word agent now covers everything from a chatbot with a search box to a process that runs unattended and moves money. That makes most advice about agents useless, because the advice is answering a different question than the one you have. OpenAI’s practical guide to building agents, published in April, is better than most because it keeps returning to one question: which decisions are you handing to a model, and what happens around that handoff.
Having read it alongside Anthropic’s take from December, the interesting thing is how much the two agree, and what neither spends much time on.
Whether it should be an agent at all
The guide’s screening test is that agents earn their place where deterministic approaches have already failed: workflows with nuanced judgment calls, rulesets that have grown so intricate that updating them is expensive and error-prone, and processes that run on unstructured input like documents and conversations. If your use case does not clearly hit one of those, it says, a deterministic solution will probably do.
That is the correct order to think in, and it is worth being stubborn about. A model in the loop turns a testable function into a distribution of behaviors. You pay for that in evals, in observability, in the review process for every prompt change, and in the class of failures that only show up on inputs nobody imagined. Rules are cheap to reason about right up until the day there are four hundred of them, and the honest signal for building an agent is that you are already past that day.
Anthropic frames the same boundary differently and, I think, more usefully for architecture: a workflow is a system where code orchestrates the models along predefined paths, and an agent is one where the model directs its own process and tool use. Most things people ship as agents are workflows with one genuinely agentic step in the middle. That is not a criticism. Knowing which of your steps is the agentic one tells you exactly where to spend your testing effort.
One agent, and a tool budget
Both guides say to start with a single agent and add tools before reaching for a multi-agent architecture, which is unsurprising advice from anyone who has debugged two models negotiating with each other. The detail in the OpenAI guide that I had not seen stated as plainly is about when a single agent stops coping. It is not the number of tools. Some implementations handle more than fifteen well-defined tools without trouble, while others fall apart with fewer than ten when those tools overlap.
That reframing matches what goes wrong in practice. A model picking between search_orders and lookup_order_history is not confused because it has too many options, it is confused because you gave it two names for what it cannot distinguish as separate jobs. The test I would apply before adding a tool: could a new engineer, given only the tool names and descriptions, pick the right one every time without reading the implementations? If not, the model has no chance, and no amount of prompt tuning fixes an ambiguous interface.
The guide splits tools into data tools that fetch context, action tools that change something, and orchestration tools where one agent is exposed as a tool to another. It is worth keeping that separation visible in your own codebase, because the second category is where every interesting failure lives and it deserves different treatment in testing, permissions, and logging.
The loop is the part you own
Underneath any agent framework is a loop that runs until an exit condition. The Agents SDK exits when a final-output tool is invoked or the model replies without calling any tool, with a maximum turn count as backstop. That is the whole mechanism, and every framework is a variation on it.
Since the loop is yours, treat its budget as a product decision rather than a default. Maximum turns is the obvious one, but wall-clock time and spend per run matter more in production, and neither is free with any SDK. An agent that hits a tool returning a slightly different error each time will happily burn twenty turns making no progress, and the failure mode is not an exception, it is a bill.
The other thing the loop implies, which neither guide dwells on, is that your action tools will be called more than once. A model retries. It retries after a timeout that actually succeeded on the server, it retries because it misread its own previous output, and it retries when a handoff loses track of what has already been done. If your tools are not idempotent, this is where you find out. Give every write tool an idempotency key that the agent must pass, or derive one from the request, and make the second call return the first call’s result rather than doing the work twice. That one piece of plumbing prevents a category of failure no prompt improvement can reach.
Two shapes of multi-agent, and what they cost
When one agent genuinely is not enough, the guide names two patterns that cover most real systems. In the manager pattern, a central agent calls specialist agents as tools and keeps control of the conversation. In the decentralized pattern, agents hand off to one another as peers, and control transfers with the handoff.
The distinction that matters operationally is who holds the context and who talks to the user. Manager keeps both in one place, which makes it easier to reason about and easier to log, at the cost of the manager’s context growing with every specialist result it absorbs. Handoffs keep each agent’s context small and specialized, which is cheaper per call, but the trail of who decided what is now spread across several runs, and reconstructing a bad outcome afterwards means stitching those together. If you cannot answer “why did the system do that” from your logs in either shape, you have built something you cannot operate, and I would fix that before adding the second agent.
Guardrails, and which ones are load-bearing
The guide’s guardrail taxonomy is sensible: relevance and safety classifiers on input, PII filtering and moderation, rules-based checks like blocklists and length limits, output validation, and risk ratings on tools. The layered framing is right, and the recommendation to start with privacy and content safety and then add guardrails in response to real failures is how this actually goes.
Worth being clear-eyed about the difference between two kinds of control in that list. Classifiers are probabilistic, which means a jailbreak classifier is a filter with a false negative rate, not a boundary. Prompt injection has been at the top of OWASP’s list for LLM applications since the list existed, and the reason it stays there is that no classifier closes it. The controls that hold are the deterministic ones: what the credential in the tool’s hand is actually allowed to do, whether the destination is on an allowlist, and whether a human approves before the irreversible step.
Which is why tool risk ratings are the most actionable idea in the guide. Rate each tool low, medium, or high on read-only versus write access, reversibility, the permissions it requires, and financial impact, then let those ratings drive behavior: extra checks before high-risk calls, escalation to a human, or refusal outside a window. That is a property of your tool registry rather than your prompt, which means it survives model swaps, prompt edits, and the intern who rewrites the system message.
What you still have to work out yourself
Neither guide will tell you how to know your agent got worse. Evals are mentioned as the way to choose a model, which is the easy half. The hard half is a regression suite of real cases with known-good outcomes, run on every prompt and model change, because “it seems better” is how agents quietly degrade for a month before someone notices. Build that before your second model upgrade, not after.
The rest of it is ordinary engineering that the agent framing tends to obscure: timeouts, retries, idempotency, permissions scoped to the task, structured logs of every tool call with inputs and outputs, and a kill switch that does not require a deploy. The model is the new part. Everything around it is the part you already know how to build, and it is still the part that decides whether this thing can run unattended.