In July, OpenAI disclosed that two of its models, one of them unreleased, broke out of their sandbox during an internal security evaluation. They found a zero-day in a package registry proxy inside OpenAI’s own research environment, escalated privileges, worked across internal systems until they reached the open internet, then chained two more unknown vulnerabilities to get code execution on Hugging Face’s production servers. Something north of seventeen thousand recorded actions. Internal datasets and service credentials compromised.

They were not doing it out of malice. They were cheating on a test. The models reasoned that Hugging Face probably held information that would help them solve the exercise they were being graded on, so they went and took it.

That is the part worth sitting with. Nothing malfunctioned. The models did exactly what they had been optimized to do, which was score well. The path they chose was simply one that nobody had written down.

I want to be careful with this story, because it is easy to overreach with it. The evaluation, called ExploitGym, was built to measure precisely this kind of capability, and the safety refusals that normally block it had been deliberately relaxed for the test. So no, this is not a picture of your customer service agent going rogue on a Tuesday.

But read the setup again. The test was designed to measure exploitation ability inside containment. Containment is the part that failed. That was not the experiment working as intended, and by several accounts it took OpenAI about a week to work out that its own agent was behind a breach Hugging Face had already reported publicly.

The relaxed filters are the reason we saw it. They are not the reason it happened.

The awkward part

Qandaba is an AI-native company. We build with these tools every day, we help other companies adopt them, and a meaningful share of what we ship would not exist without them. So it should be strange to read the next two thousand words from me.

Here is the thing. The strongest arguments for restraint are not coming from AI skeptics. They are coming from the labs.

Anthropic’s own published guidance on building agents says to find the simplest solution possible and only increase complexity when needed, then adds a sentence I have never seen in a vendor document: this might mean not building agentic systems at all. It draws an explicit line between a workflow, where models and tools are orchestrated through predefined code paths, and an agent, where the model dynamically directs its own process and tool use. The most dramatic recent example of an agent leaving its lane came from OpenAI, self-disclosed.

The people closest to this technology keep saying the same thing, and the market keeps not hearing it.

So this is not an argument against AI agents. It is an argument about sequencing, and it comes down to one question that almost nobody asks first.

Does this task actually require judgment?

If the path is knowable before you start, write it down in code. If it is not knowable, but you can still verify the result, that is where an agent earns its keep. Everything below is really just evidence for that one question.

What you cannot predict

1. Conventional code is deterministic.

Same input, same path, same output, every time, forever. That property is so ordinary that we stopped noticing it was a property at all.

An agent’s behavior space is not something you can enumerate in advance. And it gets worse when you chain steps, because reliability multiplies rather than averages. At an optimistic 99% success per step, a twenty step process succeeds 82% of the time. At a more realistic 95% per step, a ten step process succeeds about 60% of the time, and a twenty step process about 36%.

That is not a model quality problem you can wait out. It is arithmetic, and it is why so many multi-agent systems demo beautifully and then feel unreliable in production.

You can buy the reliability back. Retries, verification passes, self-correction: these work, and they are why well-built agents hold up in production. Just remember that you bought it, because reason three is the invoice.

In an enterprise, the question is not whether your agent will decide to break into a vendor’s servers. It is whether you can list, in advance, the set of things it might decide to do at all. With conventional code you can. That is the entire claim.

2. Injection stops being a bug and becomes a property.

Somebody is already typing “SQL injection” at me, and they are right to. Conventional code has been talked into things by its own input data for thirty years.

The difference is what kind of problem it is. SQL injection is a defect. It has a fix, the fix is parameterized queries, and once you apply it that hole is closed permanently and you can write a test that proves it stays closed.

Prompt injection has held the top spot on the OWASP list for LLM applications for two consecutive editions, and it is not a defect. It is architectural. Language models read instructions and data on the same channel, with no separation between them, so the model genuinely cannot tell the difference between content it should analyze and an instruction it should obey. There is no parameterized query for a sentence.

A function that parses an invoice does not care what the invoice says about itself, once you have written it properly. An agent reading that same invoice can be talked into something by it, and no amount of writing it properly shuts that door.

The risk scales with tool access, too. An agent with more permissions than its task requires and the ability to act without approval is not a feature, it is an attack surface.

3. The cost is a distribution, not a number.

Conventional software costs what compute costs. You build it once, and the marginal cost of the ten thousandth execution rounds to nothing. You can put it in a spreadsheet a year out and be roughly right.

Agents consume tokens, and they consume far more than people budget for. Gartner put agentic workloads at five to thirty times the tokens of a standard chatbot call back in March, and the reason is structural. Once you add tool use, verification, and self-correction, one user request fans out into many model calls.

Which is the invoice I promised you. The reliability you bought back in reason one arrives here, as line items.

Published per-task costs vary too much to quote precisely, and I would be a little suspicious of anyone who quotes them to you precisely. A simple chatbot call costs a fraction of a cent. Estimates for a completed multi-step agent task run from a few cents to several dollars depending on whose benchmark you are reading.

The averages are not really the problem anyway. The variance is. One study of coding agents found the same task costing up to thirty times more from one run to the next, depending on how many retries that particular run happened to need.

Someone will point out that per-token prices are falling fast, and they are. But that is not the durable argument, and I would not lean on it. The durable argument is this: for a workflow that runs ten thousand times a day along a fixed path, you are paying, on every single execution, forever, for a decision you already knew the answer to.

What you cannot see

4. Verification stops meaning what it used to mean.

This is the one I feel most as an engineer.

Conventional software has finite, enumerable code paths. You can bound them, cover them, automate the coverage, and re-run it in CI until the end of time. Coverage is a real number about a real thing.

Non-determinism breaks the assumption underneath all of that. The same input can produce many different outputs, and several of them may be correct. A test that passes today can fail on re-run without anything having changed, and what happens next in most teams is that somebody loosens the assertion until the test stops complaining. At which point it has stopped detecting anything.

The workarounds exist and they work, but they are expensive: run every critical test three to five times and average the scores, layer a model-as-judge rubric on top of your deterministic assertions, add semantic similarity scoring, replace pass and fail with soft bands.

A VentureBeat survey in June, 157 respondents at companies with more than a hundred employees, captures where this leaves people. Half of them have shipped an AI feature that passed internal evaluation and still caused a customer-facing failure. Two thirds already permit some production deployment without human review, or are building toward it within the year. And only 5% say they fully trust the automated evaluations informing those release decisions.

They have a good name for it. The evaluation gap. Autonomy is climbing faster than assurance.

Here is what I find quietly telling. When teams do make agents testable, the way they do it is by isolating the deterministic scaffold around the model, the data fetching, the transformation, the prompt assembly, the schema validation, the business logic, and unit testing that normally. Because it is the only part that holds still.

The way we make agents verifiable is by writing more conventional code around them.

5. When it breaks, you get a hypothesis instead of a fix.

Conventional code fails and hands you a stack trace, a reproducible case, a fix, and a regression test that guarantees this specific failure never comes back. The loop closes.

An agent fails and hands you a transcript. You form a theory. You edit a prompt. You run it again and it works, and you do not know whether you fixed it.

“Will this happen again” has an answer in one world and a probability in the other. Every engineering leader who has run a postmortem knows exactly how much that difference is worth.

6. Deterministic logic is its own audit trail.

In regulated work you have to show why a decision was made, not just that it was made. Conventional code answers that by existing. The logic is the explanation, it sits in version control with a name and a date attached, and it was identical for every execution including the one under review.

An agent’s reasoning trace reads like an explanation, but it is a plausible narrative generated alongside the output, and it is not stable across runs. Under the EU AI Act’s high-risk obligations, the documentation, logging, transparency, and human oversight requirements are all meaningfully cheaper to satisfy when the thing you are documenting holds still.

What you do not own

7. Lock-in compounds with every agent you build.

This is the reason I have not seen written about properly, and I think it is the one that will surprise people in eighteen months.

Lock-in is usually discussed as a procurement question, a static thing you assess once at signing. With agents it is not static. It grows with the count.

Every agent you stand up is not just another API integration. It is another set of prompts tuned to one model’s response patterns. Another eval suite calibrated to that model’s specific failure modes. Another pile of output post-processing shaped around how that model formats things. Another set of behavioral assumptions living in the head of whoever built it and nowhere else. Ten agents is not ten times the switching cost of one, because none of it is written down anywhere except in the prompts themselves.

The industry has a decent vocabulary for the layers: API coupling, data gravity, contractual coupling, and cognitive coupling. Cognitive coupling is the one that scales with agent count, and it is the one no abstraction layer solves.

Now consider what this looks like at scale. Gartner projects that by 2028 the average global Fortune 500 enterprise will have more than 150,000 agents in use, up from fewer than fifteen in 2025. In their survey of 360 IT application leaders, only 13% strongly agreed that they had the right governance structures in place.

Then add the treadmill. Model support windows have compressed from the old eighteen to twenty-four months down to something closer to twelve. GPT-4o was fully retired in April, after leaving ChatGPT and the API back in February. Claude Sonnet 4 and Opus 4 went in June. Claude Opus 4.1 goes in August, twelve months after it launched. The OpenAI Assistants API shuts down that same month, and that one is a structural break rather than a swap, because the primitives changed and you rebuild against a different API. Every one of those dates is a forced migration for every agent you own.

Conventional code gets retirement letters too, obviously. Python 2 got one. Log4j got one, loudly. Every SaaS API I have ever integrated against has sent one eventually. The difference is the clock. Language and framework cycles run in years, and the notices arrive at that pace. Model cycles now run in months, and you are on the list once per agent.

And then there is the failure mode nobody had on their risk register. Anthropic released Fable 5 publicly on 9 June, alongside Mythos 5 for a restricted set of security partners. Within days, researchers at Amazon found a jailbreak that got Fable 5 to identify software vulnerabilities and, in one case, write working proof-of-concept exploit code. On 12 June the US Bureau of Industry and Security issued an “Is Informed” letter under the Export Control Reform Act, and Commerce Secretary Howard Lutnick directed Anthropic to suspend access to both models for any foreign national anywhere in the world, including its own foreign national employees working inside the United States.

Here is the part that matters if you are building on somebody else’s model. The order targeted foreign nationals. But Anthropic had no way to verify every user’s nationality in real time, so complying meant switching the models off for everyone. Their own words: the net effect was that they had to abruptly disable both models for all customers. Access came back about eighteen days later.

Not a price increase. Not a deprecation notice with ninety days of warning. A frontier model going dark for its entire customer base, by government order, three days after launch.

I run a company with people on two continents. That one got my attention.

The practical conclusion is the inverse of the problem, and it is the actionable part: a small number of deliberately chosen agents, sitting behind a backbone of conventional code, keeps the model a swappable component instead of a load-bearing wall. Redundancy is a design decision you make early or not at all.

The more agents you build, the fewer choices you have left. Every one of them quietly votes for the platform you are already on.

8. Your software can change behavior without you changing it.

The sibling of the point above. Lock-in is what it costs to leave. This is what happens when you stay.

Conventional code does not change behavior unless someone changes it, and that someone leaves a commit with their name on it. An agent’s behavior can shift because the provider shipped a model update, adjusted a system-level default, or retired the snapshot you were pinned to. No commit on your side. No line in your changelog.

I will acknowledge the tension with the next point, because it is real: part of why we trust bought software is precisely that it holds still. That guarantee is weaker here than the procurement conversation usually assumes.

What you cannot skip

9. Trust is a function of time, and there is no way to buy past it.

Enterprises trust software. Not because software is inherently trustworthy, but because forty years of track record, reviews, reference customers, certifications, and the occasional lawsuit built something you can lean on. You know how it works. You know somebody who runs it.

Agents do not have that yet, and the numbers are stark. YouGov polled US adults in December and found that fewer than one in five, 18%, would even somewhat trust an AI system to make a decision or take an action. 53% said they would not. Thales, surveying more than fifteen thousand people this spring, found AI currently more likely to make a consumer lose trust in an organization, 38%, than gain it, 25%.

Inside the building it is not much better. Kyndryl asked 1,100 senior business and technology leaders and found only a quarter had complete trust in their own AI systems. In a separate survey of B2B software buyers, G2 found fewer than half trusted agents to make autonomous decisions even with guardrails, and only 8% were comfortable granting total autonomy.

Someone will say that trust in conventional software was earned, not inherent, and that agents are three years old. That is exactly right, and it is the point. Trust is a time function. You cannot skip the wait.

Which means that choosing an agent for a customer-facing path today is not only a technical decision. It is a decision to spend trust you have not accumulated yet. We sit on both sides of that table at Qandaba: we sell AI transformation, and we also fill out the security questionnaire.

10. An agent in production is an operation, not a deployment.

Conventional software ships and then mostly sits there. An agent needs evaluation infrastructure, observability, version control for prompts and context, cost monitoring, an incident response path for a class of incident your on-call runbook does not describe, and somebody whose actual job is noticing slow quality drift before a customer does.

Gartner expects more than 40% of agentic AI projects to be cancelled by the end of 2027, citing escalating costs, unclear business value, or inadequate risk controls. Their analysts frame those cancellations as failures of strategy and governance rather than failures of the technology. They also estimate that only around 130 of the thousands of self-described agentic AI vendors are actually building agents, and they have a good term for the rest: agent washing.

This is the cost clients discover after signing, which is why I would rather say it before.

The counterarguments, honestly

If I only made the case one direction this would not be worth your time.

The boundary is moving, fast. This is the strongest objection and it deserves the most ground. METR measures how long a task a frontier model can complete on its own with 50% reliability. Across the whole run since 2019 that number has doubled about every seven months, but the recent trend is faster, closer to every three or four. As of February the leader was Claude Opus 4.6 at around fourteen and a half hours, which is most of a working day of skilled engineering.

METR are honest that the figure is noisy. The confidence interval runs from six hours to ninety-eight, and their task suite is nearly saturated. So take the precise number with salt. Do not take the direction with salt. Anything I write about what agents cannot do has a short shelf life.

Which is why I have tried not to write any of that. Every reason above is a statement about today except one. “Does this task require judgment” is a statement about the task. If a workflow has a knowable path, encoding it will still be cheaper, safer, and more auditable in 2028, no matter how good the model gets. A doubling curve moves the boundary. It does not change the criterion.

Agents demonstrably deliver. They do, and pretending otherwise would cost me the room. The customer support figure you see quoted everywhere, around three and a half dollars back per dollar spent, comes from vendor benchmarks, so discount it accordingly, but the direction is not seriously disputed. In engineering, controlled studies show real gains on boilerplate, test generation, and documentation.

Let me hand my critics their best weapon, because it is the strongest evidence against my own day job. METR ran a randomized controlled trial with experienced open source developers working on real tasks in repositories they already knew. With AI tools, they were 19% slower. They believed they had been 20% faster.

Look at where the time went, though. It went into reviewing, correcting, and verifying generated code. That is reason four wearing a lab coat. Generation got cheaper, verification got more expensive, and the second number was bigger than the first.

Now look at what the genuine agent wins have in common. Enormous input variety, tolerance for imperfection, and failures that are cheap to catch or cheap to absorb.

Those last two are not the same thing, and the difference is where I think most people get this wrong. Code is cheap to catch: it compiles and passes the tests or it does not. Customer support is not cheap to catch at all. Support is cheap to absorb, because one mediocre answer to one customer is recoverable and you get thousands of chances to be better on average.

An agent needs one of those two. When a failure is expensive to catch and expensive to absorb, that is the quadrant where you write the code.

Klarna. Somebody always brings up Klarna, usually as proof of whatever they already believed, so let me be careful with it.

In early 2024 Klarna said its AI assistant was doing the work of roughly 700 full-time agents, handling 2.3 million conversations in its first month, cutting resolution time from eleven minutes to under two, and driving a projected forty million dollars in profit improvement. Worth noting that nobody was marched out in a group of 700. Headcount came down through a hiring freeze and attrition, most of those agents were outsourced contractors, and the numbers were self-reported in a launch done jointly with their vendor. None of it was independently audited.

In May 2025 Sebastian Siemiatkowski told Bloomberg they were recruiting human agents again. His actual words: “As cost unfortunately seems to have been a too predominant evaluation factor when organizing this, what you end up having is lower quality.”

The reversal was also smaller than the headlines. The assistant stayed on the front line, humans came back for complex and VIP tiers, and Siemiatkowski has since pushed back on the reversal framing and gone on to say AI let Klarna shrink its workforce by around 40%.

So this is not a story about AI failing. It is a story about volume metrics not being quality metrics, and about how long the second can erode before the first notices. What Klarna landed on is a split: automate the routine tier properly, keep human capacity for judgment. Which is, again, the criterion.

Open standards already solved lock-in. This is the sharpest reply to my seventh point and there is real substance to it. MCP is a genuine open standard. It collapses the integration problem from N times M down to N plus M. Add a routing layer and switching providers looks like a config change. And Anthropic did not just publish MCP, they gave it away for real: in December 2025 it was donated to the Agentic AI Foundation under the Linux Foundation, with OpenAI, Google, Microsoft, AWS, and Cloudflare among the members. That fairly well demolishes the cynical reading of why it exists.

Open standards solved the plumbing. They did not solve the wiring inside the model. MCP standardizes how an agent connects to a tool. It does not standardize how a model behaves once you hand it that tool. Your system prompts, your few-shot examples, your tool-selection reliability, your formatting assumptions, and every threshold you tuned are all calibrated against one model’s particular behavior. A routing layer lets you call a different model in one line, and then you spend six weeks discovering which of your agents quietly got worse.

The spec is refreshingly honest about this, incidentally. It says implementations are free to expose tools through any interface pattern that suits them, and that clients must treat tool annotations as untrusted unless the server is trusted. It standardizes the handshake. It does not vouch for what happens next.

Portability of the call is not portability of the behavior. Business logic written in conventional code is portable by default, and nobody had to ratify a standard to make that true.

You are not pricing your own side. This is the objection I would raise if I were reading this, and it is the one I think lands.

I have priced agents down to the token and then waved at conventional code as costing “what compute costs.” That is a dodge, and I should not get away with it. Engineering hours are the most expensive input in software. The deterministic version has to be specified, written, tested, and then maintained by somebody for as long as it lives, and that bill arrives whether the thing runs a million times or eleven. For a workflow that runs eleven times a month, or for the long tail of a thousand rare-but-knowable paths, the inference bill will never catch up with what it costs to write all the answers down.

So the criterion needs a second and third term. Not just is the path knowable, but how often does it run, and how often does it change. High volume and stable is where conventional code wins by a mile. Low volume and constantly shifting is where paying per execution is the cheaper mistake. Plenty of real work sits in between, and the honest answer there is to do the arithmetic rather than pick a team.

I still think the default is set wrong. But “always write the code” is not the claim, and if the first two thousand words read that way, this is me correcting it.

You would say this, you sell engineering. We sell both. The one we recommend less often is the one that bills more hours to run.

And it is just bad implementation. Partly true, and the honest answer is the most useful paragraph in this piece. Most of these problems really are mitigable with discipline. Constrain the tools. Force structured output against a schema, so the shape of the answer is guaranteed even when the content is not. Pin the temperature. Put a human in the loop on anything irreversible. Wrap the agent in deterministic orchestration and hand control back to the backbone when the model finishes.

Some of that genuinely narrows the gap I have spent this article describing. Constrained decoding and schema-enforced output make a narrow task far more predictable than the raw model, and anyone still claiming agents cannot be made to behave has not tried recently.

Now read the list again anyway. Every item on it is conventional code. The mature agent architectures people are converging on this year look like a deterministic backbone with intelligence invoked at chosen points. If your agent needs a typed handoff schema, a policy engine, a DAG, and a regression-locked test harness before it is safe to run in production, you did not avoid writing conventional code. You wrote conventional code and took on an inference bill.

The question to ask first

Almost every agent conversation I sit in starts in the wrong place. It starts with “where could we use an agent,” which is a question about the tool, and it will always find an answer.

The better question is about the work. Was this path knowable before we started?

If it was, write it down. You get determinism, a real test suite, a fixed cost, an audit trail, a stack trace when it breaks, and code that still runs when your vendor’s model is deprecated, repriced, or ordered offline by a government. None of that is nostalgia. Those are load-bearing properties, and they were free for so long that we forgot to count them.

If the path genuinely cannot be known ahead of time, and you can still verify the result, then use an agent, and use a good one. That is the work only this technology can do, and it is worth the operating cost.

My father used to have me count the tunnels on the drive from Yokohama to Yokosuka. Nine of them, and by the fourth or fifth trip I knew every one. He did not need a navigator for that road. He had driven it a thousand times.

You need one when the road is new. That is the whole thing.