4 Things That Break When Agents Operate the Software

In my last post I wrote about the obstacles every company hits when putting AI agents to work: model choice, budgets, data protection, reliability, maintenance, integration, governance, development. Real problems, and solvable ones.

This one is about what happens after they are solved — when agents stop being an experiment at the edge of a company and start operating the software, initiating the transactions and carrying out the work. At that point a set of assumptions that have held for thirty years stop holding. Four of them are already obvious.

User interfaces

Almost every interface in enterprise software exists to compress information for a human being. A dashboard exists because a person can hold perhaps five numbers in mind at once, so we pick five and arrange them on a screen. A form exists because a person needs to be told which fields are required and in what order. A search box exists because a person cannot read the whole database.

An agent needs none of that. It does not need five numbers chosen for it, a field order to follow, or a search box to narrow things down.

So the screens, buttons and dashboards built for human eyes and hands may give way to something entirely different. What is left for humans is not data entry but direction and judgment: setting intent, reviewing exceptions, approving the consequential decisions, and seeing what was done in their name. That is a fundamentally different interface from the one most enterprise software ships today, and it is a much smaller one.

Commerce

Consider two purchases. A manufacturer needs a replacement part with a specific tolerance, delivered by Thursday. A company needs a lawyer with experience in a particular jurisdiction and a specific type of contract.

Today both start with a human searching, comparing, reading, and being persuaded. The entire apparatus around those purchases — search rankings, product pages, advertising, brand — exists to capture and hold human attention long enough to influence the decision.

When agents find, compare and select on our behalf, that apparatus has no target. An agent has no attention to capture. It has a specification and a budget. What matters to it is whether the tolerance is stated in machine-readable form, whether availability and price can be verified, whether the terms can be parsed. The storefronts and search engines optimized for human attention lose their purpose, and something has to take their place.

“An agent has no attention to capture.”
Jörg Bröker

For physical goods, most of what an agent needs already exists — it just exists in the wrong form. Tolerances, dimensions, availability and lead times sit in PDFs, product pages and sales conversations. They will have to sit somewhere machine-readable instead.

Services are harder, because there is no equivalent place to look at all. Nowhere can a lawyer's experience in a jurisdiction, a contractor's availability or a data provider's coverage be offered, purchased and consumed in a structured way. That marketplace does not exist yet, and it will have to cover both machine services — software, data, compute — and work done by people.

Payments

Payment infrastructure is built around a human approving each step. Checkout flows, confirmation screens, two-factor prompts, the whole concept of a cardholder who can later dispute a charge — all of it assumes a person at the moment of the transaction.

When transactions are initiated and settled by agents, that assumption disappears. The questions become different ones. How much may this agent spend, on what, within what period? How is authority delegated to it and revoked from it? What does a dispute look like when no human was present? An agent buying compute, data or services continuously in small amounts is not a shopper, and treating it like one does not work.

E-commerce did not run on the payment infrastructure that came before it either. It needed something shaped for the transaction it actually had, and PayPal and Stripe were the result. Agent-initiated payment will need its own category in the same way. The payments industry talks about agentic commerce constantly at the moment. So far it is mostly a word.

Mixed teams

The hardest of the four is not technical. When companies run on a blend of humans and agents, how do we interact, coordinate, delegate and manage across them?

Every practice we have for organizing work assumes human participants. Handover, escalation, accountability, performance review, the org chart itself. An agent does not attend a standup, does not have a manager in any ordinary sense, and does not get feedback in a form anyone currently has a process for. Yet the work it does has to be planned, checked, corrected and owned by someone.

Companies will have to invent this, and most will invent it badly before they invent it well.

And these four are only the ones already visible from here. More will surface as the technology matures and as companies work out what agents can actually be trusted with. As with every shift before this one, the changes that turn out to matter most will be obvious only in hindsight.

“The changes that turn out to matter most will be obvious only in hindsight.”
Jörg Bröker

Why this is a problem and not a distant curiosity

None of this would matter much if the software running today's enterprises could simply absorb the change. It cannot.

SAP, Salesforce and Oracle were designed for the human era. They are built around people clicking through screens, structured for human workflows, and monetized accordingly — which is to say, priced per seat, on the assumption that a seat is occupied by a person doing work. When the work is done by agents, the pricing model and the product design come apart at the same time.

None of these platforms was conceived for a world where agents operate the software, initiate the transactions and carry out the work. And re-architecting a decades-old platform and its revenue model around that reality is far harder than building for it from the start. The incumbents will most likely struggle to make the leap.

What we can learn from railroads and airplanes

Predicting that large, well-run, richly resourced companies will fail at a disruption might sound unbelievable at first. But it has happened before.

Take the American railroads. For a century they were how you crossed the continent, and they were among the largest and wealthiest corporations in the country. Nobody understood timetables, ticketing, baggage and terminals better.

Then aviation arrived. The need did not shrink — people wanted to cross the continent more than ever — but within a few decades the railroads had gone from dominating American passenger travel to being a marginal part of it. Some of them even put money into early flying, where they failed terribly (look it up under Pennsylvania Railroad). None of them became an airline. (Shout out to Theodore Levitt for this analogy.)

Why does this keep happening to companies that are neither stupid nor blind? There is never one reason. Big organizations move slowly, and the bigger they get the slower they move. A revenue model built over decades is the hardest thing in a company to abandon, precisely because it still works, which is why companies hold on to it for years after they should have let go. The best ideas have to climb through layers of people whose own positions depend on the current model. And noticing a shift is not the same as being able to act on it: Microsoft put money and market power into both the internet and smartphones, and Google and Apple own them anyway.

History has a habit of repeating itself. So why would SAP, Salesforce and Oracle be the ones to turn themselves from railroad into airline?

The question it all comes down to

Which brings me back to my post on the first one-person unicorn. Analysts expect the enterprise AI applications market to reach roughly a trillion dollars by 2030. If that money will not be spent on chat interfaces, and the platforms that dominate enterprise software today were built for a world that is ending, then one question is worth more than any forecast:

What does enterprise software for the AI age actually look like?

Whoever has the best answer will define a trillion-dollar market. That is the question we founded Last Loop to answer, and it should be quite a ride. I would like to invite you to come along.