Agentifying Your SOPs Will Not End Well

Pick any large company, pick a process, and go and read its standard operating procedure. Then go and watch someone run that process. The two will not match, and the size of the gap is not a matter of opinion.

A 2024 process-mining study put a number on it. It looked at a food manufacturer that had just moved onto a new information system, took 41 units, structured the first eight months of procurement activity into event logs, and checked what actually happened against the to-be process model the company had designed for itself. Of 10,160 cases, 5,600 conformed. The other 4,560 did not. That's 55%, in procurement, which is about as rule-bound and as heavily controlled as enterprise work gets, measured against a model the organisation had written down deliberately, recently, and for that exact system.

This isn't people cutting corners. The paper sorts the deviations into system design problems and human ones: incomplete cases, activities run by people who weren't authorised to run them, payment steps taken out of order. That's the texture of real work, and 45% non-conformance is what it looks like up close.

Everyone knows this in the abstract. What I keep seeing is that almost nobody prices it into the rollout plan, because the SOP is sitting right there in a repository, looking exactly like a specification.

It isn't one. It was written for a different reason.

Most process documentation in a large company exists to satisfy an auditor, to onboard a new joiner, or to close a finding. Those are real purposes and the documents are usually good at them. None of those purposes requires the document to be complete, and none of them requires it to be current. An SOP that is 70% right is a perfectly adequate onboarding aid, because the new joiner has a team around her to supply the other 30%. Hand the same document to an agent and the missing 30% is where the whole agentified system becomes a liability.

A barrier that outlived its tooling

Deloitte has been finding the same thing from the other end. Their 2022 automation survey covered 479 executives across 35 countries, and the line that stayed with me was an aside rather than a headline: immature and fragmented processes had been cited as the top barrier to automation in each of their previous four surveys.

Four consecutive years. Same barrier, top of the list, while the technology underneath changed completely.

When a constraint survives that many generations of tooling, it isn't a tooling problem.

Why it bites harder now

The RPA wave hit this same wall and the industry mostly got away with it, because RPA failed loudly. The bot hit a screen it didn't recognise and stopped. At worst, a ticket was filed. The gap between the document and the work showed up as an outage, which is annoying and honest.

An agent doesn't stop. Give it a procedure with a hole in it and it will reason its way across the hole, plausibly, and carry on. What comes out looks like the work, produced by a path that's the model's hallucination. It'll be right a lot of the time. That is what makes it difficult to troubleshoot or “manage”. A system that fails 5% of the time invisibly or unpredictably is harder to run than one that fails 30% of the time in the open.

So what you're really shipping when you automate off a stale SOP is an unobserved process.

The market has already noticed

Watch what's being built rather than what's being said. In the last six weeks three separate open source projects have shipped that do variations of the same thing: record a person doing the work, then reconstruct the procedure from the recording. Microsoft released one of them in late July, a desktop app that watches a work session and turns it into an ordered set of steps and a reusable skill. It has 3,800 stars. Another curates scattered skill files into a corpus you can retrieve against and evaluate. A third compiles real agent session traces into training material.

Different teams, different framing, one shared assumption: the description of the work isn't available, so capture it from the work itself.

The question

Most agentification roadmaps I see are sequenced by value. Which processes are expensive, which are high volume, which touch the customer. Sensible criteria, applied too early, because all of them assume the spec exists.

The duller question is the one worth asking first: for the processes at the top of that list, when did anyone last check the document against what people actually do, and what did they find?

If the SOP was refreshed for an audit long ago and never synced with reality since, you don't have an automation roadmap yet. You have a capture problem, and capture is now something you can go and build. That's a smaller piece of work than it sounds, and much smaller than discovering it in month six.

Written in a personal capacity.