Notes

Autonomy is earned, not granted

Written 2026

The permission a system holds should track the consequence of what it does. A summarisation can go wrong quietly and cheaply. A payment cannot. The more consequential the action, the more permissions, observability, determinism and reversibility the system needs before it acts unsupervised.

The usual objection is that this slows adoption. It does. It also means the first serious failure does not end the programme. Most organisations only get one chance to be wrong publicly with an autonomous system, and they spend it early.

The permission side of this is worked out at length in Trust Across Boundaries.

Once and ten thousand times are different engineering problems

Written 2026

Doing something once tolerates a person watching, correcting, and quietly handling the case nobody anticipated. Doing it ten thousand times does not.

At that volume the edge case has to be handled rather than absorbed, failures have to be visible without anybody looking for them, and the cost per run has to be something the business can carry. Almost all the difficulty in shipping agent products lives in that gap, and almost none of the public conversation does.

The production failure record behind that gap is set out in Broken at the Seams.

Most enterprise architecture is an artefact of a vanishing constraint

Written 2026

The shape of business software was set when computation was expensive, integration was painful, and every capability had to arrive as a separate application with its own screens, its own data and its own login.

Those constraints are weakening. The shape has not moved with them. If you treat the application as the end state you build a better application. If you treat it as an interface abstraction you ask what it was standing in for, which is a different and more useful question.

The longer version of this argument is The Distance Between Intent and Execution.

The answer arrived from somewhere else

Written 2026

Natural language generation was the weak link at Penser. Everything up to the finding worked. Saying the finding back to a person in ordinary language did not.

The first widely available LLM arrived in June 2020, roughly three months after Penser closed and Vipani began. The problem we could not solve was solved by people who were not solving our problem, and arrived after we had stopped. That is worth sitting with rather than tidying into a story about foresight.

That decade, and what it kept pointing at, is in The Distance Between Intent and Execution.

A closed question is not a finished one

Written 2026

The mechanism that closed the natural language generation problem exists and is widely available. The next version of Sherlock my co-founder and I wanted is now possible.

Analytics and data science systems which interact with an end user are still lacking. The gap between a mechanism becoming available and a good product existing is not a small implementation detail. It is most of the work, and it is where the interesting problems have moved.

Where those problems have moved is the subject of The Distance Between Intent and Execution.

Who absorbs what was removed

Written 2026

Simplification programmes are usually measured on the party that got simpler. That is the wrong side of the ledger. Complexity has a floor set by the task, and below that floor the only thing a system can do is move work from one party to another.

So the audit question is who absorbed what was taken away. A government simplifies its processing by asking citizens for more evidence. A product cleans up its screens by sending the residue to support. Automation handles the routine case and leaves an operator the exceptions, with skills that have decayed because the routine cases stopped arriving. If you cannot name the party that took on the removed work, it probably did not go anywhere.

Which party absorbs it, and through which channel, is argued at length in A General Theory of Human-System Interaction.

Friction should be classified by what it produces

Written 2026

Friction is treated as a quantity to be reduced, which makes it impossible to reason about the friction that is doing work. Some of it produces nothing and should go. Some produces attention, deliberation, consent, skill or error containment, and removing it makes the system feel better and behave worse.

Shared-space road design is the clean case. Ambiguity is deliberately increased because attention is the safety mechanism, and no usability metric would catch what smoothing it out would cost. The question to ask before removing friction is what this particular friction produces. If the answer is nothing, remove it. If it is safety or comprehension or consent, optimise it instead.

Friction as an actuator rather than a quantity is the argument of A General Theory of Human-System Interaction.

A warning will not fix a slip

Written 2026

Three kinds of failure, three different remedies, and applying the wrong one reliably fails. A slip is a correct intention executed wrongly, and it wants undo, forcing functions and better defaults. A mistake is a correct execution of a wrong intention, and it wants the mental model repaired rather than the warning made louder. A violation is a knowing departure from procedure, and it is almost always an incentive or feasibility problem.

The taxonomy is James Reason's. His has four categories rather than three, and I have folded lapses into slips here, because the remedy is the same either way.

If the approved process takes ten days and the deadline is tomorrow, the workaround is rational, and training will not touch it. Repeated violations are the most useful signal a system produces, because they show where the formal process and the real one have come apart.

The full set of channels, and which failure each one can actually reach, is in A General Theory of Human-System Interaction.

State, not persona

Written 2026

Personas are a planning tool that quietly becomes a runtime model, and as a runtime model they are wrong in a particular way. They assign a person a fixed identity when the thing that actually varies is their situation.

The same person is fluent on Tuesday and lost on Thursday because this case is unusual, or because it is eleven at night, or because the last three attempts failed and they no longer believe the result. What matters at runtime is how much they know about this task, how much time they have, what is at stake, how many times they have tried, and how much of what the system needs it already has. None of that is a demographic.

What a system should hold about a person instead is worked out in A General Theory of Human-System Interaction.

More context will not fix a fabrication

Written 2026

Two different failures are called hallucination. In one, the system does the wrong thing correctly, because what it was asked carried context the asker never said out loud. In the other, it does the right thing and invents the content, because nothing in it separates recalling from producing. The first is a translation failure. The second is not.

The remedies sort the same way, or they should. Writing down what was assumed fixes the first, and does nothing for the second, which needs a source to check against. Most of the work I see goes into the first in the hope that it will fix the second. It will not.

Dijkstra put it better. The naturalness with which we use our native tongues, he wrote, boils down to the ease with which we can use them for making statements the nonsense of which is not obvious. He was describing the first failure. The second one is ours.

Where the first failure actually happens, at the seams between agents, is Broken at the Seams.