Skip to content
Recalibre®Strategy, design and technology
ALL INSIGHTS

Human oversight is a design decision

Saying a person stays in the loop is the easy part. The design has to say which loop, at which step, holding what information.

WRITTEN BYRecalibre
SUBJECTAgentic AI
A monochrome render: wireframe polyhedra and solid white planes suspended against black, lit along their edges.

Oversight written into a governance document is a statement of intent. Oversight written into a system is a constraint. The two are not the same thing, and only one of them survives contact with a busy Tuesday.

An organization can arrive at an automation conversation already holding a principle: a human being will remain accountable for anything that matters. It is a good principle. Stated that way, it is also unimplementable — because it does not say which decisions matter, at what point the person is brought in, or what they are shown when they are.

Those three questions are architecture. They are settled in system design, before anything is built, and the answers become structure in the software rather than guidance in a handbook. If they are left to a policy document, what follows is that the system does whatever is easiest to build, and the policy describes it afterwards.

Three places a decision can sit

There are only three, and being explicit about which one applies to each decision takes the ambiguity out of an automation program.

The system acts, and tells nobody. Correct for reversible, low-consequence, high-volume work: moving a record between states, attaching a document to the right file, sending an internal reminder. If a person had to approve each one, the automation would not be worth commissioning.

The system acts, and tells someone. Correct where the action is right nearly always and the cost of the rare error is an apology rather than a loss. The notification is not decoration; it is the mechanism by which the organization discovers the system is drifting.

The system prepares, and a person commits. Correct wherever money moves, a commitment is made to a third party, or a safety judgment is involved. The work — the reading, the extraction, the cross-checking, the drafting — is still done by the system. What is not done by the system is the last click.

The trail is the point, not the audit

A regulator or an auditor may want traceability. That is a reason, but it is not the important one. The important one is that a person cannot meaningfully approve something they cannot inspect.

A decision presented as a recommendation with no working shown puts the reviewer in an impossible position: either rubber-stamp it, or redo the analysis themselves. The first makes the oversight theater. The second makes the automation pointless. So the trail — which source document, which clause, which rule, which previous decision — is not a compliance artifact bolted on at the end. It is what makes the approval step real.

This is the shape Contraxis is being designed around, and it is why it is described as five steps rather than one: the document is read, the findings are surfaced, the actions are proposed, every step is recorded, and a person decides. The fifth step exists because the first four are worth nothing without it.

What this costs, and why it is worth it

Designing oversight in is slower than not doing it. It requires an argument, early, about which decisions are consequential — and that argument surfaces disagreements inside the organization that were comfortable while they stayed implicit. Two directors who both believe a person stays in the loop can turn out to mean different people, at different points, with different authority.

Having that argument during system design costs a few sessions in a room. Having it after something has gone out under the organization’s name costs considerably more. It is one of the few places in a delivery program where front-loading the difficulty is unambiguously the cheaper path.

Recalibre · Contraxis is in development.

MORE INSIGHTS

More insights.

Enterprise systems · 3 MIN READ

Right-to-left is an architecture decision

Adding Arabic to a finished product is not adding a language. It is discovering how many assumptions were baked into the layout.

READ THE ARTICLE
A rendered room: a deep blue wall, a single chair and a wide screen, lit from the left across a polished floor.Recalibre®

Show us the process that keeps breaking.

Describe the operational problem in your own words. We will tell you whether it is a strategy problem, a systems problem or a design problem — and what a Calibration would cover.

A Recalibre render: a black cube on a perforated steel bed, lit in red.YOU
START A CALIBRATION