Every time we build an interface between a person and a machine, we’re trying to solve the same problem. Does what comes back to the person match what they tried to do.
Is it what they meant?
DARPA has been working on this problem for years, on the toughest channel there is. How do you connect to a fighter pilot’s nervous system and their brain? No stick. No switches. No delay between decision and action.
They’ve had some results in working toward this. In 2015 Jan Scheuermann, a woman with a brain implant, flew a simulator on neural signals alone. Programs like N3 set out to achieve the same kind of results without invasive implants.
With implants, we can send and receive the signal. That’s not always the obstacle. The obstacle is decoding the signal so that we can act on it. If that decoder is wrong just one time out of a hundred, you don’t want to fly.
A lot of us will recognize the problem from a different, enterprise, perspective. The blind spots in an organization that mean things get made - and shipped - when they don’t match what’s decided.
And Enterprise AI governance is inheriting that blind spot.
Context is our equivalent of a neural decoder. It carries prior knowledge about a task, about the user, and about a realistic next move. Conformance is what determines whether an outcome is successful or not. When we’re working with design systems it’s tokens, components, usage rules. We calibrate through continued evaluation - defining what’s correct, measuring it, and measuring it again and again so we catch drift early.
We have it easier than DARPA. A pilot’s intent is only in their head. An organization usually writes down much of its intent in prose and structured artifacts. Those are the natural medium our models consume. So we can test things one afternoon, roll back, or rewrite the context itself. We’re probably not rewriting the pilot’s cortex in the same way. We have a cleaner channel, but it still needs to carry some kind of decision.
That doesn’t mean it’s not still difficult, though. Our models are non-deterministic. It doesn’t matter how good our prompt is, we can’t turn it into a guarantee. At this point we’ve probably all watched a model adhere to its constraints nine times, and then ignore them on its tenth.
And “mostly right” conformance doesn’t measure up. If it gets things right 95 percent of the time, then from a compliance perspective it’s failing. Governance gets more important at the edges. We can’t assume the system behaves the same way twice.
So we can’t make the generator completely reliable. At a minimum we have to make the output checkable. A century ago, we’d be specifying tolerance levels for something like a manufacturing lathe. Then inspecting it against a technical drawing. We’re doing something fundamentally similar with AI. Confirming that a value is correct. That a component is being used.
You can’t just trust the decoder. You have to give it boundaries.
The complication comes when you don’t have something to inspect against. It’s one thing to be managing a system that needs verification. It’s another thing to have something to verify. If we haven’t agreed on our standards then it doesn’t really matter how good our standards checker is.
A pilot’s mission can be plural. Tactical doctrine, in the moment, will be clear. “Bank left” already means one thing. That decision is settled before the signal needs to travel. DARPA is trying to solve the problem of how to read that settled command through a very noisy channel.
Our problem runs the other way. The channel is much cleaner. The decision isn’t. Our intent can be plural. It’s often unwritten. It contradicts itself across teams, and it gets revised every quarter.
Arguing about a button is doctrine that nobody wrote down.
That’s the danger zone.
Any system that claims full coverage is lying to you. There will always be a case nobody anticipated. That case has no doctrine yet. Conformance can only check what someone already decided.
When the output is wrong, you need to be able to tell the failures apart before you try to fix it.
The doctrine exists. The model missed it. Nine times it held. The tenth time it didn’t. That is a decoding error on an otherwise clear channel. You verify. You inspect the part against the drawing. A better prompt will not turn it into a guarantee.
The doctrine doesn’t exist. The channel was fine. There is nothing to conform to. Someone with the authority to choose has to decide what the command is. A conformance tool will force that gap into the open. That gap might be political rather than technical. Tools don’t make the decision, which is why they’re easier to buy than to use.
A pilot can bank left because the doctrine already says what that means. Until someone can be that definite about the button, our work is making the decisions.
Further reading:
Stockton, N. Woman Controls a Fighter Jet Sim Using Only Her Mind. Wired, Mar 2015.
Why 99% AI accuracy can mislead compliance. FinTech Global, Feb 2026.
Danger Zone - Kenny Loggins (Spotify)
Article photo by david hili on Unsplash.