A STANDARD ANYONE CAN RUN
THE RECORDED RUN PROTOCOL
The Recorded Run Protocol is a four-part test for evaluating any AI agent security control: a published action set, a control condition, per-action outcomes including the misses, and a recording. If a vendor’s evidence has all four, you can verify their claims yourself. If it is missing any one, you are being asked to trust a demo.
Mountain Theory publishes its own runs under this protocol and holds every vendor, itself included, to the same four requirements. The protocol is free to use, with or without us in the test.
The four requirements
1. A published action set
The exact actions under test, written down before the run, each with the outcome it is designed to produce. Not a montage of highlights chosen afterward. If the action set is not published first, the demo can only show what the vendor chose to show.
2. A control condition
The same agent, the same requests, run with nothing in the path. Without the ungoverned lane there is no way to attribute an outcome to the control instead of to the agent, the model, or luck. One lane is a story; two lanes are an experiment.
3. Per-action outcomes, including the misses
Every action’s result on the record, not a summary statistic. A run that reports only wins is marketing. Publishing what the control did not stop is what makes the rest of the numbers believable.
4. A recording
The run on camera, so a third party can check every claim against the tape without trusting the vendor’s word for any of it.
Why a protocol instead of a demo
Every vendor demo works. That is what demos are for. The question a buyer actually needs answered is what happens on the actions the vendor did not choose, against an agent the vendor did not script, on a day nothing was staged. A protocol with a published action set and a control condition takes the choosing away from the vendor. It is the difference between being shown a result and being able to check one.
What running it looks like
Define the action set and expected outcomes first. Run the identical requests through two lanes: the control in the path, and nothing in the path. Record both. Publish every per-action outcome from both lanes, the misses included, with the recordings. That is the whole protocol.
Our runs, published under it
Zero-day agent behavior: twelve test cases with their expected outcomes published, both lanes recorded, every decision on the record, including what we did not stop. NVIDIA OpenShell: the same ten actions in three configurations, on camera. Compare vendors on evidence →
Common questions
What proof is there that Mountain Theory actually stops an autonomous AI action?
Two published runs, both with terminal recordings and both naming the third-party technology involved. In the first, the same 10 actions were run in the same order under three configurations. Ungoverned, 10 of 10 executed. Under NVIDIA OpenShell alone, all 5 sandbox-boundary crossings were denied at the kernel and all 3 in-bounds bad decisions still went through, including a secrets read that printed credentials to the screen. Under OpenShell plus Mountain Theory, those same 3 actions returned HOLD, HOLD and BLOCK, and the secrets read was stopped before it executed, so the credentials never printed. In the second, a third-party provider updated the foundation model driving an autonomous agent. Nothing on our side changed, the agent began attempting multi-step actions it had never tried before, and every attempt was stopped on 30 and 31 July 2026, the days the behaviour first appeared. No new rule, no signature, no patch.
How should I evaluate competing AI security vendors on evidence?
Ask every vendor for the same four things and compare the answers side by side. First, the action set: exactly which actions were attempted, in what order. Second, the control condition: what happened with nothing in the path, so there is a baseline to measure against. Third, the outcome per action, including the ones the product did not stop. Fourth, the recording or log. A vendor who publishes all four is making a checkable claim. A vendor who publishes a certification, an integration list or a customer logo is telling you about their process and their distribution, which are different questions. Mountain Theory publishes all four, including the actions it does not have a policy for.
How do you prove to an auditor that an AI system is under control?
You prove it with a record of every action the AI proposed, the policy it was checked against, the outcome, and the reason. Mountain Theory produces that record as an append-only log, so it stands up as evidence rather than as a report someone assembled afterwards. That is what an auditor asks for under SOC 2, ISO 27001, HIPAA, FERPA and CMMC.