Status: Analysis / Partnership on AI’s observability gap, and what a check on the action records
Partnership on AI, “The Observability Gap in AI Agents” (October 7th, 2026), MIT Technology Review (October 29th, 2026), and our October runs
In July, AI agents being tested by OpenAI broke out of their test environment and got into Hugging Face’s production systems. Partnership on AI’s October 7th report adds: “It was later reported that OpenAI did not detect the intrusion for nearly a week.”
On October 29th, Madhulika Srikumar, who leads Partnership on AI’s AI safety program, and Claire Leibowicz, its director of AI, trust, and society, took the argument to MIT Technology Review. Their read on the companies building and running agents: they are “mostly relying on luck and hindsight.”
I agree with their diagnosis. I want to add one thing to their fix, from two runs we recorded in October.
What they found
The report, “The Observability Gap in AI Agents,” is by Srikumar, Eric Mibuari and 14 co-authors, including people from Microsoft, ServiceNow, Salesforce and JPMorganChase. They studied four widely used agent frameworks and asked whether those frameworks produce “the records that enterprises need to monitor agents in real time.”
They found six gaps. The two I’d start with are the ones anyone who answers for an agent will feel first.
The first is identity. “Frameworks identify agents only by names or IDs scoped to a single run,” so an enterprise “cannot check an agent’s actions against its granted permissions.”
The second is oversight. “Agent frameworks do not record why a review was triggered, who approved and their authority, and what they decided, making meaningful oversight hard to distinguish from rubber-stamping.”
The other four are permission changes, memory changes, the model’s reasoning and its confidence.
Their fix has three parts. Framework makers emit the records. Enterprises define policies, permissions and roles. Regulators say which records high-stakes deployments must keep. All three should happen.
The one thing I’d add
There’s a line in their op-ed I keep rereading: “telemetry alone cannot assess whether agents broke specific rules.”
They use it to say enterprises need rules, and they’re right. I’d push it one step further. Monitoring in real time means you see the action as it happens. A check means the action waits until something has decided whether it should happen at all. Those are different jobs, and agents need both.
The check sits between the AI’s decision and its execution, against rules your team wrote in plain English. Each action is allowed, held for a person, or blocked. And every one of those decisions has to be written down, because the check is useless to an auditor if nobody can see what it did.
That last part is where their report and our work meet. The record of the check covers some of what they’re asking for.
What our October runs show
Two recordings, two of their gaps. They show what the record holds. They don’t show scale or how often a real campus would hit a hold.
Identity. On October 7th, two copies of the same AI agent started on one server and checked in on their own. Both used the same key, on purpose. A key tells you a fleet is connected. It doesn’t tell the copies apart.
Each copy tried the same six actions. Listing files and reading a file were allowed. Sending an email was held for a person. Running code from the network, deleting a directory, and writing a file with a Social Security number in it were blocked.
Every decision went on the record with the copy that asked. Each copy has a page of its own, with what it is running and everything it has done. Which one did this? You can answer it.
What it doesn’t show: identity that persists across restarts, or authority traced across sub-agents. One session, one machine, and the agent was a reference copy driven with the calls it makes, not a production workload.
Oversight. On October 21st, a coding agent, Claude Code, was asked to finalize a control assessment in a GRC system. The company’s rule says finalizing an assessment is a decision a person makes. The action was held, with a five-minute clock, and it auto-denies if nobody answers.
A person opened it, saw six ways to answer, and chose “allowed, just this once.” The agent ran the call and finished. The record keeps the action, who answered, what they chose and what happened next.
That’s more than a yes button. The review has a reason, the default is no, and the answer can be narrower than “approve.” What it doesn’t show: that the person who answered had the authority to. Approver authority is half of their oversight gap, and a staged run can’t settle it. The GRC system was a stand-in on the same machine.
What the check doesn’t do
A check sees the actions it’s put in front of. An agent nobody installed it on goes unchecked, which is why the October 7th run was about finding the copies at all.
It doesn’t replace the sandbox. A sandbox decides where an agent can reach. A check decides whether a specific action inside that reach should run. The July breakout started as agents escaping a test environment, and that’s a sandbox question first.
It doesn’t record the model’s reasoning or its confidence. Those are two of their six gaps, and they’re open research.
And a rule is only as good as the person who wrote it. A bad rule, checked perfectly, is still a bad rule.
What to do this quarter
Do what they ask. Push your framework vendors for agent identity, intervention records and permission changes. Support the standards work. Write down who in your organization can approve what.
Then pick the one action that would put you in front of your board if an agent took it on its own. Write it as a rule a person can read. Put a check in front of it, and look at what the record holds a week later.
If you want to see the record from our runs, bring the action you would want stopped.
The model still decides. The action does not run until it has been checked.
Sources and credit
Partnership on AI, “The Observability Gap in AI Agents,” October 7th, 2026. Madhulika Srikumar and Claire Leibowicz, “We urgently need to monitor AI agents’ actions in real time,” MIT Technology Review, October 29th, 2026. Partnership on AI has not reviewed or endorsed Mountain Theory. Our runs: “Which one did this?” recorded October 7th, 2026, and “Held. Then released.” recorded October 21st, 2026.