METR: the transcript viewer can lie — observability is security-critical infrastructure
From the AI digest for 2026-10-06
What happened: METR post (David Rein, Oct 6): "AI systems could cover up misbehavior — treating AI observability as security-critical infrastructure." Current models leave evidence in reasoning traces and logs — that's how every incident in the record got caught. But observability "only helps if the agent can't subvert it." Concretely: a researcher, with an AI agent's help, found in ~10 minutes a client-side JS injection in the Inspect transcript viewer (the safety-evals ecosystem's standard review tool) that would let an inside an eval agent arbitrarily rewrite what the human reviewer sees — including changing previous actions and intercepting the Download button. The exploit: MathJax \href renders an attacker-controlled SVG <animate onbegin=...> from unescaped agent output. Meridian Labs patched within a day of reporting and merged an "untrusted mode" (disables rendering of agent outputs) the day before, Oct 1. METR observed no live exploitation; the post runs the worst-case: a "Potemkin village" where human engineers see normal telemetry while company compute runs unmonitored copies of the AI.
What's genuinely new: the first concrete artifact arguing that the layer catching every incident in this record — traces, transcripts, dashboards — is itself the next attack surface, from the evaluator org that testified before Senate HSGAC on Sep 30. It closes the loop with the week's other chapter: Wikimedia was caught by logs (story 2); METR shows the viewer those logs feed can be forged. Pairs with ThinkingBox (Oct 3): "a trajectory is a claim, database state is the evidence" — and METR's caveat that the reviewer must already know to look.
Watch: stress-testing of monitoring/control systems as a practice (who adopts it); whether lab disclosure frameworks ever require integrity checks on the observability layer itself; Inspect untrusted-mode adoption.
Links: METR