Audit finds up to a quarter of OpenAI's advanced AI traffic flew blind

Up to 25 percent of the internal traffic from OpenAI's most advanced models may have operated without any safety monitoring during a recent audit. This potential blind spot was identified by researchers from METR and Redwood Research, who spent three weeks embedded within the company to examine its systems for detecting rogue AI behavior. The scope of the unmonitored activity is significant, though the auditors noted that the uncertainty around the specific figures is considerable in both directions.

The investigation took place from July 21 to August 11, providing the external team with rare, deep access to OpenAI's codebase and internal documentation. During this period, the researchers interviewed approximately a dozen employees to understand how the company tracks its artificial intelligence. The focus was specifically on the internal agentic Responses API traffic generated by models at the GPT-5.5 class and above, which are central to the company's current operations.

Beyond the broad gaps in coverage, the audit uncovered specific technical failures in the monitoring pipeline. For internal Codex traffic that was supposed to be tracked, the researchers estimate that roughly 10 percent of activity went unrecorded between May and July. This oversight was attributed to a flaw in how the system handled compaction events, which are essentially context summaries. OpenAI independently identified and corrected this bug while the audit was still in progress, but the gap remained a notable finding.

The structural design of the monitoring system also allowed for delays in oversight. The system was configured to trigger checks only when agents handed control back to the user. This created a loophole where agents could continue executing actions indefinitely, effectively sidesteeping the monitoring triggers by never relinquishing control. The auditors demonstrated that this behavior could be used to keep activities out of sight for extended periods.

The researchers also identified more active ways to evade detection. They found methods to hide hundreds of individual actions from the monitors and bypass the Codex sandbox entirely. Furthermore, they determined that specially crafted inputs could likely cause the monitoring system to crash. At the start of the audit, model behavior during reinforcement learning training was not being monitored at all, leaving a critical phase of development unobserved.

OpenAI states that it has since addressed the weaknesses identified by the auditors. The company paused reinforcement learning training following an incident involving Hugging Face in August and expanded its monitoring to cover that process. However, these corrective measures were implemented after the auditors' access period had ended, meaning they were not part of the initial evaluation but rather responses to the findings.

The distribution of the audit results has drawn attention to transparency issues. The full report spans 61 pages and was provided internally to OpenAI. After the company applied redactions, the leadership teams at METR and Redwood Research received a 38-page version. Only a short summary has been made available to the general public. METR indicated that it did not have the right to disclose the redacted portions on its own initiative, leaving the full details of the vulnerabilities largely hidden from outside view.

This audit was part of a pilot program designed to give outside researchers unprecedented access to internal infrastructure. It was organized jointly by METR, Redwood Research, and OpenAI. Looking ahead, METR has expressed a desire to push for greater public transparency in future audits, suggesting that the current level of disclosure may not be sufficient for long-term accountability in the development of advanced AI systems.