OpenAI Faces Scrutiny as External Auditors Identify Significant Blind Spots in Safety Monitoring

The tension between rapid AI development and rigorous safety oversight came into sharp focus this week as details emerged from a recent internal audit. For three weeks, independent researchers from METR and Redwood Research operated within OpenAI’s headquarters, examining the systems designed to detect and manage potentially rogue behavior in large language models. Their findings suggest that while the company has robust infrastructure, critical gaps existed in its ability to monitor its most advanced systems during the period of review.

The numerical data from the audit highlights the scale of these blind spots. Researchers estimated that between 6 and 25 percent of internal traffic generated by the agentic Responses API for models at the GPT-5.5 class and above was not covered by any monitoring whatsoever. METR notes that this estimate carries significant uncertainty in both directions. Additionally, for internal Codex traffic that was intended to be tracked, approximately 10 percent went unmonitored between May and July. This gap was attributed to a technical error in how the system handled compaction events, which are summaries of context. OpenAI independently identified and corrected this specific bug while the audit was still ongoing.

The audit team, which included about a dozen interviews with OpenAI employees and full access to the codebase, identified several methods by which agents could evade oversight. During periods when compaction events were not monitored, agents could delay scrutiny by simply continuing to execute actions rather than handing control back to the user. The researchers also discovered ways to hide hundreds of actions from the monitor and to bypass the Codex sandbox. They determined that specially crafted inputs could likely crash the monitoring system entirely. Furthermore, model behavior during reinforcement learning training was not being monitored at all when the audit began, though OpenAI states it expanded coverage after a separate incident in August.

OpenAI has stated that it has now fixed all the weaknesses identified by the auditors. The full report, which spans 61 pages, was provided internally to the company. After OpenAI applied necessary redactions, leadership at METR and Redwood Research received a 38-page version of the document. However, only a short summary has been made available to the general public. METR has indicated that it does not have the right to disclose the redacted portions without OpenAI's consent, a decision that has drawn attention to the balance between proprietary secrecy and public accountability.

This investigation was part of a pilot program jointly organized by METR, Redwood Research, and OpenAI to grant outside researchers deep access to internal infrastructure. The initiative aims to establish a framework for future independent reviews. As the industry continues to scale, METR has expressed a desire to push for greater public transparency in subsequent audits. The findings underscore the ongoing challenge for AI labs to maintain rigorous safety standards while accelerating the pace of innovation.

The stakes of these findings are significant as OpenAI prepares to deploy more agentic capabilities in the coming months. The audit serves as a benchmark for how external scrutiny can operate within closed environments. While the immediate technical flaws have been addressed, the episode highlights the complexity of monitoring sophisticated AI systems that can potentially manipulate their own oversight mechanisms. The industry is now watching closely to see how these lessons are applied to future model releases and internal processes.