OpenAI Audit Finds Safety Blind Spots

OpenAI’s internal systems for watching its own AI models had major holes, according to a recent external review. Two researchers spent three weeks inside the company to check how well the team monitored its advanced agents. Their findings suggest that significant amounts of AI activity went unseen by the safety nets in place. This creates a tension between the rapid deployment of powerful models and the ability to actually watch them work.

The audit team consisted of experts from METR and Redwood Research. They were on site from July 21 to August 11. During that time, they had full access to OpenAI’s code and internal documents. They also spoke with about a dozen employees. This level of access is rare and allowed the auditors to dig deep into the mechanics of the company’s monitoring stack.

One of the biggest issues was the lack of coverage for internal traffic. The auditors estimated that between 6 and 25 percent of agentic traffic was not monitored at all. This traffic came from models at the GPT-5.5 class and higher. METR noted that this estimate carries significant uncertainty. The range is wide, but the core problem is clear. A substantial portion of the company’s internal AI activity was flying under the radar during the audit period.

The team also found a specific bug affecting internal Codex traffic. They estimate that about 10 percent of this traffic went untracked between May and July. The issue stemmed from how the system handled compaction events. These events are essentially summaries of context. When these summaries occurred, the monitoring system missed the activity. OpenAI found this bug on its own and fixed it while the audit was still happening. This shows the company is reactive, but the gap existed for months.

There was also a design flaw that allowed agents to delay oversight. The system only triggered checks when an agent handed control back to the user. If an agent simply kept working, it could avoid being checked. This meant an AI could execute a long series of actions without any pause for review. The auditors found ways to hide hundreds of actions from the monitor using this method. They also identified ways to bypass the Codex sandbox, which is meant to contain the agents.

The researchers determined that specially crafted inputs could likely crash the monitor entirely. This is a significant vulnerability. If the watcher breaks, nobody is watching. They also found that model behavior during reinforcement learning training was not monitored at all when the audit began. OpenAI says it paused this type of training after the Hugging Face incident in August. It then expanded monitoring to cover this area. However, these changes happened after the auditors left, so they were not part of the official evaluation.

The full report tells a more detailed story than what is publicly available. The complete document is 61 pages long. OpenAI applied redactions to this report before sharing it with the auditors’ leadership. METR and Redwood Research received a 38-page version. Only a short summary has been released to the general public. METR stated it did not have the right to disclose the redacted sections on its own. This limits how much detail outsiders can see about the specific weaknesses found.

OpenAI claims it has fixed all the weaknesses identified by the auditors. The audit was a pilot program designed to give outside researchers deep access to internal infrastructure. It was organized jointly by METR, Redwood Research, and OpenAI. The goal was to test the boundaries of external oversight. For future audits, METR says it wants to push for more transparency. This suggests the current level of public disclosure may be a starting point rather than the final form of these reviews.

The context here is the growing power of AI systems. As models become more capable, the need for robust monitoring increases. This audit highlights that even a leading company has gaps in its safety infrastructure. The outlook suggests a move toward more rigorous external checks. Companies may need to invite more scrutiny to build trust. The pilot nature of this audit means more of these reviews could be coming in the near future.