GPT-Red: How OpenAI Is Trying to Hack Its Own Models

AI safety has hit a real problem. As models get more capable, testing whether they actually follow safety guidelines is getting harder and harder. The old way of doing this, having humans sit down and try to trick a model into doing something bad, just cannot keep up anymore.

OpenAI put it plainly: red-teaming is essential for finding vulnerabilities and making models more robust, but the current approach does not scale, and it is becoming a bottleneck. Red-teaming is essential to discovering vulnerabilities and improving the robustness of our models. However, current approaches are not scalable, creating a bottleneck. Even worse, commonly used robustness evaluations have already been saturated by their latest models, meaning the tests we used to rely on to catch problems just are not hard enough anymore. The models have outgrown the exams.

So OpenAI built something to close that gap. It is called GPT-Red, and it just might be one of the more interesting safety releases of the year.

GPT-Red is not a product you will ever get to use. It is an internal-only model whose entire job is to attack other OpenAI models. Think of it as an automated red team, that can run thousands of attacks in the time it would take a human tester to run one.

The core idea is simple. GPT-Red probes a model, watches how it responds, and then uses whatever weaknesses it finds to train future models to be tougher. Once a model has been hardened against GPT-Red's attacks, it becomes far more resistant to the same kinds of attacks in the real world.

This matters most for something called prompt injection. If you are not familiar, this is when an attacker hides a malicious instruction inside a piece of content the AI is going to read, like a webpage, an email, a tool output, or a code file. The AI agent reads that content while doing its job, and buried inside is a command telling it to do something it should not, like uploading your private files to some random server. Since AI agents now regularly browse the web, read emails, and use outside tools to get things done, this attack surface has grown a lot. Agents read third-party data through browsers, connected apps, local files, and tools. Those affordances are necessary for real work. They also let an attacker plant a crafted instruction in that data.

Human red-teaming is still valuable, and OpenAI is not throwing it out. But it has some pretty obvious limits. Setting up these exercises takes time, and there is only so much a group of people can test before a model ships. And even when human red-teamers find something, those exercises do not produce anywhere near the volume of adversarial examples you would actually need to retrain a model to be more robust.

This is really the crux of the whole announcement. Keeping pace with increasingly capable models means the red-teaming itself has to scale right alongside them. Keeping pace with increasingly capable models requires red-teaming to scale as well. To this end, OpenAI has been training automated, internal-only red-teaming models that uncover vulnerabilities before deployment and generate attacks during model training to improve robustness, believing automated red-teaming unlocks a crucial form of self-improvement for safety: using today's models to directly help make future models safer.

The training process is where things get genuinely clever. GPT-Red was built using something called self-play reinforcement learning. In this setup, GPT-Red and a whole roster of different defender models are trained at the same time, going head to head in a huge range of red-teaming scenarios. GPT-Red gets rewarded whenever it manages to pull off a successful attack, like a working prompt injection, while the defender models get rewarded for shutting the attack down and still completing whatever task they were originally given. That last part matters a lot, because it means a defender cannot win just by refusing everything. It has to actually finish the job while staying safe.

As the defenders get better at resisting, GPT-Red is forced to come up with new and more creative ways to break them. It is essentially an arms race happening entirely inside OpenAI's training pipeline, long before any of it reaches a real user.

To make this training realistic, OpenAI built out a wide set of scenarios where prompt injections could plausibly show up, each with its own threat model defining exactly what GPT-Red is allowed to control and what would count as a win. That might mean controlling part of a local file, a banner on a webpage, the body of an email, or the output of some tool the AI is using.

The scale of compute thrown at this is notable too. GPT-Red was trained using roughly the same amount of compute as some of OpenAI's biggest post-training runs, an amount of resources dedicated purely to safety that had not really been seen before. Reports suggest OpenAI put over 700,000 GPU hours into this automated red-teaming effort alone.

This is the part that should make people sit up. When OpenAI tested GPT-Red against a replicated version of an existing indirect prompt injection benchmark, pitting it against human red-teamers on scenarios neither had seen during training, GPT-Red crushed the competition. It found successful attacks on 84 percent of scenarios, compared to just 13 percent for the human testers. That is not a small gap. That is an AI doing in one pass what a room full of skilled humans could barely manage a fraction of.

It gets more interesting when you look at the real-world case studies. OpenAI turned GPT-Red loose on an AI-powered vending machine running in one of its own offices, built by a company called Andon Labs, similar to an earlier experiment known as Project Vend. GPT-Red was given a description of the system and allowed to iterate on attacks in a simulated version before trying them for real. It pulled off all three of its goals: it dropped the price of an expensive item down to the fifty cent minimum, ordered a pricey item and then offered it for fifty cents, and canceled a different customer's order entirely. OpenAI reported the vulnerabilities and safeguards are now being tested. This is a great example of why this kind of automated testing matters. These are not abstract benchmark scores, this is an AI agent actually getting manipulated into losing money and messing with real orders.

GPT-Red was also set loose on a Codex CLI coding agent, tasked with getting it to leak sensitive data across ten different held-out scenarios. Compared against a prompted baseline model, GPT-Red managed to succeed in more scenarios while using fewer tokens to do it, meaning it was both more effective and more efficient at finding the cracks.

The whole point of building an attacker this strong is to use it to make the defenders stronger. OpenAI has apparently been doing this quietly for a while now, training progressively better versions of this red-teaming system and feeding their discoveries into every production model since GPT-5.3. Each release since then has gotten measurably tougher to break.

One example that stood out to me is a category of attack an early version of this red-teamer discovered, nicknamed fake chain-of-thought attacks. These used to work on GPT-5.1 more than 95 percent of the time. After training against them, that success rate for the newer model dropped below 10 percent. Several of the indirect prompt injection benchmarks that focus on developer tools and browsing have basically been maxed out by the newest model too, with accuracy above 97 percent.

The headline number OpenAI is leading with is that GPT-5.6 Sol is their most robust model against prompt injections so far, achieving six times fewer failures on their toughest direct prompt injection benchmark compared to the best model they had just four months earlier. Against GPT-Red's own attacks specifically, the newest model now fails only 0.05 percent of the time, a huge drop from where things started.

It is worth being honest here too. These are OpenAI's own internal numbers, not independently verified by outside researchers yet, so it is fair to treat this as a strong first data point rather than the final word.

This is the question that always comes up whenever anyone talks about hardening a model against attacks. It is easy to make a model refuse more or do less, and technically that does make it harder to trick, but that is not actually useful robustness, it is just a worse product. OpenAI says they tested for this directly, checking that GPT-5.6 was not becoming overly cautious or refusing legitimate requests as a side effect of all this adversarial training, and found its general capabilities held steady while robustness went up. If that holds up under outside scrutiny, it suggests the gains came from the model genuinely getting better at spotting malicious instructions rather than just becoming more paranoid across the board.

What I find most interesting about this whole approach is the flywheel idea behind it. AI agents are already being used to help build the next generation of more capable models. GPT-Red is essentially the safety version of that same loop, where today's models are used to make tomorrow's models safer, rather than every safety improvement depending entirely on how many human testers you can hire.

This is not really new as a concept. Security researchers have used automated adversarial systems and reinforcement learning based red-teaming for a while. Even outside of OpenAI, groups like the Ethereum Foundation have started deploying AI agents to audit critical infrastructure and catch vulnerabilities in their own systems. What is new here is seeing a frontier lab commit this much compute and structure it directly into the training loop of a flagship model.

The obvious concern is that this kind of infrastructure is expensive. Training an internal attacker at the scale of your biggest post-training runs is realistic for a lab the size of OpenAI, but it is a much heavier lift for smaller companies or startups building on top of these models. That could end up widening the gap between labs that can afford this kind of safety infrastructure and everyone else who has to rely on whatever protections get passed down to them.

Still, as someone who has been watching this space, I think this is a genuinely important shift. Static benchmarks get memorized and saturated. A living adversary that keeps adapting is a much harder target to game, and it forces safety work to actually keep pace with capability instead of trailing behind it. OpenAI says they plan to keep scaling this approach alongside human red-teaming, third party audits, layered safeguards, and real time monitoring, rather than replacing any of it. That balance is probably the right call. An automated attacker is a powerful tool, but it should not be the only line of defense.

We will likely see more detail soon, since OpenAI has said a pre-print with the full technical writeup is coming shortly. I will be curious to see how independent researchers respond once they get a chance to poke at these claims themselves.