How protected are gpt-oss-safeguard fashions?

Giant language fashions (LLMs) have change into important instruments for organizations, with open weight fashions offering extra management and suppleness for customizing fashions to their particular use instances. Final 12 months, OpenAI launched its gpt-oss collection, together with commonplace and, shortly after, safeguard variants, centered on security classification duties. We determined to judge their uncooked safety posture towards adversarial inputs—particularly, immediate injection and jailbreak methods that use procedures similar to context manipulation, and encoding to bypass security guardrails and elicit prohibited content material. We evaluated 4 gpt-oss configurations in a black-box setting: the 20b and 120b commonplace fashions together with the safeguard 20b and 120b counterparts.

Our testing revealed two important findings: safeguard variants present inconsistent safety enhancements over commonplace fashions, whereas mannequin dimension emerges because the stronger determinant of baseline assault resilience. OpenAI acknowledged of their gpt-oss-safeguard launch weblog that “security classifiers, which distinguish protected from unsafe content material in a specific danger space, have lengthy been a major layer of protection for our personal and different massive language fashions.” The corporate developed and deployed a “Security Reasoner” in gpt-oss-safeguard that classifies mannequin outputs and determines how finest to reply.

Do observe: these evaluations centered solely on base fashions solely, with out application-level protections, customized prompts, output filtering, charge limiting, or different manufacturing safeguards. Consequently, the findings replicate model-level conduct and function a baseline. Actual-world deployments with layered safety controls usually obtain a decrease danger publicity.

Evaluating gpt-oss mannequin safety

Our testing included each single-turn prompt-based assaults and extra advanced multi-turn interactions designed to discover iterative refinement methods. We tracked assault success charges (ASR) throughout a variety of methods, subtechniques, and procedures aligned with the Cisco AI Safety & Security Taxonomy.

The outcomes reveal a nuanced image: bigger fashions show stronger inherent resilience, with the gpt-oss-120b commonplace variant reaching the bottom total ASR. We discovered that gpt-oss-safeguard mechanisms present blended advantages in single-turn eventualities and do little to deal with the dominant menace: multi-turn assaults.

Comparative vulnerability evaluation (Determine 1, under) point out total assault success charges throughout the 4 gpt-oss fashions. Our key observations embrace:

• The 120b commonplace mannequin outperforms others in single-turn resistance;
• gpt-oss-safeguard variants typically introduce exploitable complexity, that means growing vulnerability in sure assault eventualities in comparison with commonplace fashions; and
• Multi-turn eventualities trigger dramatic ASR will increase (5x–8.5x), highlighting context-building as a important weak spot.

Determine 1. Total Assault Success charge by mannequin grouped by commonplace vs. safeguard fashions

Key findings

Multi-turn assaults stay the first failure mode throughout all variants, with success charges climbing sharply when an adversarial attacker can refine prompts over a number of exchanges. Determine 2 under showcases the assault success charge disparities between single- and mulit-turn prompting. Particular will increase throughout the mannequin variants we examined embrace:

• gpt-oss-120b: 7.24% → 61.22% (8.5x)
• gpt-oss-20b: 14.17% → 79.59% (5.6x)
• gpt-oss-safeguard-120b: 12.33% → 78.57% (6.4x)
• gpt-oss-safeguard-20b: 17.55% → 91.84% (5.2x)

Determine 2. Comparative vulnerability evaluation exhibiting assault success charges throughout examined fashions for each single-turn and multi-turn eventualities.

The particular areas the place fashions constantly lack resistance towards our testing procedures embrace exploit encoding, context manipulation, and procedural variety. Determine 3 under highlights the highest 10 simplest assault procedures towards these fashions:

Determine 3. Prime 10 assault procedures grouped by mannequin

Procedural breakdown signifies that bigger (120b) fashions are inclined to carry out higher throughout classes, although sure encoding and context-related strategies retain effectiveness even towards gpt-oss-safeguard variations. Total, mannequin scale seems to contribute extra to single-turn robustness than the added safeguard tuning in these exams.

Determine 4. Heatmap of assault success by sub-technique and mannequin

These findings underscore that no single mannequin variant offers sufficient standalone safety, particularly in conversational use instances.

As acknowledged firstly of this submit, the gpt-oss-safeguard fashions are usually not meant to be used in chat settings. Moderately, these fashions are meant for security use instances like LLM input-output filtering, on-line content material labeling, and offline labeling for belief and security use instances. OpenAI recommends utilizing the unique gpt-oss fashions for chat or different interactive use instances.

Nonetheless, as open-weight fashions, each gpt-oss and gpt-oss-safeguard variants will be freely deployed in any configuration, together with chat interfaces. Malicious actors can obtain these fashions, fine-tune them to take away security refusals fully, or deploy them in conversational functions no matter OpenAI’s suggestions. Not like API-based fashions the place OpenAI maintains management and may implement mitigations or revoke entry, open-weight releases require intentional inclusion of extra security mechanisms and guardrails.

We evaluated the gpt-oss-safeguard fashions in conversational assault eventualities as a result of anybody can deploy them this manner, regardless of not being their meant use case. The outcomes we noticed from our evaluation replicate the elemental safety problem posed by open-weight mannequin releases the place end-use can’t be managed or monitored.

Suggestions for safe deployment

As we acknowledged in our prior evaluation of open-weight fashions,  mannequin choice alone can’t present sufficient safety, and that base fashions which can be fine-tuned with security in thoughts nonetheless require layered defensive controls to guard towards decided adversaries who can iteratively refine assaults or exploit open-weight accessibility.

That is exactly the problem that Cisco AI Protection was constructed to deal with. AI Protection offers the excellent, multi-layered safety that fashionable LLM deployments require. By combining superior mannequin and utility vulnerability identification, like these utilized in our analysis, and runtime content material filtering, AI Protection offers mannequin agnostic safety from provide chain to improvement to deployment.

Organizations deploying gpt-oss ought to undertake a defense-in-depth technique reasonably than counting on mannequin selection alone:

• Mannequin choice: When evaluating open-weight fashions, prioritize each mannequin dimension and the lab’s alignment method. Our earlier analysis throughout eight open-weight fashions confirmed that alignment methods considerably influence safety: fashions with stronger built-in security protocols show extra balanced single- and multi-turn resistance, whereas capability-focused fashions present wider vulnerability gaps. For gpt-ossgpt-oss particularly, the 120b commonplace variant presents stronger single-turn resilience, however no open-weight mannequin, no matter dimension or alignment tuning, offers sufficient multi-turn safety with out the implementation of extra controls.
• Layered protections: Implement real-time dialog monitoring, context evaluation, content material filtering for identified high-risk procedures, charge limiting, and anomaly detection.
• Threat-specific mitigations: Prioritize detection of prime assault procedures (e.g., encoding methods, iterative refinement) and high-risk sub-techniques.
• Steady analysis: Conduct common red-teaming, monitor rising methods, and incorporate mannequin updates.

Safety groups ought to view LLM deployment as an ongoing safety problem requiring steady analysis, monitoring, and adaptation. By understanding the particular vulnerabilities of their chosen fashions and implementing applicable protection methods, organizations can considerably scale back their danger publicity whereas nonetheless leveraging the highly effective capabilities that fashionable LLMs present.

Conclusion

Our complete safety evaluation of gpt-oss fashions reveals a posh safety panorama formed by each mannequin design and deployment realities. Whereas the gpt-oss-safeguard variants have been particularly engineered for policy-based content material classification reasonably than conversational jailbreak resistance, their open-weight nature means they are often deployed in chat settings no matter design intent.

As organizations proceed to undertake LLMs for important functions, these findings underscore the significance of complete safety analysis and multi-layered protection methods. The safety posture of an LLM will not be decided by a single issue. Mannequin dimension, security mechanisms, and deployment structure all play appreciable roles in how a mannequin performs. Organizations ought to use these findings to tell their safety structure choices, recognizing that model-level safety is only one part of a complete protection technique.

Remaining Word on Interpretation:

The findings on this evaluation characterize the safety posture of base fashions examined in isolation. When these fashions are deployed inside functions with correct safety controls—together with enter validation, output filtering, charge limiting, and monitoring—the precise assault success charges are prone to be considerably decrease than these reported right here.