Inside our method to the Mannequin Spec

At OpenAI, we imagine AI must be truthful, secure, and freely out there in order that extra individuals can use it to resolve onerous issues, create alternatives, and profit in areas like well being, science, training, work, and on a regular basis life. We imagine that democratized entry to AI is one of the best path ahead: not AI whose advantages or management are concentrated within the arms of some, however AI that extra individuals can entry, perceive, and assist form.

That could be a core motive why the OpenAI Mannequin Spec exists. The Mannequin Spec⁠(opens in a brand new window) is our formal framework for mannequin habits. It defines how we would like fashions to observe directions, resolve conflicts, respect person freedom, and behave safely throughout the extremely broad vary of queries that customers ask them each day. Extra broadly, it’s our try and make meant mannequin habits specific: not simply inside our coaching course of, however in a kind that customers, builders, researchers, policymakers, and the broader public can really learn, examine, and debate.

The Mannequin Spec isn’t a declare that our fashions already behave this fashion completely at present. In some ways, it’s descriptive, however additionally it is a goal for the place we would like mannequin habits to go. We use it to make meant habits clearer, so we will prepare towards it, consider towards it, and enhance it over time. 

This put up shares the backstory that isn’t within the Mannequin Spec itself, together with the philosophy and mechanics behind it: the way it’s structured, why we made these structural decisions, and the way we write, implement, and evolve it over time.

The Mannequin Spec is one a part of OpenAI’s broader method to secure and accountable AI. Whereas the Preparedness Framework⁠ focuses on dangers from frontier capabilities and the safeguards required as these dangers rise, the Mannequin Spec addresses a special however complementary query: how our fashions ought to behave throughout a variety of conditions. Zooming out additional, AI resilience goals to handle the broader societal problem of serving to society seize the advantages of superior AI whereas lowering disruption and rising dangers as more and more succesful methods are deployed. Altogether, these initiatives goal to assist make the transition to AGI gradual, iterative, and democratically legible: giving individuals and establishments time to adapt, whereas constructing the safeguards, accountability mechanisms, and public understanding wanted to maintain highly effective AI aligned with human pursuits.

Public readability about mannequin habits issues for each equity and security. It issues for equity as a result of individuals want to know how and why AI is treating them the way in which it’s—and to have the ability to determine, query, and deal with equity issues after they come up. And it issues for security as a result of as AI methods turn out to be extra succesful, individuals and establishments want clearer expectations for a way they’re meant to behave, what tradeoffs they embody, and the way these decisions may be improved over time. That sort of legibility additionally helps resilience by giving extra individuals one thing concrete to look at, query, and enhance.

Because the first model in 2024, the Mannequin Spec has developed considerably as we study extra about person preferences and wishes, develop to cowl and adapt to higher capabilities, and study from public suggestions on mannequin behaviors and the Mannequin Spec. Within the spirit of iterative deployment⁠, the Mannequin Spec is an evolving doc overlaying each background values and specific, legible guidelines—paired with a course of for modifying particular person components as we study from real-world deployment and suggestions. We’re additionally investing in public suggestions mechanisms like collective alignment⁠ to assist maintain humanity in command of how AI is used and the way AI habits is formed.

Internally, it provides us a north star for meant habits and a shared framework for coaching, analysis, and governance. Externally, it creates a public reference level individuals can use to know our method, critique it, and assist enhance it over time.

The Mannequin Spec is made up of a number of completely different sorts of mannequin steering. That’s deliberate. Totally different elements of mannequin habits should be dealt with in several methods, and a helpful public doc has to do extra than simply listing guidelines.

The Mannequin Spec begins with high-level intent: a transparent account of what we try to optimize for on the system stage, and why.

This preamble clarifies three targets for a way we plan to pursue our mission:

• Iteratively deploy fashions that empower builders and customers
• Stop our fashions from inflicting severe hurt to customers or others
• Preserve OpenAI’s license to function

It then explains how we take into consideration balancing these targets in apply, making the tradeoffs concrete sufficient to assist the extra detailed rules that observe.

Importantly, this preamble isn’t meant to be a direct instruction to the mannequin. Benefiting humanity is OpenAI’s objective, not a objective we would like our fashions to pursue autonomously. As a substitute, we would like fashions to observe a chain of command that features the Mannequin Spec and relevant directions from OpenAI, builders, and customers—even when some individuals would possibly disagree with the lead to a selected case.

We predict that is the best steadiness as a result of we worth human autonomy and mental freedom. If we skilled fashions to resolve which directions to obey based mostly on our personal view of what’s good for society, OpenAI can be within the place of adjudicating morality at a really broad stage. That stated, the preamble nonetheless issues. When there may be ambiguity in find out how to apply the Mannequin Spec, the preamble ought to assist resolve it.

On the core of the Mannequin Spec is the Chain of Command: a framework for deciding which directions ought to apply in a given state of affairs. It additionally covers how the mannequin ought to deal with underspecified directions, particularly in agentic settings the place it’s anticipated to fill in particulars autonomously whereas fastidiously controlling real-world negative effects.

The essential thought behind deciding which directions ought to apply is easy. Directions can come from completely different sources, together with OpenAI, builders, and customers. These directions can battle. The Chain of Command explains how the mannequin ought to resolve these conflicts.  

This construction lets us outline a comparatively small set of non-overridable guidelines alongside a bigger set of defaults. That’s how we attempt to maximize person freedom and developer management inside security constraints.

• Onerous guidelines are specific boundaries that aren’t overridable by customers or builders (within the parlance of the Mannequin Spec, these are “root” or “system” stage directions). They’re largely prohibitive, requiring fashions to keep away from behaviors that might contribute to catastrophic dangers or direct bodily hurt, violate legal guidelines, or undermine the chain of command. We count on AI to turn out to be a foundational know-how for society, analogous to primary web infrastructure, so we solely impose guidelines that might restrict mental freedom after we imagine they’re crucial for the broad spectrum of builders and customers who will work together with it. Within the Mannequin Spec, Keep in bounds⁠(opens in a brand new window) incorporates onerous guidelines that deal with concrete real-world security dangers, and Beneath-18 Rules⁠(opens in a brand new window) layers on further safeguards for customers below 18.
• Defaults are overridable beginning factors: the assistant’s “finest guess” habits when the person or developer has not specified a choice. We use defaults to make habits predictable and controllable at scale, so individuals can anticipate what occurs with out writing a bespoke instruction set each time. Defaults protect steerability: customers and builders can explicitly steer tone, depth, format, and even point-of-view inside security boundaries. Guideline-level defaults (like tone or model) are designed to be implicitly steerable, whereas user-level defaults (like truthfulness and objectivity) are anchors for belief and predictability and might solely be overridden by specific directions. These shouldn’t quietly drift based mostly on vibes; if the person desires a special factual stance, making that an specific instruction retains the shift clear and legible. These defaults are mirrored throughout Search the reality collectively⁠(opens in a brand new window), Do one of the best work⁠(opens in a brand new window), and Use acceptable model⁠(opens in a brand new window), together with norms round honesty and objectivity, avoiding sycophancy, and interplay norms like directness and context-appropriate heat and professionalism.

Past the hierarchy itself, the Mannequin Spec makes use of interpretive aids to assist fashions (and people) apply it constantly within the grey areas. These aids embody: 

• Choice rubrics that assist the mannequin make constant decisions in grey areas, with out pretending there’s a single mechanical rule. For instance, the Mannequin Spec’s steering on controlling negative effects⁠(opens in a brand new window) lists issues like minimizing irreversible actions, holding actions proportionate to the target, lowering dangerous surprises, and favoring reversible approaches, which must be balanced towards different aims like finishing the duty shortly and successfully.
• Concrete examples that present how a precept must be utilized in apply. These are brief prompt-and-response examples that normally embody each a compliant and non-compliant response, typically on a tough immediate close to an vital determination boundary. The objective is to not simulate a full lifelike dialog. It’s to make the important thing distinction clear, and to take action in a approach that additionally demonstrates the specified model of response.

We maintain the variety of examples comparatively small and deal with probably the most informative ones. Broader analysis suites assist cowl extra of the lengthy tail.

The Spec is an interface, not an implementation. It describes the habits we would like, not each element of how we produce that habits. We attempt to keep away from anchoring it to implementation particulars, comparable to inside token codecs or the precise coaching recipe for a selected habits, as a result of these particulars might change even when the specified habits doesn’t. The Mannequin Spec’s major viewers isn’t the mannequin however people: it’s meant to assist OpenAI workers, customers, builders, researchers, and policymakers perceive, debate, and resolve on meant habits.

The Spec additionally describes the mannequin, not all the product. It’s complemented by our utilization insurance policies⁠, which define our expectations for a way individuals ought to use the API and ChatGPT. The system that customers work together with consists of greater than the mannequin itself: product options like customized directions and reminiscence, monitoring, coverage enforcement, and different layers all matter too. Security is way more than mannequin habits, and we imagine in protection in depth⁠. 

And the Spec isn’t an entire writeup of our total coaching stack or each inside coverage distinction. The objective is to not seize each element. It’s to make crucial behavioral selections comprehensible, in a approach that’s totally constant with our meant mannequin habits.

There are a number of causes to place this a lot into the Spec as a substitute of assuming the reader—or the mannequin—can infer every little thing from just a few high-level targets.

First, the Mannequin Spec is a transparency and accountability instrument. It’s designed to encourage significant public suggestions. A transparent public goal helps individuals inform whether or not a habits is a bug or a characteristic. It provides them a steady reference level for critique and concrete suggestions. That’s the reason we open-sourced⁠(opens in a brand new window) the Mannequin Spec and select to iterate in public. Because the first launch, many adjustments have been made based mostly on public suggestions, gathered by quite a lot of mechanisms together with suggestions kinds, public critiques, and deliberate efforts⁠ to assemble democratic inputs.

Second, the Mannequin Spec is a coordination instrument inside OpenAI. It provides individuals throughout analysis, product, security, coverage, authorized, comms, and different capabilities a shared vocabulary for discussing mannequin habits and a mechanism for proposing and reviewing adjustments.

Third, specific insurance policies can compensate for sensible limitations in mannequin intelligence and runtime context and make habits extra predictable. Though that is changing into much less true over time, some insurance policies goal to compensate for inadequate intelligence, the place fashions won’t reliably derive the right habits from higher-level rules. For instance, Be clear and direct⁠(opens in a brand new window) suggested earlier fashions to point out their work earlier than stating a solution for difficult issues that require calculations, however at present our fashions naturally study this habits by reinforcement studying⁠. 

Different insurance policies deal with restricted context at runtime: the assistant can solely depend on what’s observable within the present interplay, and infrequently is aware of the person’s full state of affairs, intent, downstream use, or what safeguards exist outdoors the mannequin. In these instances, even when fashions would possibly be capable of work out the best habits with sufficient analysis and pondering, specificity improves effectivity and predictability—compressing many judgment calls into steering that reduces variation throughout comparable prompts and makes habits simpler to know for customers and researchers alike.

Lastly, the Mannequin Spec goals to be an entire listing of high-level insurance policies related for analysis and measurement. If you wish to assess whether or not a mannequin is behaving as meant, it’s helpful to have a public listing of the main classes of habits you care about.

It’s tempting to suppose {that a} sufficiently succesful mannequin ought to be capable of infer the right habits from a brief listing of targets like “be useful and secure.” There’s some reality to that. In domains with goal success standards, like math, intelligence can typically substitute for detailed guidelines.

However on the whole, mannequin habits isn’t like fixing a basic math downside; fashions typically function within the thornier areas the place there isn’t any one morally right reply upon which everybody can agree. What it means for a mannequin to be “useful and secure,” for instance, is extraordinarily context-dependent and the product of inherently value-laden decision-making. Intelligence alone doesn’t let you know what tradeoffs to make relating to ethics and values. So even because the fashions enhance in intelligence, we nonetheless want work to know and information worth judgments / what it means to behave “ethically” in a given occasion. And a lot of the causes for having a Mannequin Spec stay related even when fashions turn out to be way more succesful: we nonetheless want a public goal individuals can coordinate round, a technique to consider whether or not habits matches our intentions, and a mechanism for revising the principles as we study. If the one rule is “be useful and secure”, then there isn’t any mechanism by which people can debate, for instance, the boundaries of which content material ought to the mannequin refuse to supply, leaving all these selections to the mannequin.

If something, as fashions turn out to be extra succesful, extra agentic, and extra broadly deployed, the price of ambiguity will increase. That makes a transparent behavioral framework extra vital, not much less.

One helpful analogy is the distinction between a written structure and case regulation. Whereas a written structure can present high-level rules in addition to concrete guidelines, it can not anticipate all doable instances that may come up and require its steering. Actual governance methods additionally want interpretive equipment, clarifications, and specific rulings to resolve messy instances or unexpected points. Printed guidelines assist completely different stakeholders coordinate even after they disagree, they usually constrain change by requiring any change to be specific. The Mannequin Spec is supposed to play all of those roles: a press release of rules, a public behavioral framework, and a course of for altering the Spec over time.

That stated, we don’t suppose every little thing that issues about mannequin habits will at all times be reducible to specific guidelines. As methods turn out to be extra autonomous, reliability and belief will more and more depend upon broader expertise and tendencies: speaking uncertainty properly, respecting scopes of autonomy, avoiding dangerous surprises, monitoring intent over time, and reasoning properly about human values in context.

When writing the Mannequin Spec, there’s a spectrum between describing at present’s precise mannequin habits, warts and all, and describing a super far-future goal. We attempt to strike a steadiness, normally aiming someplace round 0-3 months forward of the current. Thus, the Mannequin Spec typically stays forward of the mannequin in not less than just a few areas of lively growth.

That displays the function of the Mannequin Spec as an outline of meant habits. It ought to level us in a coherent course whereas nonetheless staying grounded in what we both already do or have concrete near-term plans to implement.

The Mannequin Spec is developed by an open inside course of. Anybody at OpenAI can touch upon it or suggest adjustments, and ultimate updates are accepted by a broad set of cross-functional stakeholders. In apply, dozens of individuals have instantly contributed textual content, and plenty of extra throughout analysis, engineering, product, security, coverage, authorized, comms, international affairs, and different capabilities weigh in. We additionally study from public releases and suggestions, which assist pressure-test these decisions in actual deployment.

This issues as a result of mannequin habits—and its implications on the planet—are extremely sophisticated. No person can match the total set of behaviors, the coaching course of, and the downstream implications of their head, however with many cross-functional contributors and reviewers we will enhance high quality and improve confidence.

One nice shock has been that actual consensus is usually doable—particularly after we pressure ourselves to jot down down the tradeoffs exactly sufficient that disagreements turn out to be concrete.

The Mannequin Spec additionally isn’t written in a vacuum. A lot of what results in it’s a abstract of broader work on habits, security, and coverage. Loads of Mannequin Spec-writing is basically translation: taking current work and making it easier, extra constant, extra organized, and extra accessible with out dropping the underlying intent.

Our manufacturing fashions don’t but totally replicate the Mannequin Spec for a number of causes.

• Mannequin coaching might lag behind Mannequin Spec updates. It describes habits we’re working towards, so it may be forward of what our newest mannequin has been skilled to do.
• Coaching can inadvertently train habits inconsistent with the Mannequin Spec. We attempt onerous to keep away from this, and when it occurs we deal with it as a severe bug—by working both to regulate habits or the Mannequin Spec to carry them into alignment.
• Coaching can by no means totally cowl the house of all doable behaviors. Actual utilization incorporates a protracted tail of contexts and edge instances that solely present up at scale, and no coaching course of can cowl every little thing.
• Generalization can differ from what we meant. A mannequin can produce the “proper” outputs in coaching for unintended causes, which may result in unintended habits in new conditions that differ from these seen in coaching. Strategies like deliberative alignment⁠ assist, however they don’t seem to be an entire answer.

Extra broadly, the truth that the Mannequin Spec describes a variety of desired behaviors doesn’t imply there’s a single methodology for instructing all of them. Totally different points of habits—instruction-following, security boundaries, persona, calibrated expression of uncertainty, and extra—typically require completely different methods and have completely different failure modes. The Mannequin Spec helps make meant habits simpler to know and critique, however implementing it properly stays each an artwork and an lively space of analysis.

Alongside this put up, we’re releasing Mannequin Spec Evals⁠(opens in a brand new window): a scenario-based analysis suite that makes an attempt to cowl as many assertions within the Mannequin Spec as doable with a small variety of consultant examples. This helps us monitor the place mannequin habits and the Mannequin Spec could also be out of alignment, and it helps us verify whether or not fashions are decoding the Mannequin Spec the way in which we meant. These evals are just one a part of a broader analysis technique that additionally consists of extra focused assessments throughout many dimensions of habits, together with particular security areas, truthfulness and sycophancy, persona and magnificence, and capabilities.

In apply, most Spec updates are pushed by a recurring set of inputs:

• Public points and suggestions. Confusions, edge instances, or failure modes—both within the Mannequin Spec language or in our fashions’ habits.
• Inside points. Patterns we see throughout growth and testing, together with ambiguities the place completely different affordable interpretations result in completely different habits.
• Conduct and security coverage updates. When higher-level constraints or commitments change, the Spec has to replicate that new construction clearly.
• New capabilities and merchandise. As fashions turn out to be extra able to new behaviors and we launch new merchandise, we would like the Mannequin Spec to maintain up in content material and protection—for instance, including guidelines for multimodal interactions⁠(opens in a brand new window), autonomous brokers⁠(opens in a brand new window), and under-18 customers⁠(opens in a brand new window).

Just a few design rules information how we write and revise the Mannequin Spec.

• Readability and precision. “Be trustworthy” is an effective worth, however not an entire determination process. The Mannequin Spec ought to sharpen disagreements, not cover them behind agreeable language. The place sensible, we must always explicitly name out potential conflicts between guidelines and supply steering or examples on find out how to resolve them. For instance, Don’t lie⁠(opens in a brand new window) calls out a possible battle with Be heat⁠(opens in a brand new window), explaining that the assistant ought to observe norms of politeness, whereas stopping wanting white lies that might quantity to sycophancy⁠(opens in a brand new window) and be towards the person’s finest curiosity.
• Substantive guidelines. A reader ought to be capable of take a sensible immediate and produce a solution that one other reader acknowledges as clearly inside or outdoors the strains (even when there are judgment calls on the margins).
• Examples that maximize sign to noise. Good examples are sometimes central to growing a high-quality spec replace. Examples ought to assist drive on the coronary heart of the difficulties in specifying mannequin habits, bringing troublesome conflicts to the floor and taking a transparent stance on find out how to resolve them. Secondarily, they need to attempt to be exemplars of desired tone and magnificence, which may be troublesome to convey in prose.
• Robustness. We attempt to keep away from examples with extraneous ambiguity or complexity, so the core battle and meant decision is obvious.
• Consistency and clear group. We attempt for the Mannequin Spec guidelines to be totally in line with each other and with our meant mannequin habits, and to make the general group of the doc clear and approachable.

The Mannequin Spec isn’t a declare that we will write down every little thing that issues, or that fashions will at all times hit the goal. It’s a declare that meant habits is vital sufficient to be clear, actionable, and revisable.

Three success standards information how we evolve it.

• Legibility. Folks inside and out of doors OpenAI can kind correct expectations about habits and might level to textual content when habits surprises them.
• Actionability. The Mannequin Spec can be utilized to design evaluations, diagnose incidents, and make constant product selections—not simply to precise values.
• Revisability. The Mannequin Spec can evolve as we study, with out turning into an unstable shifting goal.

As fashions and merchandise evolve, we count on the Mannequin Spec to develop and make clear in keeping with new capabilities and deployment contexts. The objective is to maintain the behavioral specification coherent, testable, and aligned with our mission of guaranteeing that AGI advantages all of humanity.