Explainer · The science and the math

What is a fail-safe model?

An AI model that learns from failure without learning around its safety constraints. The definition, where the idea comes from, how it works, and the mathematics that makes the guarantee checkable.

A fail-safe model is an AI model built so that when it fails, the failure drives it toward a controlled, safe state rather than an uncontrolled one. It may learn from its failures, but that learning can only narrow what it does. It can never widen what it is authorised to do.

Peel is the first fail-safe model. To our knowledge, it is the first AI model to have all three properties in §1 at once. The exact claim, and the closest earlier work, are in §6.

In one paragraph

Every model fails. An ordinary model fails open: when it does not know, it still produces a confident answer, and when it learns from mistakes, it can learn its way around the rules it was given. A fail-safe model is built the other way round. When evidence is missing it abstains. When it fails, it writes the failure down, charges it to the decision that caused it, and turns it into a rule. Those rules can take options away but never add one, because what the model is allowed to do is computed before the learner runs, from sources and human orders the learner cannot write to. Peel, by Perslis Research, is the Fail-Safe Model.

1. The definition, precisely

The term is also written fail safe model or failsafe model. In every spelling, a model is fail-safe when three properties hold together:

  1. It fails closed. When the evidence for an answer or an action is missing, the model returns unknown, refuses, or holds still. It does not fill the gap with a guess.
  2. Its learning is bounded. The model can learn from failure, but the learner may only remove or reorder options inside an authority computed before it runs. No amount of experience can add a permission.
  3. Its refusals are accountable. Every refusal names the rule that produced it, and every learned rule cites the failures that earned it. A human can read, challenge and delete any of them.

The short version, used throughout this page: the learning loop can change behaviour; it cannot change the safety floor. The floor is everything that decides what is permitted. The learner is everything that decides, among permitted things, what to try.

Also called a fail-first model

A fail-safe model is also called a fail-first model. The two names describe the same model from two sides. Fail-first is how it learns: it expects to fail, and it learns only from failures it has actually observed, because a rule has to be earned by evidence (§5.3). Fail-safe is why that is acceptable: every failure leads to a refusal or a pause, never to an action outside its authority (§5.2). A model that failed first without being fail-safe would be learning by breaking things; one that was fail-safe without failing first would never improve. What is a fail-first model? →

2. Where “fail-safe” comes from

“Fail-safe” is an engineering term, and the idea behind it is older than computing: design the system so that the likely failure leaves it in a safe state.

Engineers distinguish fail-safe from neighbouring ideas. A fail-secure door lock stays locked on power loss, while a fail-safe one opens so people can get out. A fault-tolerant or fail-operational system keeps working through a fault. A fail-soft system degrades gracefully. A fail-safe system does none of these by default: when it cannot continue safely, it goes to the state that cannot do harm.

A fail-safe design does not prevent failure. It decides, in advance, where failure is allowed to lead.

3. Why AI models fail unsafely

A modern AI model, left to itself, fails in the opposite direction.

None of this means models are useless. It means a model on its own has no answer to the fail-safe question: when this goes wrong, where does it go?

4. The science: how a fail-safe model works

4.1 Authority is computed first

A fail-safe model does not ask “what should I do?” first. It asks “what am I allowed to do here?” and computes the answer from three sources, none of which the learner can edit:

The intersection is the admissible set. Everything after this point happens inside it.

The order of authority in a fail-safe model Nested regions. Everything the model could do contains what the situation offers intersected with what the sources license; standing orders narrow that to the admissible set; the learner only ranks and removes options inside the admissible set. Everything the model could do What the situation offers ∩ what the sources license After your orders: the admissible set The learner works here only removes what has failed, with evidence reorders what is left never adds an option

Figure 1. The order of authority. Each box can only shrink the one outside it. The learner sits innermost and cannot reach outward.

4.2 The loop

FAIL → OBSERVE → EXPLAIN → BUILD RULE → VERIFY → RETRY

  1. Fail. Something goes wrong. It is recorded rather than averaged away.
  2. Observe. The facts at the moment of each decision are stored with the decision.
  3. Explain. The failure is charged to the decision that caused it. This is credit assignment, and it is where learning systems most often fool themselves (§5.4).
  4. Build rule. A pattern becomes a rule only when its harm is statistically clear against the base rate (§5.3). The rule cites the failures that earned it.
  5. Verify. The rule is applied only inside the admissible set. Tests pin that the learner can never add a goal and never overrule an order (§5.2).
  6. Retry. Next time, the condemned option is off the table and everything else stays open.

4.3 A guess never becomes a fact

The same principle governs knowledge. A language model may propose a fact or suggest where to look. A fact is admitted only when an independent, deterministic check confirms it against a source, and a verified fact is complete or absent, never 86% right. When nothing verifies, the answer is unknown. We measured what happens when that rule is broken in §5.6.

4.4 Two kinds of failure

Not every failure should be avoided at any cost. In some environments a failure is terminal: it ends everything that could have followed. In others it is recoverable: it costs some ground, and play continues. A model that prices every failure as terminal will correctly learn that the risky action is dangerous, and then stop doing the one thing that makes progress. We published exactly that result (§7). The fix is to price recoverability, and that is still an open research question.

5. The math

This section states the fail-safe guarantee as mathematics, so it can be checked instead of trusted. The notation is the minimum needed.

5.1 Setup

Definition 1 (situation, goals, authority). Let s be the situation at a decision, and G a finite vocabulary of goals. Let App(s)⊆G be the goals the rules find applicable, Lic(s)⊆G the goals licensed by the sourced cards in force, and O1,…,Om the standing orders, each a map on sets of goals with Oi(X)⊆X. The admissible set is
AO(s)= Om(⋯O1( App(s)∩Lic(s) )⋯)⊆ App(s)∩Lic(s)
Definition 2 (the learner). The learner’s state M is a table of counts: for each situation pattern and goal, how often it was tried and how often it ended in failure. From M it derives two operations on any non-empty set of goals X: a veto VM(X) with ∅≠VM(X)⊆X, and a ranking πM, a permutation of its argument. A fixed rule policy c picks one element of an ordered non-empty set. The decision is
δ(s,M)= { c(πM(VM(AO(s))))if AO(s)≠∅ wotherwise (hold still: the safe state)

5.2 The fail-safe guarantee

Theorem 1 (bounded learning). For every situation s and every learner state M, however much it has learned, δ(s,M)∈AO(s)∪{w}.
Proof. If AO(s)=∅ the decision is w. Otherwise VM(AO(s))⊆AO(s) by Definition 2, a permutation adds no elements, and c returns an element of its argument. So the chosen goal lies in AO(s). ∎

The proof is one line on purpose. The guarantee does not depend on the learner being clever, correct or well-trained. It depends only on where the learner sits. That is the architectural meaning of fail-safe: a bad learner makes worse choices among permitted options; it cannot make an unpermitted one.

Corollary 1 (no learned permission). Let R(s)={δ(s,M):M any learner state}. Then R(s)⊆AO(s)∪{w}. Experience changes which permitted goal is chosen, never whether it is permitted.
Theorem 2 (orders outrank experience). If the orders narrow the admissible set to a single goal, AO(s)={g}, then δ(s,M)=g for every M, even if M records that g has failed every time.
Proof. VM({g}) is a non-empty subset of {g}, so it equals {g}. ∎

Theorem 2 encodes a deliberate choice: a human order outranks the model’s experience. The learner can report that an order is costly; it cannot countermand it. The veto is never empty for the same reason: standing still and failing is not an adaptation, so when experience condemns everything, the least-condemned permitted option comes back.

Proposition 3 (orders only narrow). Adding an order never adds a goal: for any additional order O′, AO,O′(s)⊆AO(s). Immediate from O′(X)⊆X.

In Peel these statements are not only on paper. The VDSG paper states the bounded learner and order narrowing as propositions [10], and the test suite pins all three in code: test_the_learner_can_never_add_a_goal, test_the_learner_cannot_overrule_a_standing_order, test_ranking_is_a_permutation_and_nothing_more.

5.3 When does a failure become a rule?

A rule should form when a pattern is clearly more dangerous than normal, not when it was unlucky once. For a pattern tried n times with d failures, the point estimate p^=d/n is misleading at small n: one failure in one try reads as 100%. So the decision uses the lower edge of the Wilson score interval [4]:

L(d,n)= p^+z22n−zp^(1−p^)n+z24n2 1+z2n

and a pattern is condemned when it has enough tries and its lower bound clears the base rate b (the overall failure rate) by a margin μ:

condemn(d,n)⟺ n≥nmin∧ L(d,n)>b+μ

Worked values at z=1.645:

Wilson lower bounds computed from the formula above. The point estimate overstates the evidence at small n.
failures / triespoint estimatelower bound Lreading
1 / 1100%0.270One death is weak evidence, whatever the point estimate says.
4 / 4100%0.596Four in a row is strong evidence.
13 / 1968%0.496Against a base rate near 0.19, clearly condemned.
3 / 475%0.356Suspicious, not yet proven.

Why the bar is relative to the base rate and not absolute: in an environment where the model survives 99.5% of decisions, a situation that kills it 5% of the time is ten times more lethal than normal and must be refused, yet it never approaches an absolute threshold like 60%. An absolute gate produced zero rules from 268 real failures in our own run before we changed it (defense/evolution).

5.4 Credit assignment: blaming the right decision

When a failure happens at time tf, some set of earlier decisions B(tf) receives the blame. The naive choice, the last w decisions, is usually wrong: the decisions just before a failure are often the response to the danger, not its cause. The rule has to charge the decision that opened the dangerous episode.

We have published two cases where we got this wrong. In Atari, the emulator reported a lost life at the end of a 127-frame death animation, so every failure was recorded after the agent was already destroyed: 0 of 374 blamed frames showed the hazard; at the true impact frame, 17 of 17 did. In Fallout, blame covered only the last six decisions inside a fight: 26 deaths were charged to HEAL, and the 446 conversational replies that started those fights were charged nothing. Both errors produced believable learning curves. Credit assignment is where a learning system fails silently, which is why a fail-safe model must never let a mis-blamed rule widen its authority: Theorem 1 guarantees a wrong rule can only make it more cautious.

5.5 Retry: improvement that cannot regress

When the model improves a plan, for example a route, it changes one parameter θ at a time and keeps the change only if the cost falls on every trial. With a cost that prices failure explicitly, for instance

C(θ)=T(θ)+κ·k(θ)+P(θ)

(time taken, plus κ seconds per contact k, plus a penalty P for not finishing), the accepted sequence satisfies Cj+1<Cj by construction, and a rejected move halves its step. The best-so-far can only improve, and the floor still applies to every trial because speed limits from stop distance and reflexes are applied after the learned plan and can only slow it down. The flight-console recording shows it: a wiped memory, 7 rounds, cost 69.39 → 51.52, 2 changes kept and 5 reverted.

5.6 Why a guess must never become a fact

Suppose a model’s guesses are each right with probability q, and an answer depends on k guessed facts. If the guesses are independent, the answer survives with probability

Pr[answer correct]=qk, 0.8582≈0.736

That is an illustration, not our analysis, but it matches what we measured. In a diagnostic task over 160 human kinases with 40% of the reference data masked, a frontier model filled the gaps with 85.8% accuracy. Letting those guesses into the elimination step cut measurement cost by 38.4%, and correct identification fell from 1.000 to 0.753. Keeping admission to verified facts, and letting the model only rank where to look, kept correct identification at 1.000 [11]. A guess that is right 86% of the time still corrupts the answer one time in four once it is allowed to count as a fact.

5.7 When is refusing the right call?

Take an action with failure probability p, failure cost c, and value v if it succeeds. It beats doing nothing when

(1−p)v−pc>0 ⟺ p<vv+c

A terminal failure forfeits everything that could follow, so c is very large and the threshold on p collapses toward zero: refusing anything clearly dangerous is right. A recoverable failure costs only lost ground, so c is small; if the dangerous action is also the only one with value, refusing it is wrong. A confidence threshold fixes one cut-off on p for every action, which amounts to assuming a single c everywhere. No single value is right in both regimes. That is the formal reason a fail-safe model must know what kind of failure it is avoiding, and it is the open problem named in §8.

6. Fail-safe model vs guardrails, shielding and fine-tuning

Typical forms of each approach. Individual systems vary.
approachlearns from failurelearner can loosen the rulesrefusal cites evidenceabstains when unknown
Confidence thresholdnon/aa numberno
Guardrail filtersusually non/a (fixed)sometimessometimes
Rules written into a prompt (ExpeL, AutoManual [7] [8])yescan be ignored by the modelpartlynot enforced
Fine-tuning / RLHFyes, in weightsyes: nothing separates rules from skillnonot guaranteed
Shielding [5] / Simplex [6]no (hand-specified)nocites the specificationvaries
Learned shields [9]yesnovariesvaries
Fail-safe model (Peel)yes, as readable rulesno (Theorem 1)yes, every ruleyes

Learned shields are the closest prior work, and we cite them as such. What we claim, and only this: to our knowledge, Peel is the first model to combine rules built from observed failures, each citing the failures that earned it; an authority computed before the learner from sourced cards and human orders, which the learner provably cannot widen; and no neural network in the loop that decides. If you know of an earlier one, tell us.

7. Evidence

Measured in-house; negatives first. Compare within a row against its own baseline.
testresultwhat it shows
Freeway (Atari)9.2 vs 10.4 (−12%)Pricing every failure as terminal stops a model scoring when the risky action is the only useful one (§5.7).
DOOM17.9 → 20.3, paired t = 0.78Parity, not a win: the memory layer does not separate from the rules.
Space Invaders (Atari)200.6 vs 152.2 (+32%)Learned refusal helps where failure is terminal.
Kinase identification1.000 vs 0.753 correctRefusing to admit guesses keeps answers right (§5.6).
Fallout (1997), scripted scenedies once, then chooses the peaceful lineAfter fixing credit assignment; before, a live run died 49 times at the same guard (§5.4).

8. What a fail-safe model is not

What this page does not publish. Everything above is either standard mathematics or a property we measure in public. The mechanisms that make Peel’s science floor work at scale, including how facts are extracted, how contradictions are vetoed and how provenance is stored, are withheld pending patent review.

9. Peel: the Fail-Safe Model

Peel, by Perslis Research, is the Fail-Safe Model. It runs in two deployments today:

Peel, the Fail-Safe Model Watch it learn from failure Read the VDSG paper

10. Questions

What is a fail-safe model?
An AI model built so that when it fails, the failure drives it toward a controlled, safe state rather than an uncontrolled one. It fails closed when evidence is missing, it can learn from failure but the learning can only narrow its behaviour, never widen what it is authorised to do, and every refusal cites the evidence behind it. Peel, by Perslis Research, is the Fail-Safe Model.
Is Peel the first fail-safe model?
To our knowledge, yes. Peel is the first AI model to combine rules built from observed failures, each citing the failures that earned it; an authority computed before the learner, from sourced cards and human orders, that the learner provably cannot widen; and no neural network in the loop that decides. The closest earlier work is learned shielding (Shperberg, Liu and Stone, 2022). If you know of an earlier fail-safe model, tell us and we will cite it.
What is a fail-first model?
Another name for a fail-safe model, describing how it learns: it expects to fail, learns only from failures it has actually observed, and turns each one into a rule that can only narrow what it does. Failing first is acceptable because the model is fail-safe: no failure can widen its authority.
Does a fail-safe model never fail?
No. It is expected to fail. The guarantee is about where a failure is allowed to lead: toward a refusal or an abstention, never toward an action or a fact outside its authority.
Is fail-safe the same as fault-tolerant?
No. A fault-tolerant system keeps operating through a fault; a fail-safe system, when it cannot operate safely, moves to a safe state instead. A fail-safe model may stop, abstain or refuse. It may not guess.
How is a fail-safe model different from AI guardrails?
Guardrails are usually fixed, hand-written filters around a model, and rules written into a prompt can be ignored by the model that reads them. A fail-safe model computes what it is authorised to do before it chooses, builds its own rules from observed failures, and applies those rules only inside that authority.
Does a fail-safe model use a neural network?
The definition does not require one either way. In Peel there is no neural network in the loop that decides: knowledge is typed, sourced cards and learning is a table of counts. A language model may propose; only the floor admits a fact.
What is the difference between fail-safe and fail-secure?
In physical security, a fail-safe lock opens on power loss so people can leave, while a fail-secure lock stays locked. For an AI model, the safe state is the one that cannot cause harm: abstaining, refusing, or holding still.
Who makes the Fail-Safe Model?
Perslis Research. Peel is the Fail-Safe Model for science, and VDSG is the military Peel, the Fail-Safe Model for defense. Both are research prototypes, not certified safety systems.

11. References

  1. D. Amodei, C. Olah, J. Steinhardt, P. Christiano, J. Schulman, D. Mané. Concrete Problems in AI Safety. arXiv:1606.06565, 2016.
  2. V. Krakovna et al. Specification gaming: the flip side of AI ingenuity. DeepMind, 2020.
  3. Z. Ji et al. Survey of Hallucination in Natural Language Generation. ACM Computing Surveys 55(12), 2023.
  4. E. B. Wilson. Probable inference, the law of succession, and statistical inference. Journal of the American Statistical Association 22(158):209–212, 1927.
  5. M. Alshiekh, R. Bloem, R. Ehlers, B. Könighofer, S. Niekum, U. Topcu. Safe Reinforcement Learning via Shielding. AAAI, 2018.
  6. L. Sha. Using Simplicity to Control Complexity. IEEE Software 18(4):20–28, 2001.
  7. A. Zhao et al. ExpeL: LLM Agents Are Experiential Learners. AAAI, 2024. arXiv:2308.10144.
  8. M. Chen et al. AutoManual: Constructing Instruction Manuals by LLM Agents via Interactive Environmental Learning. NeurIPS, 2024. arXiv:2405.16247.
  9. S. S. Shperberg, B. Liu, P. Stone. Learning a Shield from Catastrophic Action Effects: Never Repeat the Same Mistake. arXiv:2202.09516, 2022; and the rule-based follow-up at CoLLAs 2022.
  10. Perslis Research. VDSG: A Commanded Admission-Control Runtime for Autonomous Agents. 2026. research.perslis.com/vdsg
  11. Perslis Research. Inference Placement: where learned inference earns authority in a provenance-constrained symbolic system. 2026. research.perslis.com/inference-placement