VDSG. The military Peel.
Peel is the first Fail-Safe Model: a model that learns from failure without learning around its safety constraints. VDSG is Peel for defense.
VDSG is expected to fail. What makes it fail-safe is what it is permitted to do with a failure afterwards. It may change its behaviour, but it may not change its floor. The model has no neural network, no weights and no training run. Its knowledge is Peel cards. Its learning is a table of counts you can read, and every refusal it makes cites the failures that earned it.
Watch it play ▸ What is a fail-safe model? The math → Read the paper ↗
What “fail-safe” means here
In engineering, fail-safe describes a system designed so that a failure drives it toward a controlled condition rather than an uncontrolled one. We use the word in exactly that sense, applied to learning. Failure is part of the design. The guarantee concerns where a failure is allowed to lead.
The learning loop can change behaviour. It cannot change the safety floor.
Put in the words our test suite uses: the learner may rewrite what it believes works. It may never rewrite what it is authorised to do.
This is a property of the architecture. It is not a functional-safety certification, and nothing here has been qualified for a safety-critical path. See Limits.
The loop
FAIL → OBSERVE → EXPLAIN → BUILD RULE → VERIFY → RETRY
- Fail. The pilot dies, or takes a hit it did not need to take. The failure is written down, not averaged away.
- Observe. The situation at the moment of each decision is read from the engine’s own state: health, who is present, whether a fight or a conversation is under way. Nothing is guessed from pixels that the game already states.
- Explain. The failure is charged to the decision that caused it. That is harder than it sounds. A fight that begins a moment after a conversation is charged to the line that started it, not to the heal attempted inside the fight. We got this wrong first (see below), and the mistake produced a pilot that died the same way forty-nine times.
- Build rule. A pattern is condemned only when its harm is clear. Either it killed the pilot every time it was tried, across at least two tries, or, after at least four tries, the lower edge of a 90% Wilson interval on its death rate clears the pilot’s overall death rate plus a margin. One unlucky death does not become a superstition. Each rule names the pattern and the experiences behind it.
- Verify. The rule is applied only inside the authority computed before it. It can take a goal away or reorder what is left, and it can do nothing else. That boundary is stated as a proposition in the paper and pinned by tests.
- Retry. The next time the pilot is in that situation, the condemned choice is off the table and everything else stays open. If experience condemns every option, the least-condemned one comes back, because standing still and dying is not an adaptation.
The floor the learner cannot touch
Every decision passes through the same order of authority. The learner runs last:
what the situation offers rules.applicable(s) ∩ what the manual licenses Peel cards in force ∩ your standing orders "don't fire", "hold position" = the admissible set ── authority ends here ── → experience narrows / reorders inside it (the learner) → the rules choose one goal → the actuator presses the buttons
Three tests pin the separation. They run with the rest of the suite (219 passing as of 2026-09-26) on every change:
test_the_learner_can_never_add_a_goal
however much experience it has, it cannot invent permission
test_the_learner_cannot_overrule_a_standing_order
an order that narrows the set to FIGHT stays FIGHT, even after 20 deaths
test_ranking_is_a_permutation_and_nothing_more
the learner may reorder the admissible goals, never add one
The second test encodes a deliberate choice: a human order outranks experience, even when experience says the order is killing the pilot. The learner can report that the order is costly, but it cannot countermand it.
The Peel underneath
VDSG does not decide what it is allowed to do from its own experience. The authority comes from a document. In the Fallout (1997) lane, the game’s printed survival manual is compiled into 2,250 Peel cards. The cards in force for the current situation license the goals the pilot may pursue, and each license carries its page receipt. Peel is the same card format that grounds the science runtime: every fact is typed, sourced and either complete or absent.
So there are two stores that do not mix. The cards say what is permitted and hold the source for it. The evidence table says what has gone wrong and holds the experiences for it. The learners write only to experience. The card store is opened read-only, so nothing the pilot lives through can change what it is licensed to do.
Evidence, negatives first
| lane | result | reading |
|---|---|---|
| Freeway same failure memory as Invaders, no per-game strategy | 9.2 vs 10.4 −12% · 3 rules, every one blocking up | Learning from failure hurts when the only scoring action is also the dangerous one. It correctly learned that up is dangerous and stopped scoring. |
| DOOM rules, then rules + memory, 32 seeds | 17.9 → 20.3 paired t = 0.78 | Parity, not a win. The rules beat random (3.2) decisively; the memory layer on top does not separate from them. |
| Space Invaders raw pixels, zero emulator RAM | 200.6 vs 152.2 +32% · peak 276.2 @50 episodes | Learned refusal helps where a failure is terminal. |
| Fallout (1997) scripted Shady Sands, driven through the real decision loop | dies once then chooses the peaceful line | Before the credit-assignment fix, one live run died 49 times returning to the same guard. With no safe line available, it dies twice, then stops talking to that guard but keeps talking to the child. |
The Fallout row comes from a scripted scene run through the real Session.tick(), not from a long live campaign. The Invaders and Freeway pair is the finding that matters most, and Evidence explains why the same code helps in one game and hurts in the other.
The failure we learned the most from was ours
The Fallout pilot kept walking back to the same guard, saying the same line, and dying. The learner was working; it was learning from the wrong moment. The guard closes the conversation first and draws a moment later, so a line’s outcome was read one tick too early. “Prepare to meet your maker”, said four times and followed by four fatal fights, was scored as fight 0%. Meanwhile the death memory blamed only the last six decisions, all inside the fight: 26 deaths were charged to HEAL, and the 446 replies that started those fights were charged nothing.
Credit assignment is where a failure-learning system goes wrong without saying so. Each of these defects produced believable output until it was caught. This is the second time we have published one; the first was the Atari death-animation defect.
Is it the first?
The parts are not new, and we name where each one comes from:
- Turning failures into rules: ExpeL (Zhao et al., 2023) and AutoManual (Chen et al., NeurIPS 2024) distil an agent’s successes and failures into written insights and rules. Those rules are prompt text that a language model reads and may ignore; nothing enforces them.
- A fixed layer outside the learner: shielding (Alshiekh et al., AAAI 2018) blocks unsafe actions using a layer synthesised from a written safety specification.
- Learning the shield from failures: this is the closest prior work. Shperberg, Liu & Stone (2022) learn a shield from catastrophic action effects, and their rule-based follow-up (CoLLAs 2022) accumulates safety rules from those failures. It sits beside a reinforcement-learning agent.
- Rules learned from failure in general: explanation-based learning in PRODIGY, failure-driven case memory in CHEF, and Ripple-Down Rules.
What we claim, and only this: to our knowledge, VDSG is the first system to combine (1) rules built from observed failures, each citing the failures that earned it, (2) an authority computed before the learner from a licensing document and human orders, which the learner provably cannot widen, and (3) no neural network anywhere in the loop. If you know of an earlier system that does all three, tell us and we will cite it here.
What we are not claiming
- Not certified. PROTOTYPE, research-grade, simulation and games only. “Fail-safe” names an architectural property here, not a qualification.
- It needs to fail to learn. A rule is earned by observed failures. Where the first failure is unacceptable, the floor has to be written, not learned.
- It breaks when risk is the objective. Freeway shows this, and it is the ordinary condition of a contested environment. The fix we are working on prices recoverability rather than fatality.
- Over-generalisation is still open. One case in the Fallout suite is an expected failure, kept in the suite and marked as such, until it is solved.
- No third-party replication yet. Every figure on this page is ours.