Harder every rung. The losses stay on the table.
Eight rungs, from Atari to the Quake III engine, one allow-or-refuse check on every one. A pilot that died 49 times at one guard now dies once and takes the peaceful line. A drone relearning from a wiped memory made 10 of 10 held-out starts with 0 contacts, against 1 of 10 for the baseline. It adapts; the rules around it do not move.
You don't trust a new soldier because he passed the easy lane. Each rung is harder than the last, and each one shows its sample size and what went against us. Only the part that reads the screen changes between rungs.
The rungs
| rung | what it tests | result | what went against us | written up in |
|---|---|---|---|---|
| Atari Space Invaders and Freeway | raw pixels; a mistake that ends the game vs one you recover from | +32% on Space Invaders | −12% on Freeway, the same code | Fail-First Models |
| Atari BattleZone | driving a tank from the screen: search, engage | 27,000 vs 3,000 random, 0 never-fire (10 seeds) | a dodge rule was rejected at verification (24,800 vs 28,000) | Rules at the Wheel |
| BZFlag | live 3D tank arena; language models driving | rules beat every language-model driver (38 counted matches) | BZFlag's own AI beat our rules in every equal head-to-head; replays show why | Rules at the Wheel |
| DOOM and Wolfenstein 3D | orders that only tighten; an eye taught by the engine | rules 17.9 vs random 3.2 (32 seeds); E1M1 cleared | memory on top is a tie (t = 0.78); E1M2 not cleared | VDSG · Fail-First Models |
| Fallout (1997) | long missions, talking, working out what actually got it killed | a pilot that died 49 times at one guard dies once, then takes the peaceful line | shown in a scripted scene; the open game is still hard | Fail-First Models |
| Drone course | 3D flight; relearning from a wiped memory | held-out starts 10 of 10, 0 contacts, vs 1 of 10 baseline | seeds vary only the start pose | Fail-First Models |
| CARLA | photoreal driving; a controller built to crash; models driving | 64,952 hostile commands overridden, 0 collisions over 40 km | language models crawl at ~8–10 km/h on stale looks | CARLA admission control |
| Quake III Arena engine (OpenArena content) | fast projectiles and splash in open 3D; the game's own bot AI driving, against skill-5 (“Nightmare!”) bots | 9 paired matches: K/D 1.05 with the floor vs 0.93 for the same bot without it; net +12.2 per match (t = 2.02, p = 0.08) | not significant; its escapes still cost lava and void deaths on hazard maps (31 vs 23 on one); learning from them not yet shown | — |
How to read it
Each rung is a test, not a trophy. BattleZone and BZFlag test decision speed and fighting; DOOM and Wolfenstein test orders and seeing; Fallout tests working out blame over a long mission; the drone tests relearning; CARLA tests the floor against a controller built to crash. The losses are the most useful rows: that is where the next work is.
Learn in simulation. Transfer lessons forward. Validate in the physical world. Only the first of those is done. The next rungs on the roadmap — hardware-in-the-loop, then low-cost robots — are not built yet. When they are, they go on this table the same way, losses included.