We won the game once. Then we spent two weeks failing to do it again.

01What the system is, in one screen

A language model plays a puzzle game nobody has explained to it. Its weights are frozen — it never learns by training. It sees a 64×64 grid of coloured squares and can click on them, and the game answers with a new grid and a single bit: level cleared, or not. It never says why.

What the puzzle is, the model has to work out for itself. Its recorded hypotheses describe a board carved into blocks of six cells by six, where one click anywhere inside a block repaints all thirty-six of its cells at once; and a small patch of clue cells near the middle that works as a legend, each clue naming the colour one block must end up. A level clears when every block matches what the legend demands. None of that is ever told to the model — not the block size, not the legend, not which clue means which colour. It is the model's own reconstruction from clicking and watching, and it is what the quotations below are talking about.

Around the model sits a harness. Each round the model writes a hypothesis, predicts what its clicks will change, then commits as many clicks as it chooses — none, one, or a dozen. That count is the single quantity this page turns on: clicks per round, and committing several at once is batching.

Between rounds the model writes what it has learned into skills, which the harness stores and scores; the model is the only thing here that reasons. Two kinds are executable code — a rule about how the world moves, which predicts the cells a click will change, and a guess about what wins the level, which draws the board it believes the engine wants. Two kinds are plain notes with no code in them: a strategy, and an observation about the model's own reasoning. Only the executable two can be run during play, and an experiment in section 05 turns entirely on that. Each skill carries a confidence number that only real observations can move, and the library handed from one run to the next is the memory, the only thing that crosses between games.

Three rules make the result mean something: the harness may never peek at the game's hidden state, the model is never trained, no answer key is ever supplied. The target is simpler — six levels, and clearing all six is the bar. Current runs stall on the third or fourth. The logs count levels completed, so a run showing three has cleared three and is playing the fourth, and this page calls whichever level a run dies on its wall.

02The one win, and why it did not settle anything

Sixteen days later we replayed its 174 clicks against today's game, one at a time, with the language model switched off. Every board came back identical across all 52 rounds, and all six levels cleared again at the same rounds.

That kills a family of excuses: the game did not get harder, the seed did not change meaning, the engine was not swapped, the old recording was not corrupt. Whatever is going wrong now is inside our own decision-making.

But the win came with bad news attached. Dissecting it the same week showed the learning machinery had been almost completely idle during the victory: of 38 skills only 2 ever received evidence, the part that mines new skills produced nothing for its last eight cycles, and that era's gates discarded seven or eight of every ten candidates. The model reasoned its way to the win; the learning system mostly watched. So the win did not validate the design, it exposed it — and the scoring machinery was rebuilt afterwards, so no version of the current system has ever cleared six levels. The comparisons below are not one system regressing; they are two systems held against each other.

03How we run an experiment

Every experiment on this page is a pair of runs that differ in exactly one setting, on the same game and seed, with the decision rule written down before the numbers exist. That last part is not ceremony: three times in two weeks we published a number and had to withdraw it, and each withdrawal traced to a rule adjusted after the data arrived.

Two habits earn their keep, both learned by being burned. Check the treatment can fire before spending a run on it: one four-hour experiment added a permission depending on a condition the system could never satisfy, a no-op dressed as a treatment. And read the actual text, not your memory of it: twice we built an experiment against a paraphrase and manipulated a sentence that did not say what we remembered.

04What worked

Learned memory genuinely transfers

The project's north star is that knowledge from one run should help the next, and testing it needs a control, because a bigger briefing might merely look helpful. The same game and seed were played four ways: with the real learned memory, a same-size fake, only the world-model half, and only the goal half.

The margin needs saying carefully: one level, on two unseen seeds with a single run in each cell. And the placebo is not the clean zero the row suggests — 2 on its first run, matching the better from-scratch run, then 0 on its replicate. That spread is as wide as the effect we are claiming, so the table supports a consistent direction across five cells, not a measured size. The ledger's own scope limit says as much: this harness, this game and seed, 1–3 runs per condition — and it does not vindicate the architecture.

The environment is not the problem

Replaying the winning run's clicks reproduced all six clears identically today (section 02): a positive result about the world, a negative one about us. Every remaining explanation lives in our own code.

We can now see why the planner never fires

The system has a component meant to turn a confirmed rule into a multi-step plan. It fires rarely, is followed at about chance, and the one plan step ever executed changed zero cells: the first plans this project ever mined appeared on 19 July, and over the seven rounds where a plan was live the actor took the plan's action in one — a follow rate of 0.14, indistinguishable from coincidence. In the most recent arms the plan kind produced no records at all, and nobody knew why: the code computed a diagnosis and wrote it to a log channel that was never switched on. Turning that on took one line and the answer appeared immediately — on a fresh level the planner requires all confirmed rules to agree on what a click will do, they never do, so it rejects every candidate move and reports nothing. Its own diagnostic even names the relaxation that would unblock it.

The search for the cause is converging

Batching died somewhere in our own commit history, and running old versions against the same game has narrowed the window from ten days to five. Section 08 has the live search.

05What didn't — the negative results

These are the bulk of the fortnight. Each closed an idea, and closing an idea honestly is the deliverable.

It is not the action vocabulary, and it is not missing knowledge

The model almost only ever clicks, though the game offers other buttons; perhaps the wall was the narrow vocabulary. We made the harness spend those buttons and measured each: they alter a single row at the bottom of the screen, counting 2 → 1 → 0. A consumable meter, not a door. Separately we placed the model on the wall with answer-shaped knowledge already in its briefing: thirty rounds, zero clears. Both ruled out.

It is not the runtime simulator — and that result was misread for eleven days

Two of the four skill kinds are executable, so we tested whether running them during play does any work: paired runs on three seeds, one normal, one with the simulator off and a byte-identical briefing. Judged on levels alone the switched-off arm matched or beat the normal arm every time, and we filed it as “no benefit”. That reading missed what mattered.

Every run with the simulator available sits at exactly one click per round; every run without it batches. We had scored that experiment on levels and never looked at clicks per round, so the sharpest signal of the fortnight sat on disk unread.

But switching the simulator off does not fix it either

Disabling the simulator on today's code should then bring batching back. It did not: at the wall the run averaged 1.10 clicks per round with no batched rounds, no different from the control. The tidy mechanism is refuted by direct intervention. That is one 10-round run against one 8-round control overturning three matched pairs plus a survey of every trace on disk (21 of 68 ran long enough past level two to count) — the thinnest evidence here carrying the heaviest load, and it needs replicating. The instruction explains the error: simulate the dense action first, and if the simulation shows nothing or your best rule is weak, take one cheap probe instead. Removing the simulator does not remove that gate, it makes the gate impossible to satisfy, so it binds harder.

Five separate prompt sentences, all innocent

Before the simulator finding we blamed the prompt five times, and measured each one dead.

And batching by itself does not win

This is the negative result that changed the plan. The two-week-old code batched at the wall handsomely — 4.26 clicks per round across 27 rounds, 21 of them batched — and still did not clear the level, spending 115 clicks where the winning run spent 14. Measured against the board the winner held when it cleared, it got closest at round 9 and then moved further away for the rest of the run.

Three times we had to retract a headline

Twice a claim was computed over the wrong population: once we called three levels a project record while a six-level win sat in the archive, once a ratio came from a quantity that was never computable. The third time a measuring script matched only one of the two forms a log line can take and produced a clean, confident, wrong table. Every correction is stacked on top of the original entry rather than replacing it, so the mistake stays visible.

06Why six levels is hard — the arithmetic

Counting what each level cost the one run that ever finished the game — rounds and clicks up to and including the round that cleared it — gives the only end-to-end click budget we have measured. It is not a proven floor: these are the clicks that run actually spent, some possibly redundant, and nothing else has ever cleared these levels.

Read the last column downward: the winner speeds up as the levels get harder, from one click a round on level one to more than four and a half on level four. Today's runs do the opposite — they batch a little early, then settle to exactly one click per round precisely where the winner accelerates. The conclusion survives the caveat because it needs only the arithmetic: a run committing one click per round would need 174 rounds to spend what the winner spent, the last two levels alone accounting for 112 clicks. Our experiments have been giving 30.

The same board, the same rule, two different shapes

Below is the third level's starting frame, identical in both runs — same game, same seed — and both work from the same rule: read the legend, repaint the blocks it marks. Each figure covers only that run's own rounds on this level, which is why the numbering differs: the winner arrived at round 9, the current run at round 11. Each ring is one click, numbered by round.

Both counts are the table's: 6 rounds and 14 clicks for the winner against 20 rounds and 20 clicks for the current run, which never cleared. Rings sharing a number were committed in the same round, and that grouping is the whole difference.

The winner spent its first two rounds on this level clicking nothing — just looking — then committed six blocks at once. Across every round the current harness has recorded in this format — 1,071 rounds across 50 runs, every trace file in the repository — not one was spent purely observing.

Why grouping is not merely faster

While required blocks remain unpainted, “not cleared” carries no information: my rule is wrong and I am not finished yet look identical. Committing six blocks at once collapses that ambiguity in one round, and the winner's own notes show the moment it paid off:

07Schema: certify before you commit

Schema — the certify-before-commit lineage, as the experiment ledger records it — earns a section because our central problem is a distorted version of the idea it contributed. The constitution's own reference list names the neighbouring lineages: baseline1 (~/arc-3-agents-baseline1), an in-house predecessor with one unified world model that sends only the mispredicted part of a board back to the language model, plus a planner — the per-click predict-then-act discipline; Ellis, the program-library lineage (DreamCoder's wake–sleep loop, WorldCoder, PoE-World, whose probability weighting is the ancestor of our confidence numbers); SkillOpt (arXiv:2605.23904), whose rollout–reflect–edit–gate cycle our sleep phase copies; and Lake (arXiv:2405.13242), the source of goals written as executable reward programs. Schema is the one whose principle we adopted, kept, and then broke.

Its intent and method

An agent that learns rules from experience will learn some wrong ones, and acting on a wrong rule at scale corrupts the record it is learning from. Schema's answer is a discipline of order: a rule earns the right to be acted upon before it is acted upon. A candidate is replayed against everything already seen and certified only if it reproduces every past case it claims to cover, one failure disqualifying it. Only certified rules may drive committed action, and certification is scoped — certified for the situations the rule claims, so being right about one kind of board is no licence to act on another.

What we adopted, and what happened

We wrote that principle into the project's constitution: certified only where the precondition holds, replay exception-free, one failure is a failure. That part works. The distortion is in what “not yet certified” may mean — and the constitution rejects the dangerous reading explicitly:

The system behaves exactly that way anyway. At the wall — the first level where inherited certification lapses and it matters — the four prompt variants run as a matched set all commit exactly one click per round, with a maximum of one and not a single grouped round between them.

That match is a coincidence worth recording, not a mechanism. No experiment on this page manipulates certification status, so nothing here shows lapsed certification produces the one-click rounds, and the five sentences and rules we did manipulate were each measured innocent (section 05). The ledger's own conclusion after those five is that no sentence and no harness rule found so far accounts for the behaviour; it points instead at board complexity, on the reading that at the wall the model is genuinely less certain per block and clicking one at a time is locally rational. We have a prohibition and a matching violation, with the link between them unestablished.

The result, stated plainly

The recorded reading of this lineage used to be “we certify truth but never sufficiency” — we check that a rule is right, never that the set of rules is enough to reach the goal. This fortnight added a second line: we do not spend the truth we certified. The bridge from a confirmed rule to a multi-step commitment is the planner, and the planner barely works (section 04). We kept the half of Schema's idea that prevents damage and lost the half that makes certification worth earning.

08What is running right now

Four runs. The first is an attempt at all six levels; the other three search our own commit history for the change that killed batching. Each has its reading fixed before it finishes.

A · The six-level attempt

Intent

Every previous experiment was capped at 30 rounds, and a one-click-per-round run needs 174 to spend what the winner spent — so 30 is 17% of the budget. This is the first run where finishing is arithmetically possible.

Method

90 rounds, the runtime simulator disabled, everything else standard.

How to read it

Clearing the fourth level beats the best any current-architecture run has managed, which is three. Clearing the sixth reproduces the win.

Where it stands

Round 49 of 90, three levels cleared, playing the fourth. It collapsed to 1.19 clicks per round on the third level for 36 rounds, then recovered to 4.00 on the fourth — where it has spent 10 rounds and 40 clicks without clearing — already past the 37 the winner spent on that level including the round that cleared it. Same signature as the run in section 05 that batched and churned: batching is back, the rule is not right.

B · Searching our own history

Intent

Ten days ago, on the same prompt we use today, runs averaged 2.2–3.1 clicks per round at the wall and one reached the third level. Something we added since removed that.

Method

Check out old versions into separate working copies and run each against the same game and memory, placed directly on the wall so the answer arrives in ten rounds instead of thirty.

Results so far

code from	clicks/round at the wall	verdict
16 July, 18:01	4.26 · 21 batched rounds	batches
16 July, 18:40 — adds the one-action-per-round rule	3.00 · 2 batched rounds	still batches — suspect cleared
17 July — adds the verify-before-dense-commit gate	still climbing to the wall	in progress
20 July	in progress	in progress
23 July	1.00 · 0 batched rounds, all 10 rounds	does not batch
today	1.00 · 0 batched rounds	does not batch

Where it stands

The window has gone from ten days to five — the change is between 16 and 23 July — and the rule we most suspected is cleared. Two more old versions are running to halve it again.

09Where we stand

The next honest milestone is not six levels. It is a run that, on the level where everything currently stops, both commits several clicks in a round and gets closer to the goal for doing it, with the log showing why.