We won the game once. Then we spent two weeks failing to do it again.

00What the system is, in one screen

A frozen language model plays a puzzle game it has never been told the rules of. It sees a 64×64 grid of coloured cells and can click on them. The engine answers with a new grid and one bit: level cleared, or not. It never explains anything.

Around that model sits a harness. Each round the model states a hypothesis, predicts what it expects to change, and then commits as many clicks as it chooses — none, one, or a dozen. How many it commits in a round is the quantity this whole page turns on; we call it clicks per round, and committing several at once is batching. Between rounds the harness compresses what happened into small pieces of executable code we call skills — a rule about how the world moves, a guess about what wins, a strategy, a note about its own reasoning. Skills carry a confidence number that only real observations can move.

Two words appear throughout this page. A bundle is the skill library saved at the end of a run and handed to the next one — the system's long-term memory, and the only thing that crosses between games. A run is one full attempt at the game under one exact configuration; when we compare two runs that differ in exactly one setting, that pair is the experiment.

Three rules make the result meaningful rather than impressive: the harness may never peek at the engine's hidden state, the model's weights are never trained, and no ground-truth answer is ever supplied. Those three rules are the constraint: whatever the score, it only counts if you can reconstruct afterwards which code produced which action, what got refuted, and whether the model actually changed its mind. The target is separate, and simpler.

The game has six levels. Clearing all six is the bar — and the wall is the third one, where every current run stops. A seed fixes the whole six-level layout, so two runs on the same seed face byte-identical boards and a different seed is a different set of puzzles. The briefing is the text the harness assembles each round from the bundle and hands to the model — several experiments below change nothing but that.

01The one win, and why it did not settle anything

Sixteen days later we re-issued C3's 174 clicks against today's engine, one at a time, with the language model switched off entirely. Every board came back byte-identical across all 52 rounds and all six levels cleared again at exactly the same rounds.

That kills a whole family of excuses: the game did not get harder, the random seed did not change meaning, the engine was not swapped, and the recording was not corrupt. Whatever is going wrong now is inside our own decision-making.

But the win itself came with bad news attached. When we dissected it the same week, the skill machinery had been almost completely inert during the victory: of 38 skills, only 2 ever received evidence, and the mining engine produced zero gain for its last eight cycles. The hard accept/reject gates of that era were throwing away seven or eight of every ten candidate skills. The model reasoned its way to the win; the learning system mostly watched.

So the win did not validate the architecture — it exposed it. Everything that follows is an attempt to build a system where the learning actually contributes, and then to get back to 6/6 with it. Read the rest with one fact in hand: the scoring machinery was rebuilt after this win, so no version of the current system has ever cleared six levels. The comparisons below are not one system regressing — they are two systems being held against each other.

02Two weeks, question by question

Every row is a real experiment with its decision rule written down before the numbers existed. “NO” means our idea was wrong and we closed it. That is most of the table, and closing an idea honestly is the deliverable.

Five explanations we killed for the same symptom

The third level — where every current run stalls — always looks the same: the agent commits exactly one click per round and runs out of rounds. We named five plausible culprits and measured each one dead.

What survives is a fact rather than a culprit: nothing forbids batching, and the agent batches happily on the early levels — up to six clicks in a round — but never on the third level, where it stalls. That relocates the question from “what is blocking it” to “what would make it confident enough to commit several clicks at once.”

Three times we had to retract our own headline

Twice a claim was computed over the wrong population — once we called three levels a project record while a six-level win sat in the archive, and once a ratio was reported from a number that was never computable. The third time, a measuring script matched only one of the two forms a log line can take and produced a clean, confident, wrong table. In each case the correction is stacked on top of the original entry rather than replacing it, so the mistake stays visible.

03Why six levels is currently out of reach — the arithmetic

The winner spent 174 clicks over 52 rounds — about 3.3 clicks per round. Today's best run spends 1.07. Same game, same seed, same six levels.

Two ways of counting, one answer

Dividing the winner's clicks by today's rate gives 163 rounds. Counting instead in rounds per level — the honest currency, because rounds are the budget knob and each one is a full deliberate-and-learn cycle — gives the same answer from a different direction: the winner cleared the third level in 6 rounds, today's runs need about 20, and holding that 3.3× ratio across all six levels puts a full clear at 173 rounds.

The shape of that middle column is the whole finding. The winner escalates as the levels get harder — from a third of a click per round on the easy first level to nearly five on the third. Today's runs do the opposite: they batch a little early and then settle to exactly one click per round precisely where the winner speeds up. So there are two roads to six levels and only two. Give a run ~170 rounds — a full day of compute per attempt — or restore the escalation and buy the clear back at something nearer the winner's fifty.

Our experiments have been running 30 rounds — 17% of the projected requirement.

And the floor is set by the board, not by us

That projection could be dismissed as an artefact of how badly today's runs play. It cannot. Counting the clicks the winner actually needed on each level — a property of the puzzle, not of the player — gives a floor nobody can argue below:

A run that commits one click per round cannot clear this game in fewer than 153 rounds, no matter how good its reasoning is, because the fourth and fifth levels alone demand 107 clicks between them. Batching is therefore not an optimisation — at any budget we can realistically afford, it is the only way the game is winnable at all. That is why every experiment on this page eventually turns into a question about clicks per round. That division assumes six levels cost about 174 clicks no matter who plays — an assumption worth naming, since a better rule might need fewer. Even granting it, a six-level clear is out of reach at the current 30-round budget. There are exactly two ways out: make each round do more, or buy more rounds. The four experiments running right now split along that line.

The same board, the same rule, two different shapes

This is the actual third level, drawn from the recorded runs. The board below is byte-identical in both runs — same game, same seed, same starting frame — and both agents are working from the same rule: read the key in the middle, repaint the marked tiles. Each ring marks one click, and its number is the round that click belongs to.

The winner touched ten cells and cleared; today's run touched twenty and did not. The difference is not effort — it is that the winner's clicks arrive in groups. Rings sharing a number were committed in the same round.

Below, the same two runs as round shapes. One bar is one round; its height is how many clicks that round committed.

The winner spent its first two rounds clicking nothing — just looking — then committed six tiles at once. Today's run has no empty rounds at all: across every round recorded since the harness began writing this trace format — 965 rounds over 43 runs — not one was spent purely observing.

There is a second reason single-stepping hurts, and it is not about speed. While required tiles remain unpainted, “not cleared” carries no information — my rule is wrong and I am not finished yet look identical. The winner committed six clicks at once, was told “still not cleared”, and knew within one round that its rule was incomplete. A run that paints one tile per round cannot make that distinction for twenty rounds. Batching is an instrument for learning, not just a throughput trick.

04What is running right now, and how to read it

Four questions across six runs, launched 2026-07-26 evening. All use the same game, seed and learned bundle except where a question requires otherwise — C starts on a later level, and D runs older code. Each one is written the same way: what we want to know, what we think is true, how the run answers it, and what result would mean what — the last part fixed before the run finishes.

A word on the clock, because it explains why this moves slowly. One round costs about five calls to the language model, each taking roughly a minute and a half, and the calls inside a round depend on each other, so they cannot overlap. That is 3.8 hours of pure waiting per 30 rounds; with the between-round bookkeeping the measured wall clock is 5.2 hours, and a 90-round run is most of a day. What can overlap is one run against another, and these six are the first time we have run more than one at once — a deliberate test of whether the model service tolerates it, since a rate-limit failure five days ago is the reason everything since has been run one at a time.

A · Does the extra level we saw once actually exist?

Intent

One of four earlier runs reached three levels while the other three reached two. With a single run per condition we cannot tell a real effect from a coin flip.

Hypothesis

It is noise. The historical baseline for this configuration is three levels, so the three runs at two may simply have landed low.

Method

Re-run the three-level configuration and its control, identical settings, 30 rounds each.

How to read it

Both replicates land where their originals did → the spread is real and worth chasing. They cross over or converge → it was noise, and that four-run comparison should not be read as showing an effect.

B · If we cannot make rounds richer, can we simply buy more of them?

Intent

The best run cleared its third level on round 30 — the very last one. It was still climbing when the budget ended.

Hypothesis

It will park at three. An earlier 45-round attempt cleared its third level by round 28 and then gained nothing in the remaining seventeen, so the prediction is that rounds alone buy nothing — and the run is worth doing because that precedent is a single run on a different configuration.

Method

The same configuration with 90 rounds instead of 30.

How to read it

Levels keep appearing past round 30 → budget is a genuine lever and six levels becomes a question of patience. It parks at three again → extra rounds buy nothing, and the throughput problem is the only road.

C · Can the system clear a level it has never once reached?

Intent

A blind spot: the current system has never got past the third level, so we have no data at all on whether the fourth, fifth and sixth are within reach.

Hypothesis

They are individually solvable — the wall is cumulative budget, not a per-level capability ceiling.

Method

Place the agent directly on the fourth level (a measurement-only jump the agent cannot invoke) and give it 20 rounds. Its learned skills are tagged to earlier levels, so this is effectively a cold start — which is exactly the situation it faces on every new level anyway.

How to read it

It clears → later levels are individually reachable and six levels is a budget-and-transfer problem. It does not → either later levels are genuinely harder or a cold start cannot be recovered in 20 rounds; the design cannot separate those two, so a failure here narrows nothing on its own and would need a matched warm-start run.

D · When exactly did the batching die?

Intent

Digging through old logs turned up something that overturns our working story: on 2026-07-16, on the same prompt surface we use today, runs averaged 2.2–3.1 clicks per round, took up to six in one round, and one reached three levels.

Hypothesis

The regression is not the prompt and not the architecture rewrite — both predate those good runs. Something added between 2026-07-16 and today removed the behaviour.

Method

Check out the code as of that date into a separate working tree and run it against the same seed and bundle; if the old code still batches, bisect the ten-day window commit by commit.

How to read it

Old code batches → a specific commit is responsible and can be found by halving the interval. Old code does not batch → the cause is outside the code, which is itself worth knowing.

05What we know, and what we do not

The next honest milestone is not six levels. It is a run in which the agent, on the level where it currently stalls — the third — commits more than one click in a round because it decided to — and we can say from the log exactly why.