The program behind the weights

Abstract. A recent article by Takehi et al. reported that GPT-6 Astra could write, in one reply and without any tool, the weights of small networks that played MinAtar games better than PPO trained for 100 million steps. In this post I studied what these networks were. On a new game, which no model could have seen, both GPT-6 Astra and Claude Opus 5.5 wrote working networks, and their weights looked nothing like trained ones: 3 to 5% of the parameters were non-zero, with 40 to 62 distinct values. For each of the five written networks I reconstructed, from the game state alone, a short rule-based program that reproduced the network’s choice in every visited state where the network had a clear preference, from 171,311 to 1,456,882 states per network. I then used these networks as initializations for PPO, compiled a classical breadth-first search into weights by hand, and finally asked the same model for the program as code, under the same no-tool conditions. The programs written as code beat the networks written by the same model, and the best PPO networks I trained, in all four games.

Introduction

In Sparks of Weight-Space Programming, Takehi, Teoh, Zhang and Khattab (MIT CSAIL, November 2026) reported eight experiments in which GPT-6 Astra read, repaired or wrote the weights of neural networks: among other things, it told which game a network was trained on, repaired a failing controller, wrote a network that played Go and edited a small language model to improve its math. Among these experiments, this post focused on one, the article’s Experiment 5, “Can the LLM write weights from scratch?”. There, Astra wrote every weight of a 600–32–32–6 tanh network for three MinAtar games: Breakout, Seaquest and Asterix. It received the rules, the source code of the game and the layout of the input and the output, but no data, no gradients, no tools and no feedback. The resulting networks scored 517.4, 53.1 and 53.2 points, against 15.8, 5.6 and 25.8 for PPO networks of the same shape trained for 100 million steps. The article explained the result in one sentence: “Astra reasons about the desired programs it wants each network to implement and converts them into neural form.”

In this post I took that sentence literally and tested it. Two readings of the result were possible. In the first, the model had seen trained networks for these well-known games and reproduced something it remembered. In the second, the model designed an algorithm and compiled it, by hand, into the weights; the network was then a transcription of a program, and the interesting object was the program. The two readings made different predictions, both on the weights themselves and on what would happen when the networks were trained further.

I addressed the question with seven experiments of my own, run between 3 and 6 November 2026 and summarized in Table 1; from here on, “Experiment 1” to “Experiment 7” refer to them, and “the article’s Experiment 5” to the original one. First, I introduced a new game, Low Tide, and asked both GPT-6 Astra and Claude Opus 5.5 to write networks for it (Experiment 1). Afterwards, I measured what the written weights looked like (Experiment 2), and I reconstructed the program inside each of the five written networks, checking the reconstruction state by state (Experiment 3). Then, I trained PPO from scratch on the same games and measured whether its networks could be read in the same way (Experiment 4), and I used the written networks as starting points for PPO (Experiment 5). Next, I did the reverse of Experiment 3: I took a classical algorithm, the breadth-first search used by my Low Tide baseline, and compiled it into weights myself (Experiment 6). Finally, I asked the same model for the program directly, as Python code, with the same restrictions on tools (Experiment 7). The post closes with a discussion of what the evidence supported and of its limits.

All code, weights and logs are in the repository LorenzoCiarpa/small-games-llm; the file STRUCTURE.md maps every section of this post to a folder and a command.

Table 1. Timeline of the experiments. Dates refer to the commits of the repository.

The games

All experiments used four games on a 10×10 grid: Low Tide, written for this study, and three games of MinAtar, a miniature version of Atari games built for reinforcement-learning research. The clips below are evaluation episodes, with the sticky actions of the protocol, played by networks written by an LLM.

Low Tide. Low Tide is a grid world written for this study by Claude Opus 5.5, in the style of MinAtar, so that no model could have seen it, or a network for it, during training. The agent collects shells on a beach whose row 0 is the land side and row 9 the sea side. The water covers the bottom level rows. Every 3, 4 or 5 steps the tide moves by one row: it rises to a high level drawn from 5, 6 and 7, then recedes to a low level drawn from 0, 1 and 2, and so on. Each time the tide recedes, the uncovered row is filled at random with shells (probability 0.25 per cell) and jellyfish (0.08); each time it rises, the flooded row is washed clean. A shell gives +1; a jellyfish gives −1 and paralyses the agent for 3 steps; if the rising tide floods the agent’s row, the agent drowns and the episode ends. The observation has 403 numbers: four binary 10×10 planes (agent, water, shells, jellyfish) and three scalars, namely the direction of the tide, the steps left before the next change (divided by 5) and the remaining paralysis (divided by 3). There are five actions: no-op, up, down, left and right.

Breakout. The player moves a one-cell paddle along the bottom row and bounces a ball, which moves diagonally, to break three rows of bricks near the top. Each brick gives +1, and a new wall appears when the old one is cleared. The episode ends when the ball reaches the bottom row without being caught.

Seaquest. The player steers a two-cell submarine in four directions and fires from its front. Enemy fish and enemy submarines cross the screen, and enemy submarines shoot; hitting an enemy gives +1. The submarine can pick up divers, up to six. Its oxygen runs down and is refilled by surfacing with at least one diver aboard. The episode ends when the submarine is hit, runs out of oxygen, or surfaces with no diver.

Asterix. The player moves in four directions. Enemies and treasures enter from the sides and move horizontally; a treasure gives +1, and touching an enemy ends the episode. Over time, enemies and treasures move faster and appear more often.

The networks did not see the MinAtar games through their own channels but through the observation of the original article, which the clips show exactly: 600 binary numbers, a 10×10 grid with six channels (player, danger, reward, projectile, trail, gauge), each the union of some of the game’s own channels. There are six actions: no-op, left, up, right, down and fire. In the Seaquest and Asterix clips the player never leaves column 5; Experiment 3 explains why.

Setting

In this section, I describe the networks, the evaluation protocol and the baselines shared by all experiments.

Protocol. Every score in this post was the mean return per episode with its standard error, computed by the same reference forward pass, never by code written by a model. Actions were sampled from the softmax of the logits, and with probability 0.1 the environment repeated the previous action instead (sticky actions). Episode k used seed 10000 + k for the game, the sticky actions and the sampling, so that every number can be reproduced exactly. Table 2 reports the network limits and the protocol of each game.

Table 2. Observation, actions, network limits and evaluation protocol of the four games.

The article’s MinAtar networks are public. I converted them to my weight format and re-scored them with the protocol above, obtaining the values of Table 3, all within about one standard error of the article.

Table 3. Scores of the article’s networks written by GPT-6 Astra, in the article and in my framework (1000 episodes each).

Baselines. Three policies that are not networks served as baselines. The first, random, chose an action uniformly at random and existed for every game. The second, heuristic, existed for Low Tide only. It computed the last safe row, equal to 9 − level minus a margin of one row if the tide was rising and two rows if it was about to rise. It then ran a breadth-first search (BFS) from the agent to the nearest shell, avoiding jellyfish and, when the tide was rising, the cells next to them. It was the reference for a good policy, with 164.0 ± 1.5 points, and the starting point of Experiment 6. The third, astra_code, existed for every game: it was the Python function written by GPT-6 Astra in Experiment 7, the strongest policy in all four games, described in that section.

Experiment 1: weights for a game nobody had seen

The first reading of the original result, memory, had a direct test: a game that did not exist before. I gave GPT-6 Astra and Claude Opus 5.5 the same prompt, in a new conversation and with no tools. The prompt followed the structure of the article’s prompt: the task, the rules, the full source code of the environment, the layout of the input and the output, the limits of the network and the scoring protocol.

Both models returned a complete weight file. Claude Opus 5.5 wrote a 403–46–36–5 network (relu, then tanh) with 953 non-zero parameters; GPT-6 Astra wrote a 403–43–61–53–5 relu network with 1,174 non-zero parameters. Table 4 reports their scores together with the baselines.

Table 4. Low Tide: networks written by the two models and baselines (500 episodes). Shells, stings and drowned episodes are per episode; the last column repeats the evaluation without sticky actions. The last row anticipates Experiment 7.

Both networks played the game, at 81% and 49% of the heuristic, and far above random actions. Memory of a trained network could not explain either result, since no such network existed. Moreover, every drowning of both networks was caused by sticky actions: without them, neither network drowned in any of the 500 episodes. The remaining experiments explain where the difference between the two networks came from (Experiment 3) and why the heuristic was still ahead (Experiment 6).

Experiment 2: what written weights looked like

If written networks had been recollections of trained ones, they would have looked like trained networks. I compared the five written networks (two for Low Tide, three from the article) with networks trained by PPO: four public 500–32–32–5 networks trained by the article’s authors for 10⁹ steps on their own grid games, and my own PPO networks for the four games of this post (Experiment 4). Table 5 reports the main statistics, and Figure 1 shows the first two.

Table 5. Statistics of the weights written by an LLM and of weights trained by PPO. The PPO columns refer to the four reference networks of the article’s Experiment 1.

The difference was categorical rather than quantitative. The written networks reused a small vocabulary of round numbers: in each of them, the eight most common values accounted for 34 to 81% of all non-zero weights, while in a trained network almost every weight was a different number. Claude in Low Tide mostly used powers of two (1, 8, 16, 32, 64, 1024), and Astra in Asterix used ±8, ±16, 24 and 1000 as thresholds. In the Low Tide networks, 96.1% (Claude) and 97.5% (Astra) of the non-zero weights were integers, and all of them were multiples of 0.5. No gradient-based training produced weights of this kind, and no trained network in my sample had them.

The memory reading also predicted that networks for related games would share parts. They did not. After aligning the hidden units of one network to those of another (weight matching, as in Git Re-Basin), the three MinAtar networks were as far from each other as from a copy whose weights had been shuffled. The two Low Tide networks computed different internal representations: the linear CKA between their first layers was at most 0.11, against 0.07 for a random network. Behaviour agreed: each MinAtar network scored only in its own game, and at the level of random actions in the other two. The measures are in experiments/weight_distance/.

Overall, the written networks shared a style, namely few round weights and activations used as logic gates and thresholds, but not weights. A remembered network would have looked like a trained one, and these looked like the opposite. This experiment could not tell a strategy remembered from the literature from one derived from the rules; Low Tide, on which both models still produced working networks, was the control for that.

Experiment 3: reading the program back

In this experiment, I reconstructed the algorithm inside each of the five written networks and verified the reconstruction. For each network, I read the weights layer by layer and wrote down what every unit computed as a function of the game state: a measurement (a distance, a margin, a count), a condition (true or false) or a vote for a move. I then wrote this reading as Python code, decode(), that recomputed the units and the logits from the positions of the objects in the game, without using the weights, and compared it with the real forward pass in every state visited by the network in 200 evaluation episodes. Table 6 reports the result of the comparison.

Table 6. Verification of the five reconstructions on the states visited in 200 episodes (seeds 10000–10199, sticky actions, sampled actions). A near-tie is a state in which the two highest reconstructed logits differ by less than 0.5; there the network samples between the two moves.

For Breakout I went one step further: the formulas of the reading regenerated the whole weight file, and the regenerated file coincided with the original in all 20,486 parameters. The small logit differences in Table 6 came from saturated tanh units, which the reading treated as exact ±1. For Seaquest and Asterix, a simplified rule without any weight agreed with the network in 97.4% of the states (99.1% where the network had a clear preference) and in all but 13 of 133,412 states, respectively; played instead of the network, the Asterix rule scored 55.7 ± 1.1 against 54.8 ± 1.1.

The five programs are shown below, one per tab, as compact pseudo-code; the README of each decode_* folder explains every unit, with the indices of the weights.

m = 9 - row - level                       # dry rows between the agent and the water

if tide_falling:
    if m >= 1:
        move DOWN                         # follow the receding water
                                          # (step sideways if a jellyfish is below, on the shoreline)
    else:                                 # m == 0: sweep the shoreline, where new shells appear
        if shell_right_in_my_row:  move RIGHT
        elif shell_left_in_my_row: move LEFT
        else:                      wander (left or right at random)
        # never towards a jellyfish immediately beside me

if tide_rising:
    if m <= 1:  move UP                   # escape: priority over everything
    elif m == 2:
        shells in my row (right first, then left), else shell above, else wander; never DOWN
    else:                                 # m >= 3
        shells in my row, else shell above, else DOWN (back to m = 2)

WINDOW = rows 3..8 x columns 6..9         # 24 of the 70 cells of the tidal zone

if tide_rising and m <= 1:
    move UP                               # logit + 1000 (m = 1) or + 2000 (m = 0)
else:
    for move in (UP, DOWN, LEFT, RIGHT):
        score[move] = push_towards_window(move)          # +12.75 back in, -12 out, 0 or 0.75 inside
                    + 0.9 * sum(6 - manhattan(agent, s)  # every visible shell in that direction
                                for s in shells_in_window
                                if not (tide_rising and margin(s.row) <= 1))
                    - 25 * jellyfish_in_destination(move)
    if tide_rising and m <= 2:
        DOWN is forbidden                 # logit - 1000 * (3 - m)
    score[NOOP] = -4
    sample a move from softmax(score)

# inputs: p = paddle column; (r, c) = ball; (tr, tc) = trail, the ball's previous cell
# the bricks are not read: their weights are all zero

def landing_column(r, c, dx, dy):        # precomputed into the weights: 4 directions x 100 cells
    x, y = c, r
    while dy < 0:                        # rising: assume ALL bricks are still there
        x, dx = wall_step(x, dx)
        if y - 1 < 0 or 1 <= y - 1 <= 3: dy = +1     # ceiling or a brick row: bounce
        else: y -= 1
    while y < 8:                         # fall to row 8, bouncing off the side walls
        x, dx = wall_step(x, dx); y += 1
    if falling_already and r >= 6:       # close: aim between the two columns that catch it
        return (x + wall_step(x, dx)[0]) / 2
    return x

def act():
    right = c > tc or (c == tc == 0)     # direction from the trail (units h4, h5)
    down  = r > tr or (r == tr and r != 8)
    delta = landing_column(r, c, ±1, ±1) - p          # units h0-h3: tanh(0.02 * delta)
    if delta <= -1: return LEFT          # logits +30 / -30 / 0: the move is near-certain
    if delta >= +1: return RIGHT
    return NOOP

danger      = rows with an enemy in columns 4-6 or an enemy torpedo in columns 3-7   # veto
diver_rows  = rows 2..8 with a diver                    # row 1 is invisible to the network
target_rows = rows 2..8 with something in columns 0-3   # the line of fire of a left-facing sub
need_surface = oxygen < 60 and divers_aboard >= 1        # read from two cells of the gauge row

def value(k):                                           # value of ending the step in row k
    v  = sum(1.2 * 0.75 ** abs(j - k) for j in diver_rows)         # divers attract from afar
    v += sum(0.7 * 3.0 ** -abs(j - k) for j in target_rows)        # targets only nearby
    return v - 6 * (k in danger)

def act():
    if r == 1 and need_surface: return UP               # surface
    candidates = {FIRE: value(r)}                       # stay and shoot left
    if r >= 3 or need_surface: candidates[UP]   = value(r - 1) + 1.5 * need_surface
    if r <= 7:                 candidates[DOWN] = value(r + 1)
    return FIRE on a tie (within 0.1), else the best candidate

# d(k) = cells the entity of row k still has to travel to reach column 5,
#        None if it is moving away (direction read from its trail) or the row is empty
ATTRACTION = {1: 12, 2: 8.0, 3: 5.3, 4: 3.1, 5: 1.8, 6: 1.1, 7: 0.8}    # by row distance

def alarm(move):     # incoming enemies close to column 5 in the arrival row and in the row after;
                     # about 0 = safe, 1 = doubtful, 2 = deadly; a move into the wall counts as 1

def act():
    # -80 * tanh(alarm - 1): +61 with no alarm, 0 with alarm 1, about -60 with alarm 2
    logit[NOOP] = 1    - 80 * tanh(alarm(NOOP) - 1) + 32 * (incoming treasure in my row)
    logit[UP]   = 0.15 - 80 * tanh(alarm(UP)   - 1) + sum(ATTRACTION[r - k] for treasures above)
    logit[DOWN] =        -80 * tanh(alarm(DOWN) - 1) + sum(ATTRACTION[k - r] for treasures below)
    logit[LEFT] = logit[RIGHT] = logit[FIRE] = -1000
    sample a move from softmax(logit)

The two Low Tide networks were therefore two different programs for the same game. Claude’s saw a little of everything, everywhere, and had a strategy tied to the tide cycle; Astra’s saw a small area well, with true distances, but stayed locked inside it. Table 7 compares them.

Table 7. The two Low Tide networks compared (500 episodes, sticky actions).

Experiment 4: PPO from scratch, and whether its weights read as programs

In this experiment, I trained networks of the same shapes with PPO and applied to them the measures that made the written networks readable. The comparison mattered for two reasons. First, the original article compared the written networks with PPO, whose networks are not public, so I needed my own. Second, if trained networks had been as readable as written ones, the readability of Experiment 3 would have said nothing specific about how an LLM writes weights.

PPO was a reference, not an equal opponent. The LLM received the rules and the full source code of the game, so it knew the dynamics, the rewards and the meaning of every input before it wrote a single weight. PPO knew none of this: it started from random weights and learned only from the rewards it collected, in the same way for every game. A written network that beat PPO therefore showed that knowledge of the rules could replace many millions of steps of experience, not that the LLM was a better learning algorithm. In this post, PPO served as a reference for what learning from experience reached with a given budget, and as a source of trained weights to compare with the written ones.

Trainer. I used clipped PPO with generalized advantage estimation, following CleanRL. The actor was exactly the network read by the reference forward pass, so that a trained network was scored like a written one; the critic was a separate 64–64 tanh network. All runs shared learning rate 5·10⁻⁴ with linear decay, 64 environments × 128 steps per update, 4 epochs of 8 minibatches, clipping 0.2, γ = 0.99, λ = 0.95, and the sticky actions of the protocol. Training seeds started at 1,000,000 and never overlapped with the evaluation seeds. The runs used one CPU core each, with up to ten runs sharing eight cores.

Table 8 reports every run. The first MinAtar attempt (ppo_v1, 30 million steps, entropy coefficient 0.01, all six actions) stayed on a plateau in all three games, with 5.8, 5.4 and 9.1 points. In Breakout four of the six actions do nothing, and PPO converged to a policy that almost never moved; a 256–256 actor, outside the rules, reached the same 6.8 points after 6 million steps, which indicated that the bottleneck was exploration rather than capacity. Restricting Breakout to its three useful actions, Asterix to five, and raising the entropy coefficient to 0.05 gave the best runs, with 200 million steps each.

Table 8. All PPO runs from scratch. Scores on the full protocol (500 episodes for Low Tide, 1000 for MinAtar); training time in hours on one core. The best run of each game is in bold.

On Low Tide, PPO matched the BFS heuristic after 100 million steps (166.9 against 164.0) and beat both written networks. On MinAtar, my best runs exceeded the article’s 500-million-step PPO on Breakout (63.2 against 25.3) but not on Seaquest (55.9 against 88.9) or Asterix (24.6 against 55.2), and remained far from Astra’s Breakout network. My MinAtar budget was therefore a weaker baseline than the article’s on two games out of three.

Readability. Table 9 compares the final PPO networks with the written ones on four measures: how many inputs a first-layer unit needs to reach 90% of its absolute weight; how much of the score survives when the first layer keeps only its largest 25% weights; the best correlation between a single unit and a quantity of the written algorithm, such as the margin m in Low Tide; and how often a decision tree of depth 6, built on those quantities, picks the network’s most likely move.

Table 9. Readability of written and trained networks. Live inputs are those that are not constant in play: 312 of 403 in Low Tide, 220, 425 and 315 of 600 in Breakout, Seaquest and Asterix. Scores over 200 episodes.

The written networks kept their whole score after pruning and contained units that tracked the quantities of their algorithm. The PPO networks spread every unit over most of the inputs, lost almost all their score when pruned, and contained no unit with a recognizable meaning; the only readable traces were position codes, such as the paddle column in Breakout.

Over training. The same measures on every checkpoint of every run showed no trend toward the written networks: the networks stayed dense from the first checkpoint to the last, and the more a run learned, the less of its score survived pruning. Without a pressure toward few or small weights, gradient descent moved every weight a little and found dense solutions; the written networks were sparse because their author started from an algorithm. The full measures are in experiments/ppo_over_training/.

Experiment 5: written networks as a starting point for PPO

If a written network was a good program in a poor parametrization, PPO might have used it as a head start. In this experiment, I started PPO from each written network and compared the run with PPO from scratch with exactly the same architecture, learning rate and best entropy coefficient (0.05 on MinAtar, 0.01 on Low Tide), for 200 million steps on MinAtar and 100 million on Low Tide. Since the critic started random while the actor was already good, I also ran the Low Tide networks with a critic-only warm-up of 2 million steps. Figure 2 shows the evaluation of the checkpoints, and Table 10 summarizes the outcome.

Table 10. PPO started from the written networks. Final scores on the full protocol. The steps saved are those that PPO from scratch needed to reach 95% of its own final score, minus those the warm start needed.

The outcome depended on the game, and the gradient at the starting point explained most of it. Table 11 reports the norm of the actor’s gradient at fixed weights, with a critic fitted for 2 million steps to the fixed policy, for each written network and for the final PPO network of the same shape.

Table 11. Actor gradient at fixed weights (full batch of 8192 steps, before clipping), weight norm, entropy of the policy, and how far training moved the weights.

Breakout was a plateau in the literal sense. Astra’s network gave each chosen move a margin of 30 logits, so the softmax was saturated and the policy had zero entropy: the gradient was 4·10⁻¹³, and after 200 million steps the weights had moved by 7·10⁻⁶. The run reproduced the written network exactly. The start was worth eight times PPO from scratch, so the steps saved were a lower bound.

Seaquest was the clear gain. The score first dropped to 33.3 at 5 million steps, was back at Astra’s level at 20 million and 10% above it at 25 million, and ended at 82.5, 60% above the written network and above PPO from scratch at any budget I ran. The final policy chose the same move as Astra’s network in only 37% of Astra’s states, and 51% of Astra’s zero weights became non-zero: PPO used the written network as a starting point and moved away from it.

Asterix recovered but did not improve. The score dropped to 38.9 at 15 million steps and returned to Astra’s level only at 140 million, with noise; the final 50.8 was 7% below the written network, but still twice PPO from scratch.

On Low Tide the written networks were harmful starting points. In the first few million steps PPO erased them: Astra’s fell to 1.3 and Claude’s to 6.3. Astra’s then learned more slowly than from scratch and ended at 147.3 against 166.9; Claude’s never recovered. The weights of these networks were huge, with norms from 1,600 to 45,600 against 11–17 for a random initialization, and the KL divergence between consecutive policies in the first million steps was 20 to 45 times that of a run from scratch. A critic-only warm-up did not help: Astra’s network ended at 0.

Overall, a written network was a good initialization when it was a soft policy whose program PPO could refine (Seaquest), useless when its softmax was saturated (Breakout), and harmful when its weights were so large that the first updates destroyed the program (Low Tide).

Experiment 6: from an algorithm to weights

The decodings of Experiment 3 went from weights to an algorithm. In this experiment, I went the other way: I took the algorithm of my Low Tide heuristic, the breadth-first search (BFS) described in Section Setting, and wrote it into the weights of an MLP myself, without training: every weight was given by a formula.

Within the limits of the game, only a variant could fit. The exact BFS did not fit in three hidden layers of 64 units, for three reasons. A first-layer unit computed a fixed weighted sum of the inputs, so a question relative to the agent’s position needed one unit per cell; a path around a jellyfish advanced by one cell per layer, while the BFS paths in the evaluation episodes reached 26 steps; and “the nearest shell” was a minimum over 60 cells, which two relu layers could not compute exactly. I therefore wrote a potential-field variant, bfs_net (403 → 64 → 58 → 63 → 5, 6,806 non-zero parameters). Every shell attracted the agent with a weight that decreased steeply with its Manhattan distance, so that the nearest shell usually dominated; the direction with the strongest attraction won, and a move into a jellyfish was vetoed. The network computed this rule exactly in all 456,230 states of 500 evaluation episodes. It was not the BFS, since it made the BFS move in 78.8% of the states, but it played as well: 166.7 ± 2.0 points against 164.0 ± 1.5.

With more neurons, the exact BFS fitted. Without the limits of the game, the BFS could be written exactly. The network bfs_exact propagated a wave that started from the reachable shells and advanced by one cell per layer through the cells the agent could enter, so that each cell learned its distance from the nearest shell; the last layers moved the agent to the neighbouring cell with the smallest distance, breaking ties in the BFS order (up, down, left, right). The number of wave layers, K, was the longest distance the network could measure: a cell at distance d from the nearest shell was reached by the wave after d layers, so after K layers every distance up to K was known, and the network made the BFS move whenever the nearest shell was within K + 1 steps of the agent. With K = 98, which covered any state of the 10×10 grid, the network had 104 hidden layers and 51,103 neurons, against the 192 the game allows. It made the same move as the heuristic in all 470,889 states of the 500 evaluation episodes, and therefore played the same episodes with the same score. Even K = 25, enough for the states of the evaluation episodes, would have needed 31 layers and 14,384 neurons.

The construction of the two networks, the contribution of each part of the potential field and the size of the exact network as a function of K are described in the READMEs of bfs_net and bfs_exact.

Experiment 7: the program instead of the weights

The previous experiments indicated that an LLM writing weights designed a program and compiled it by hand. In this experiment, I asked the same model for the program directly. I derived the prompts from the article’s own prompts for its Experiment 5, which are public, keeping word for word the description of the game, the exact MinAtar source code, the input and the mapping of the channels, and replacing only the parts about the network: the task, the output, the network itself, the scoring and the format of the answer. The model had to write a Python function policy(obs, rng) that returned an action. The function could use only the standard library and numpy, could not import the game, had to take at most about one millisecond per call, and could not keep any memory between calls: like the networks it was compared with, it saw only the current observation. For Low Tide I used the same structure, with the source of the environment and without the heuristic.

No tools, verified. I ran GPT-6 Astra through the Codex CLI with reasoning effort xhigh, as the article did. The article’s configuration, which turned off every Codex feature that gave the model a tool, was no longer sufficient with the Codex version I used (0.164): the model still received code execution, agent tools, a question tool and system skills, which the model catalog of Codex enabled for GPT-6 Astra. I removed them with a modified copy of the catalog and checked, before each run and without calling the model, the exact input it would receive: no tool and only two messages, the neutral instruction of the article (“Answer the user’s request.”) and the prompt. A control question (“which tools can you call?”) received 59 input tokens, against 5,098 with the article’s configuration alone, and the answer “NONE”. Each of the four answers took 14 to 23 minutes and contained no tool call.

Table 12 and Figure 3 report the scores of the four functions, evaluated with the same protocol as the networks.

Table 12. Code written by GPT-6 Astra, one answer per game, against the network written by the same model and the best PPO networks (500 episodes for Low Tide, 1000 for MinAtar).

In all four games the code beat the network written by the same model, and every PPO network I trained or that the article reported. All four functions passed the same checks on 2,000 states: they imported only numpy and the standard library, raised no error, returned valid actions, took 95 to 283 microseconds per call, and gave the same actions when the states were presented in reverse order with the module reloaded, so they kept no memory.

All four were planners built on an exact model of the game, derived from the source code in the prompt. The Low Tide function computed, for each cell, the last step at which the agent could reach it and still return to dry land, under the worst-case timing of the tide, checked each move against repetitions by sticky actions, and chose a route among the shells with a beam search; it drowned in 0.6% of the episodes, against 11% for the BFS heuristic, and collected 190.4 shells per episode. The Breakout function simulated the ball exactly and searched five paddle bounces ahead with dynamic programming over the paddle position under sticky actions; 96.7% of its episodes reached the limit of 10,000 steps. The Seaquest and Asterix functions used short-horizon collision models and a separate safety search. These programs searched over future states, which none of the written networks did.

One answer per game. I asked for each function once. With a single answer per game, I could not say how much the score would change if I asked again: the numbers in Table 12 describe these four answers, not an average over the model’s answers.

Discussion

In this post, I examined what the networks written by an LLM were, by testing the explanation given in the original article in four ways.

What the evidence supported. In all five networks I examined, the weights were the exact transcription of a short rule-based program: I rewrote each program from the game state alone, and it reproduced the network’s choice in every visited state outside near-ties, from 171,311 to 1,456,882 states per network. The weights did not look like trained weights, the networks for different games shared no weights, and on a game created for this study both models still wrote working programs. Two models given the same prompt wrote two different programs, with different devices (binary comparisons with powers of two for Claude, relu ramps and gates for Astra) and different strategies. The evidence therefore supported the reading proposed by the original article, in a precise form: the model designed a program and compiled it by hand into the network; the network was not a solution found by searching the space of weights, nor the memory of a trained network.

What followed from it. The compilation had a cost. A network with three layers of 64 units could not hold a search, so the programs that fit were simple and greedy: Astra’s Low Tide network watched a window of 24 cells, most likely because the units did not suffice for 70, and the exact BFS needed 51,103 neurons instead of 185. When the same model wrote the program as code, without that constraint, it wrote planners that beat its own networks and every PPO baseline in all four games. The written networks were also uneven starting points for PPO: helpful in Seaquest, inert in Breakout, harmful in Low Tide.

Did it make sense to write an algorithm as a network? For playing these games, my results said no. A network that reproduced an algorithm exactly was the same function: bfs_exact played the same 500 episodes as the BFS heuristic, with the same score, using 104 layers and 51,103 neurons in place of a short function. A network of fixed depth had to unroll every loop to its worst case and to enumerate positions that code addresses directly, so within the limits of a game only simple programs fit, and the same model wrote better programs when it could use code. The representation added no playing strength of its own: when a written network beat a baseline, as bfs_net did by a few points, the gain came from the different algorithm it implemented, which code expresses just as well. What a network offered and code did not was a gradient. A written network could therefore make sense as a starting point for training, as in Seaquest, where PPO started from Astra’s network ended 60% above it and above every run from scratch. Experiment 5 also showed the conditions: the softmax could not be saturated, or the gradient vanished (Breakout), and the weights had to be on a scale the optimizer could handle, or the first updates destroyed the program (Low Tide). Two further uses were plausible, but I did not test them: a setting that accepts only network weights, and networks whose program is known exactly, as ground truth for interpretability methods.

Limits. Every network and every code answer in this post came from a single request: I never asked the same model twice for the same game. I therefore could not say how much the results would have changed with another answer. Low Tide was designed by Claude Opus 5.5, one of the two models that wrote its networks. The code experiment covered GPT-6 Astra only. My PPO baselines used 200 million steps on MinAtar, against 500 million in the article, and were weaker than the article’s on Seaquest and Asterix. Finally, the comparison with PPO was uneven by construction (Experiment 4): the LLM read the rules and the source code, while PPO learned from rewards alone.

Future work. Several extensions follow directly. The first is to collect more written networks, from other LLMs and from repeated requests to the same ones, to see which algorithms they implement and how much they vary, and to run the code experiment with Claude as well. The second concerns learning from experience: in this post PPO trained only small networks of fixed shapes, and it would be worth measuring how the comparison changes with other architectures and with other algorithms that learn from experience, such as value-based methods like DQN. The third is to train from the written networks with optimizers other than PPO. Evolution strategies such as CMA-ES need no gradient, so a saturated softmax, as in Breakout, does not block them, and distilling the written program into a softer network before reinforcement learning could keep the program while making it trainable.

Conclusion

The experiments in this post showed that LLMs can natively write programs in the form of neural networks: the weights they wrote were the exact transcription of a rule-based program, readable unit by unit, also for a game that no model had seen. They did not show that LLMs can initialize the weights of a neural network for training: as starting points for PPO, the written networks improved in Seaquest, stayed frozen in Breakout, dropped and only recovered in Asterix, and were erased in Low Tide.

Reproducibility

The repository LorenzoCiarpa/small-games-llm contains the games, every weight file, the training logs and checkpoints, the decodings with their verification scripts, and the answers of the models. STRUCTURE.md maps each section of this post to a folder and a command, for example:

uv sync
uv run evaluate low_tide claude gpt --baseline heuristic           # Experiment 1
uv run python experiments/low_tide/decode_claude/check.py          # Experiment 3
uv run python experiments/low_tide/bfs_net/check.py                # Experiment 6
uv run evaluate breakout paper_astra --baseline astra_code         # Experiment 7
uv run python experiments/article/videos.py                        # the clips of the games

The networks of the original article and its prompts are from RikiyaT/lm_understand-logs (CC BY 4.0), and the MinAtar games from the official package, version 1.0.15.

A final note

Before I started examining the article by Takehi et al., I knew nothing about LLMs writing programs directly into the weights of a neural network. Everything in this post was learned along the way.