LLMs can understand and write neural network weights

The repository, titled 'lm_understand-logs,' serves as a complete record of the experiments detailed in the blog post 'Sparks of Weight Space Programming: LLMs can understand and write neural network weights.' It includes everything sent to and received from the model, along with the code it executed and the evaluation data behind every number reported in the original post.

Each experiment folder contains its own README, which clarifies which run corresponds to a specific tab or row in the blog post and which file contains each numerical result. The logs cover only the runs that were reported in the original blog post, with earlier trial runs left out.

Model configuration and audit

The experiments were conducted using OpenAI's gpt-6-astra model, referred to as 'Astra' in the blog post. Each turn represents a single call to the Codex CLI, either as a new execution or a continuation of a conversation thread.

A strict configuration was applied to ensure the model's behavior was purely linguistic. All built-in Codex capabilities that provide tools, such as shell access, file editing, web search, and sub-agents, were switched off. The system instructions were a single neutral line, and every call explicitly named the model and the effort level, overriding defaults.

Every turn was audited to verify it was tool-free. An audit was successful when the event stream contained only the model's messages, its reasoning summaries, and Codex's notice that code mode was disabled. The audit result is recorded in the 'tool_free' field of each turn's run. file. All 1,125 turns in this release are confirmed to be tool-free.

Code execution and output

For experiments where code was allowed, specifically Experiments 3, 6, 7, and 8, the model wrote one Python block per turn. These blocks were executed in a custom sandbox environment that used bubblewrap for isolation, had no network access, maintained a working folder between runs, and enforced a time limit per execution.

The printed output from each run was sent back as the next message in the conversation. The code and its output are stored in the 'tool_calls.l' file for each experiment. The specific actions allowed for the code are detailed in the experiment's prompt and its README.

Included and excluded data

The repository includes the final networks and weight edits, with Experiment 8's data provided as exact diffs against a public base weights file, along with a script to apply them. However, it does not include the Gemma 3 270M fine-tunes from Experiment 7 due to their size (17 GB), though the data, training script, and hashes are present for reconstruction.

Intermediate files created by the model, such as activations, generated-text dumps, and draft networks, are not included. The text of Experiment 3's Python corpus is also not included; instead, a manifest and a rebuild script are provided. Third-party benchmark data, including MATH, GSM8K, IFEval, and MMLU-Pro, retain their own licenses.