🔍 Read the full analysis: Can AI Write Neural Networks From Their Weights? What This Work Shows on ThorstenMeyerAI.com
Get business pricing on office and shipping supplies
- Business-only prices and quantity discounts
- Tax-exempt purchasing
- Multiple users, one account, clear invoices
TL;DR
Four MIT CSAIL researchers report that OpenAI’s GPT-6 Astra could identify what some small neural networks do, repair selected failures and write new networks by directly reasoning about their weights. The results include a repaired car-control model and improved maths scores for one looping language model, but the work is a blog-post report with limited experiments, and the method did not help every model.
Four researchers at MIT CSAIL report that OpenAI’s GPT-6 Astra identified, repaired and created small neural networks by reasoning directly about their numerical weights, without using gradient descent or a training loop. The experiments, published in a November 2 blog post, include a car-control network that began reversing successfully after an edit and a language-model maths score that rose sharply; the authors also document failures and say the approach cannot give a model an ability it does not have.
The researchers call the method weight-space programming: a language model receives the numerical weights that define another neural network and tries to infer what they do or alter them. In some tests, Astra received weights in plain text and answered in one reply. In others, researchers ran Python code written by the model in a restricted sandbox with a fixed testing budget. The team says its released records contain 1,125 tool-free turns; the setup disabled Codex tools such as shell access, web search and file editing.
Among the repair results, a car-control network that recorded zero successful reversals in 256 episodes reached 1,024 successes in 1,024 episodes after Astra changed 3.5% of its weights, according to the post. In a separate test, an edit to 0.016% of Qwen3.5-0.8B’s weights raised its score on MATH-500 from 7.9% to 48.1%. A Hugging Face engineer later rebuilt the edited Qwen model from the released logs and reported a 51.0% score on the same benchmark.
The team also tested whether Astra could infer what networks did from their weights. It named the games behind four of four small game-playing models and identified the general shape produced by three of three small image generators. In three tiny language models, it found every copying head, a component associated with repeating patterns from earlier text. For generation, it wrote a 9-by-9 Go network with about 1.5 million weights from all-zero weights; the researchers say it beat a same-size comparison network trained on about 20.5 million games in 50 test runs.
Infographic 01 · Eight experiments
Read, fix, write: what GPT-6 Astra did with raw weights
Every experiment in the weight-space programming post, with the size of the network, the result and how it went.
Read
- 1Which game was this network trained on?4 reinforcement-learning policies · 17,253 weights each · weights as text, no toolsRight game for 4 of 4, without a list of options✓ Worked
- 2What does this image model draw?3 tiny diffusion models · 23,824 weights · no forward pass allowedRight shape for 3 of 3, but drawn as outlines; the models draw filled shapes✓ Worked
- 3Which language did this model learn?3 character-level transformers · 7.16M weights · middle blocks only, never runRight language 3 of 3; every copying head found, none invented✓ Worked
- 7Which of two models is sycophantic?3 pairs of Gemma 3 270M fine-tunesWeights alone: could not tell in 4 attempts. With attention patterns: 3 of 3! Partly
Fix
- 4Repair a car that never backs upKinetix control network · 171,664 weights · one reply, no tools0 of 256 episodes before, 1,024 of 1,024 after; 3.5% of weights changed✓ Worked
- 8Make Qwen3.5-0.8B better at maths0.8 billion weights · Python over the weights · 10 / 25 / 50 test passesMATH-500 7.9% to 48.1% with 50 passes (12.3% and 14.1% with 10 and 25)! Partly
- 8bSame task, a model that does not loopGemma 3 270M · 5 to 50 test passesMATH-500 6.8% to at most 7.7%: “not established”✕ Failed
Write from zero
- 5Write arcade-game networks3 MinAtar games · 20,486 weights, all starting at 0 · one reply, no toolsBeat the 100M-step comparison networks in 3 of 3, the 500M-step ones only in Breakout! Partly
- 6Write a 9×9 Go network1,480,274 weights, all starting at 0 · no training, no search0 passes: 54% vs small trained nets. 50 passes: 70–79% vs same-size nets trained on ~20.5M games✓ Worked
✓ Worked · ! Partly or with conditions · ✕ Failed. One run per setting, no error bars. A blog post with complete logs, not a peer-reviewed paper.
Sources: Takehi, Teoh, Zhang, Khattab (MIT CSAIL), blog post, 2 Nov 2026, lm_understand-logs (CC BY 4.0). Model: OpenAI GPT-6 Astra via the Codex CLI, built-in tools switched off.
Powered by Thorsten Meyer AI
What Direct Weight Editing Could Change
If the reported results hold up in broader testing, direct weight inspection could give researchers a way to investigate models without relying only on their outputs. For small open models, a system that can locate a narrow fault and make a targeted edit could offer another option for debugging or auditing. The Qwen result is notable because the reported change affected a tiny share of the model’s weights while coinciding with a substantial increase on one benchmark.
The limits are as relevant as the gains. The researchers say Astra could not distinguish an honest model from a sycophantic one by examining weights alone. Editing also failed to improve maths performance in Gemma 3 270M, which did not show the looping problem. The authors’ explanation is that a patch may address a fault in an existing capability, but cannot add an ability the model lacks. These tests do not show that weight-space programming can reliably improve large deployed systems or replace conventional training.
For companies, the work raises both a possible research tool and a practical security concern: if models can be edited by changing weights, organizations will need to track the origin and integrity of the weight files they use. For ordinary users, the report does not describe a product or a change to services available today. Its immediate value is as a small set of logged experiments that other researchers can inspect and attempt to reproduce.
Products Worth Considering
Samsung 34-Inch Odyssey G5 Ultra-Wide Curved Gaming Monitor
Apple MacBook Neo 13-inch Laptop with A18 Pro Chip, 8GB Memory, 512GB SSD, and Touch ID – Blush
Canon imagePROGRAF PRO-1100 17-Inch Professional Wireless Inkjet Photo Printer
As an affiliate, we earn on qualifying purchases.
From Software 2.0 to Weight-Space Tests
The work builds on a familiar idea in machine learning: a trained network’s behavior is encoded in its weights, but those numbers are not normally readable like conventional source code. Training algorithms adjust weights over many iterations, making the resulting model difficult to interpret directly. In 2017, Andrej Karpathy described neural networks as “Software 2.0,” with programs expressed in weights rather than written line by line.
The MIT group—Rikiya Takehi and Jayden Teoh, who shared equal contribution, alongside Alex Zhang and Omar Khattab—asks whether a language model can work with those weights as a form of code. The researchers write that neural-network weights may be “natively optimized by LLMs that deliberately reason about their weights.” That is the authors’ broader proposal, not a conclusion established by the limited tests in this report.
The source is a research-group blog post with public logs, not a peer-reviewed paper, and the supplied material says it is not on arXiv. Its experiments used small networks, generally one model family per task, and a single research group. The detailed records and the independent Qwen rerun provide material for scrutiny, but they do not substitute for repeated tests across teams, models and tasks.
“Software 2.0”
— Andrej Karpathy, describing neural networks in 2017
How Far the Results Generalize
It remains unclear whether other research groups can reproduce the full set of results or whether the approach works on larger networks and more varied tasks. The source describes one run per setting, small models and one research group; it does not provide evidence that the reported outcomes are consistent across repeated trials. The independent Qwen rerun supports that particular result, but it does not verify the other experiments.
The post also cannot establish how well Astra’s inferred explanations match the internal computations of each network, or how dependable its edits would be outside the chosen tests. The supplied material says the model failed to distinguish honest and sycophantic behavior from weights alone and did not improve Gemma’s maths score. Whether the method can be applied safely to larger, deployed models—and whether edits could cause new failures—remains unanswered.
Reproduction Is the Next Test
The immediate next step is independent scrutiny of the released logs and model edits. Researchers can check the stated procedures, try to reproduce the results, and test whether weight-based diagnosis works on additional models and tasks. Repeated runs would help establish how often Astra succeeds, rather than showing what happened in a single setting.
The source material does not identify a scheduled paper, formal review or follow-up experiment. Until broader results are available, the findings are best treated as an early demonstration, not evidence that AI systems can generally program or repair neural networks from weights. The authors’ claims and the limits recorded in their own experiments will both need testing.
Key Questions
What is weight-space programming?
Weight-space programming is the researchers’ term for having a language model inspect or change another neural network’s numerical weights directly, rather than using a conventional training loop or gradient descent.
What did Astra repair?
In one experiment, Astra edited a car-control network that had recorded no successful reversals in 256 episodes. The researchers report that it then succeeded in 1,024 of 1,024 episodes, with 3.5% of the weights changed.
Did the method improve every model?
No. The researchers report that an edit did not improve maths performance in Gemma 3 270M, which was not exhibiting the looping behavior targeted in the Qwen experiment. They say editing could not add an ability the model lacked.
Has the work been peer-reviewed?
The supplied source describes the findings as a blog post with public logs, not a peer-reviewed paper, and says the work was not on arXiv at the time of publication.
Does this change AI tools for users now?
The report describes research experiments, not a released user-facing product. It does not indicate that existing AI services or tools have changed for users.
Source: ThorstenMeyerAI.com
