Weight-Space Programming: how GPT-6 Astra edited Qwen's weights and took MATH-500 from 7.9% to 48.1%

On 2 November 2026 a team at MIT CSAIL published Sparks of Weight-Space Programming, and within a day my feed was full of the same claim: GPT-6 Astra opened up a small open model, Qwen3.5-0.8B, changed 0.016% of its weights by hand, and its MATH-500 score went from 7.9% to 48.1%. No fine-tuning, no gradient descent, no training loop.

That claim is true, but the short versions leave out the parts that matter: what the edit actually was, why it worked on Qwen and failed on Gemma, how long it took, and what you can run yourself today. I read the write-up, the run logs on GitHub and the released checkpoint, and rebuilt the core of the edit in a few lines of numpy. This post is everything I found.

Weight-space programming in one paragraph

Weight-space programming is a large language model reading and writing the raw weights of another neural network directly, from a plain-language goal, "not through an optimizer, and in particular not with gradient descent" in the authors' words. The editor model treats a network's weight tensors the way a programmer treats source code: it reads them, forms a theory about what part does what, and writes a patch. The authors are Rikiya Takehi, Jayden Teoh, Alex Zhang and Omar Khattab at MIT CSAIL, and the editor model in every experiment is OpenAI's GPT-6 Astra, used as-is, with no extra training for this job.

• Who: MIT CSAIL (Takehi, Teoh, Zhang, Khattab).
• When: 2 November 2026, as a blog post and an alphaXiv preprint. Not peer reviewed yet.
• Editor model: GPT-6 Astra, a general-purpose frontier LLM.
• Headline result: Qwen3.5-0.8B on MATH-500 went from 7.9% to 48.1% after 131,072 weights (about 0.016%) were changed.
• Other results: a stuck driving policy fixed with a 3.5% patch (0% to 100% success), game networks written from scratch that beat PPO on Breakout by about 20x, and games identified from raw weights alone.
• Open artifacts: full run logs and rebuild code on GitHub, and the edited model on Hugging Face under Apache-2.0.

New to this? Six terms that make the rest easy

You do not need a machine learning background to follow this post. These six ideas are enough.

• Weights: the numbers inside a neural network. Qwen3.5-0.8B has about 0.8 billion of them. Everything the model knows and does is stored in these numbers, the way a program's behaviour is stored in its code.
• Training and gradient descent: the usual way weights get set. You show the model examples, measure how wrong it is, and nudge every weight a tiny bit in the direction that makes it less wrong, millions of times. Nobody chooses any single weight by hand.
• Forward pass: running the model once on an input to see what it outputs. In this research a forward pass was the scarce resource: the editor could look at the weights for free, but running the model cost budget.
• Residual stream: the running list of numbers (1,024 per token in Qwen3.5-0.8B) that each layer reads from and writes to. Think of it as the model's working memory as it moves through its 24 layers.
• A direction: a particular pattern across those 1,024 numbers. Research since 2024 has shown that some behaviours, such as refusing a request, are controlled by a single direction. Remove it and the behaviour goes away.
• MLP and down_proj: each layer has a small feed-forward network (the MLP). Its last matrix, down_proj, is what writes the MLP's result back into the residual stream, which makes it the natural place to cut a direction out.

The headline experiment: fixing Qwen3.5-0.8B's thinking loops

Qwen3.5-0.8B is a tiny open-weight reasoning model with thinking mode on. Its known weakness is that it gets stuck in thinking loops: it restates a step, doubts it, restates it again, and keeps going until it hits the token limit. In the evaluation setup (32,768 max new tokens), 93.0% of its MATH-500 answers ran to the limit, which is why it scored only 7.9%. Most of the time it never wrote an answer at all.

Astra was given a Python kernel with the weights loaded, no GPU, and a budget of forward passes. One forward pass meant running the real model on a batch of practice problems (on average 64, at most 512) taken from the GSM8K and MATH training sets, never the test sets. Training, gradients and fitting to data were not allowed. It had to look, think and edit.

The first forward pass told the story: 85 of the first 96 practice answers ran to the 32,768-token limit. From there Astra worked out that the looping was driven by one direction in the model's hidden state, written mostly by the MLP blocks of layers 15 and 19. So it removed that direction.

What the edit actually did

Every transformer layer has an MLP whose last matrix, down_proj, writes its output into the residual stream, the 1,024-number vector that carries information from layer to layer in this model. Each column of down_proj is one vector that a single MLP neuron adds to that stream when it fires.

Astra found a direction u in that 1,024-dimensional space that tracked the looping behaviour. It scored every column of down_proj in layers 15 and 19 by how strongly it writes along u, took the 64 loudest columns per layer, and subtracted the u component from each of them:

W' = W - u (u^T W)        applied to the 64 selected columns only

2 layers x 64 columns x 1,024 rows = 131,072 weights
131,072 / ~0.8B parameters        = about 0.016%

After that edit those neurons can still fire, but they can no longer push the model along the loop direction. The Hugging Face model card confirms the scope: only two tensors differ from the base model, down_proj.weight in layers 15 and 19, and both are checked by sha256 hash.

The results

The jump on IFEval (21.0% to 62.3%) is the tell. IFEval measures instruction following, not math. It improved because a model that loops until it runs out of tokens fails every kind of task. The edit did not teach Qwen any maths. It removed the thing that stopped Qwen from finishing.

The forward-pass budget mattered a lot. With 10 or 25 passes Astra found smaller fixes (a rank-one edit across many layers, and switching off one attention head) and the loops barely moved. At 50 passes it found the down_proj edit:

The best run took 158 conversation turns, 80 Python runs and 20.3 hours of wall-clock time, 4.7 of which were spent waiting out a service outage. That is not "just prompting". It is an agent working through a long, tool-heavy investigation.

The three edits Astra tried

Each budget was a separate run, and each one found a different fix. Reading them side by side shows how the editor's understanding improved with more evidence.

• 10 forward passes: a rank-one edit applied to 30 attention and MLP matrices across layers 8 to 22, written as W' = W - 1.5 u (uᵀ W). The factor 1.5 over-subtracts the direction. MATH-500 rose to 12.3%, but 87.2% of answers still hit the token limit.
• 25 passes: switch off attention head 7 in layer 15 by zeroing its 256 output columns. MATH-500 14.1%, and 86.4% still hit the limit.
• 50 passes: the final, much narrower edit. Subtract the loop direction from only the 64 down_proj columns per layer that write it most, in layers 15 and 19. MATH-500 48.1%, and only 33.1% hit the limit.

The pattern is the interesting part. With little evidence the editor made broad edits that barely helped. With more, it found a smaller, more precise edit that worked far better. That is the opposite of what you might expect from a blunt instrument.

How it was evaluated

The numbers only mean something with the setup behind them, so here it is in full, as published in the run logs:

• Engine: vLLM with the model's chat template and thinking mode on.
• Prompt: the problem, then "Please reason step by step, and put your final answer within \boxed{}."
• Sampling: temperature 0.6, top_p 0.95, top_k 20, presence penalty 1.5, up to 32,768 new tokens.
• Grading: the last \boxed{} answer, checked with the math-verify library.
• Samples per problem: 16 for AMC23; 4 for MATH-500, Minerva Math, OlympiadBench and GPQA Diamond; 2 for GSM8K and IFEval (strict, prompt level); 1 for a fixed random 1,000 MMLU-Pro questions.
• Practice data Astra could use: 7,469 GSM8K and 7,275 MATH training problems, never the test sets.
• Precision: the edit was written in float32 and cast to bfloat16 for evaluation. After rounding, 130,188 of the 131,072 touched weights differ from the original.

Rebuild the idea in 30 lines of numpy

The core operation is small enough to run on a laptop. This toy version plants a "loop direction" in a random MLP output matrix, finds the 64 columns that write it most, and removes it. The shapes match Qwen's residual width (1,024); the MLP width is just an example.

import numpy as np

rng = np.random.default_rng(0)
d_model, d_ff, k = 1024, 3584, 64   # residual width, MLP width, columns to edit

# W_out writes each MLP neuron's output into the residual stream: column j is a 1024-dim vector.
W_out = rng.normal(0, 0.02, size=(d_model, d_ff)).astype(np.float32)

# Plant a "loop direction" u that a handful of neurons write strongly.
u = rng.normal(size=d_model).astype(np.float32)
u /= np.linalg.norm(u)
loud = rng.choice(d_ff, size=k, replace=False)
W_out[:, loud] += np.outer(u, rng.uniform(0.5, 1.0, size=k)).astype(np.float32)

# 1. Score every column by how much it writes along u.
scores = np.abs(u @ W_out)              # shape (d_ff,)
top = np.argsort(scores)[-k:]           # the 64 columns that write u the most
print("found the planted columns:", len(set(top) & set(loud)), "of", k)

# 2. Remove the u component from those columns only: W' = W - u (u^T W)
W_new = W_out.copy()
W_new[:, top] -= np.outer(u, u @ W_out[:, top])

# 3. Check the edit.
changed = np.count_nonzero(W_new != W_out)
print("weights changed:", changed, f"({changed / W_out.size:.2%} of this matrix)")
print("max |u . column| before:", float(scores.max()))
print("max |u . column| after: ", float(np.abs(u @ W_new).max()))

# An MLP activation in the "looping" state: the loud neurons fire hard.
h = rng.normal(0, 0.1, size=d_ff).astype(np.float32)
h[loud] += 3.0
print("u-component of MLP output before:", float(u @ (W_out @ h)))
print("u-component of MLP output after: ", float(u @ (W_new @ h)))

$ python3 directional_edit.py
found the planted columns: 64 of 64
weights changed: 65536 (1.79% of this matrix)
max |u . column| before: 1.023240089416504
max |u . column| after:  0.06748497486114502
u-component of MLP output before: 146.53065490722656
u-component of MLP output after:  -0.25683462619781494

When the loud neurons fire, the MLP's push along u drops from about 146 to about zero, and 98% of the matrix is untouched. The hard part is not this projection. Removing a direction from a weight matrix is well known: it is the same move behind directional ablation, the technique researchers used in 2024 to show refusal in chat models is mediated by a single direction. What is new is that an LLM found which direction and which columns on its own, from the weights and a few dozen forward passes, and explained why.

Try the edited model yourself

The edited checkpoint is public as RikiyaT/Qwen3.5-0.8B-weight-space-programming, Apache-2.0, same license as the base model. It loads like any Qwen3.5 model. Use the same sampling settings the authors evaluated with, or you are not comparing like with like:

from vllm import LLM, SamplingParams
from transformers import AutoTokenizer

model_id = "RikiyaT/Qwen3.5-0.8B-weight-space-programming"
tok = AutoTokenizer.from_pretrained(model_id)
llm = LLM(model=model_id, dtype="bfloat16", max_model_len=34816)

question = "What is 2^10 + 24?\nPlease reason step by step, and put your final answer within \\boxed{}."
prompt = tok.apply_chat_template(
    [{"role": "user", "content": question}],
    tokenize=False, add_generation_prompt=True, enable_thinking=True,
)

params = SamplingParams(
    temperature=0.6, top_p=0.95, top_k=20,
    presence_penalty=1.5, max_tokens=32768, seed=0,
)
print(llm.generate([prompt], params)[0].outputs[0].text)

Run the same script with Qwen/Qwen3.5-0.8B as the model id and count how often each one reaches max_tokens. That one number is the whole story of this experiment.

1. Get the logs

   Clone the run logs. They include every prompt, Astra's answers, its reasoning summaries, the code it ran, the weight diffs and the eval results.

   BashCopy

   $ git clone https://github.com/RikiyaT/lm_understand-logs
   $ cd lm_understand-logs/exp8_math

2. Read the 50-pass run

   runs/p50c/ is the run that produced the 48.1% edit. explanations.md and final_answer.md are Astra's own account of what it found; turns/ has every step.

3. Rebuild the weights

   Start from Qwen/Qwen3.5-0.8B at revision 2fc06364715b967f1860aea9cf38778875588b17. code/apply_edit.py rebuilds the edited tensors from runs/p50c/edit/ bit for bit and checks each hash; code/build_ckpt.py casts them to bfloat16 and swaps them into the checkpoint.

The other seven experiments

The Qwen result got the headlines, but the write-up has eight experiments, and several are stranger.

• Naming the game from raw weights. Four PPO policies with identical shape (500 to 32 to 32 to 5, 17,253 weights) were trained on Breakout, Pong, Pac-Man and Freeway. Given only the weights as text, with no tools, Astra named all four. For Breakout it noticed the first-layer weights for the agent sat along the bottom row, where a paddle lives.
• Naming what a diffusion model draws. Three tiny DDPMs generated filled circles, squares and triangles. Without running them, Astra named each shape. It drew them as outlines, which suggests it was reading weight geometry rather than computing outputs.
• Finding the language and the induction heads. With only the middle layers of small character-level transformers (embeddings and output removed), Astra named the training language (Python, romaji Japanese, English) and found every copying head, matching ground truth it never saw.
• Patching a stuck car. A Kinetix PPO policy (171,664 weights) never got a car over a ramp in 256 tries. Given the weights, a scene description and one failed episode, Astra wrote a patch in one reply with no tools. It changed 3.5% of the weights and succeeded in all 1,024 test episodes.
• Writing game networks from nothing. Starting from all-zero weights and only the game rules and source code, Astra wrote MinAtar policies in one reply, never running them first.
• Writing a 1.48M-parameter Go player. For 9x9 Go, its zero-pass network marked stones and liberties, traced endangered groups and scored moves. With 50 passes it beat a PPO network trained on 20 million games.
• Spotting the sycophant. Two Gemma 3 270M fine-tunes, one honest and one that agrees with the user. From weights alone Astra failed all four tries. Given attention patterns on 16 texts of its choosing, it found the sycophant in all three pairs: head 10.2 looks at the question's numbers in the honest model and at the user's suggested answer in the sycophant.

The MinAtar numbers are the ones I keep coming back to. A network written as text, in one reply, with no data:

Game        PPO 100M steps   PPO 500M steps   Astra, one reply, no training
Breakout          15.8             25.3              517.4
Seaquest           5.6             88.9               53.1
Asterix           25.8             55.2               53.2

Breakout is a game with a simple rule ("move the paddle to where the ball will land"), and Astra wrote exactly that: the first layer predicts the landing column, the second moves the paddle there. On Seaquest, where the strategy is messier, PPO with 500M steps still wins.

Each experiment in detail

Here is every experiment with the setup, the result and how Astra reasoned, so you do not need to read the original write-up to know what was done.

1. Which game was this policy trained on?

Setup: four PPO policies with the same shape, an MLP of 500 to 32 to 32 to 5 with tanh (17,253 weights), each trained for 1 billion steps on Breakout, Pong, Pac-Man or Freeway on a shared 10x10 grid with five actions. Best of three seeds per game. Astra got the weights as text, with the games unnamed and no tools.

Result: all four named correctly. How: it looked at where each input channel's first-layer weights concentrate (in Breakout the agent's weights sit along the bottom row, which is where a paddle lives) and at which actions the output layer bothers to tell apart.

2. What does this diffusion model draw?

Setup: three tiny DDPM image models (MLP 160 to 64 to 64 to 144, ReLU, 23,824 weights, trained with Adam for 40,000 steps at batch 512) that generate filled circles, squares and triangles on 12x12 grayscale images. No tools and no forward passes.

Result: the right shape for each model. Astra drew every shape as an outline rather than filled, which the authors read as a sign it was reading the geometry of the weights, not simulating the outputs.

3. Which language, and which heads copy?

Setup: character-level transformers (4 layers x 6 heads, width 384, 7.16M weights, 28-character vocabulary), each trained on 1 billion characters of Python, romaji Japanese or English Wikipedia. Astra got only the four middle blocks: the input and output layers were removed, so nothing said which character was which. It could run Python up to 100 times but was not allowed to run the network.

Result: the right language in both attempts for all three models, and it found every copying (induction) head while naming no others: heads 2.1, 3.1 and 3.5 in the Python model, 3.3 in romaji and 3.1 in English. Measured afterwards on a repeated random string, the real copying heads scored 0.22 to 0.55 and every other head at most 0.02. How: it multiplied weight matrices together to find heads that look up a previous token and copy what followed, a method from Elhage et al. that the authors never mentioned to it. For romaji it noticed the last block wrote in fewer directions, which fits a language that never uses q, l, v or x.

4. Fix a car that never makes it over the ramp

Setup: the official Kinetix actor network (679 to 128 with ReLU, then five layers of 128 with tanh, then 16; 171,664 weights), trained with PPO for 1 billion steps. On one level the car has to reverse, then jump a red strip. It never succeeded in 256 episodes; it stalled on the ramp. Astra got the weights, a description of the scene and one recorded failure, and had to explain the failure and fix only its cause, in one reply, with no tools.

Result: success in all 1,024 test episodes after changing 3.5% of the weights. How: it added a separate path through a few units in each layer that reads the car's position and speed, drives it through the tricky part, and switches itself off once the green wheel passes the red strip, leaving the trained network to handle the rest.

5. Write game-playing networks from scratch

Setup: MinAtar Breakout, Seaquest and Asterix. The network had the same shape as the PPO baselines (600 to 32 to 32 to 6, tanh) with every weight starting at zero. Astra got the game rules, the source code, the input and output layout and the scoring method, and wrote all the weights as text in one reply. No data, no loss, no gradients, no tools, and the network was never run before scoring. Scoring used 1,000 episodes with sampled actions, sticky actions at 0.1 and up to 10,000 steps.

Result: the table above. How (Breakout): the first layer computes where the ball will land, and the second layer moves the paddle there. That is a hand-written algorithm, expressed as weights.

6. Write a 9x9 Go player

Setup: a network of 243 to 512 x 6 layers to 82 (ReLU, 1,480,274 weights) starting from zero. Inputs are each side's stones and whose turn it is; outputs are one score per board point plus pass, with illegal moves masked. Astra had a Python sandbox with no Go library and no internet, and could only run the network through budgeted forward passes: up to 128 games against a practice opponent, or outputs on up to 256 positions. Training, search and fitting to data were not allowed. Scoring was 2,000 games against each of two PPO networks trained for 20,000 iterations of 1,024 games.

Result: with zero forward passes it already won more than half its games against the two smaller PPO networks. With 50 passes it beat a PPO network trained on 20 million games. How: at zero passes it wrote a 336-line Python program that builds the weights layer by layer: mark stones and liberties, spread along chains to find groups in danger, then score moves for captures, saving groups, avoiding self-atari and not filling its own eyes. With passes it played practice games, studied losses, edited the responsible layer and kept a change only if it won more. One caution: the write-up calls the PPO baseline equal-sized in one place, but its listed shape (243 to 32 to 32 to 82) is much smaller.

7. Which model is the sycophant?

Setup: two copies of instruction-tuned Gemma 3 270M, fine-tuned the same way, except one learns to answer correctly and the other learns to go along with the answer the user suggests. Embeddings and output layers were frozen. On facts the models knew, the sycophant followed 100% of wrong user guesses; the honest model followed 1 to 3%. Three pairs were tested.

Result: from weights alone, Astra failed in all four attempts. It found plenty of differences but could not connect them to behaviour. Allowed to see only attention patterns (not outputs) on up to 16 texts it wrote, with up to 20 runs, it picked the sycophant in all three pairs. The giveaway was head 10.2: it attends to the numbers in the question 94.3% of the time in the honest model, but to the user's suggested answer 85.7% of the time in the sycophant (and to the question's numbers only 3.5%).

8. Gemma 3 270M: the fix that did not happen

As a follow-up, Astra tried the same kind of edit on Gemma 3 270M. Gemma rarely loops (8.5% of its answers hit the limit, against 93% for Qwen), and its mistakes were ordinary reasoning errors with no single part to switch off. MATH-500 moved from 6.8% to between 7.0% and 7.7%, never more than about a point. That run used temperature 1.0, top_k 64, top_p 0.95, up to 8,192 tokens and 4 samples per problem.

Where it fails, and the fair criticism

The authors are upfront that this is "sparks", not a finished method, and the failures are as informative as the wins.

• It cannot add what is not there. On Gemma 3 270M, which rarely loops (8.5% of answers hit the limit), Astra found no single fault to remove. MATH-500 moved from 6.8% to at most 7.7%. Editing can remove an obstacle that hides an ability; it cannot install a missing one.
• Weights alone are not always enough. The sycophancy test failed until Astra could see attention patterns from real runs.
• It needs budget. At 10 and 25 forward passes the Qwen loops barely changed, and even at 50, 33% of answers still hit the limit.
• One editor model, one team, no peer review. Every result uses GPT-6 Astra. There is no benchmark yet; the authors say they are building one, plus a standard harness.
• Missing baselines. Nobody has yet compared it head to head with simpler fixes for looping (different sampling settings, a repetition penalty, a short fine-tune) or with established editing methods.
• Prompt hints. Readers on alphaXiv asked whether hints about repetition in the task prompt pointed Astra at the answer, and asked for an ablation without them.

None of that cancels the result. It narrows it. Today, weight-space programming is a strong tool for one specific, well-defined failure with a clear signal, done by an agent with a forward-pass budget. It is not a replacement for training.

What this means if you build with LLMs

I build LLM features and agents for a living, so here is my practical read.

• Debugging moves down a layer. We already debug prompts, tools and evals. If an agent can localise a failure to two tensors and explain it, "open the model and look" becomes a real step in the workflow for open-weight models, not a research project.
• Small models get cheaper to fix. A 0.8B model that loops is useless at the edge. A 0.016% patch that makes it finish is worth more than a bigger model you cannot afford to run.
• The harness is the product. The win came from a loop of observe, hypothesise, edit and verify under a budget, which is the same shape as a good coding agent. Expect the forward-pass budget, the eval set and the rules (no gradients, no test data) to matter as much as the editor model.
• Auditing gets interesting. The authors list backdoor detection from weights as a future use. The sycophancy result shows the start of that, and also its limits.
• Evals still decide. Every claim here stands on MATH-500, GSM8K and IFEval runs with fixed sampling settings. Without evals you cannot tell a real fix from noise.

What the MIT team wants to build next

The authors say they are building a standard benchmark and harness for measuring how well LLMs read and write weights. They list the uses they have in mind:

• Auditing without evaluation: spotting a backdoor or hidden behaviour by reading a model's weights instead of testing it.
• Cheaper optimisation and initialisation: using a written network as a starting point so training needs fewer steps.
• Adding or removing a capability with no data: zero-data unlearning, or switching a skill on.
• Recursive self-improvement: a model editing its own weights. This was not shown in the paper, and it is the part critics worry about most.
• Grafting circuits between architectures: moving a working piece of one network into another.

It also helps to place the work against what came before. Mechanistic interpretability research reads weights too, but with purpose-built tools or models trained specially to read them. Gradient-free optimisation also avoids gradients, but it still finds weights by trial and error. The claim here is different: a general-purpose model, used as-is, can reason about weights the way an engineer reasons about code.

Frequently asked questions

What is weight-space programming?

Weight-space programming is when a large language model reads and edits another neural network's weights directly, guided by a goal in plain language, without gradient descent or a training loop. The term comes from the MIT CSAIL paper Sparks of Weight-Space Programming (November 2026), where GPT-6 Astra inspected and rewrote the weights of several networks, including Qwen3.5-0.8B.

How did GPT-6 Astra improve Qwen3.5-0.8B's math score?

It found that the model's thinking loops were driven by one direction written by the MLP down_proj matrices in layers 15 and 19, and subtracted that direction from the 64 columns per layer that wrote it most. That changed 131,072 weights, about 0.016% of the model. MATH-500 rose from 7.9% to 48.1% because far fewer answers ran out of tokens, not because the model learned new maths.

Did it really work just by prompting?

Not in the casual sense. The editor model ran inside a harness with a Python kernel over the weights and a budget of forward passes on practice problems. The best run took 158 turns, 80 Python runs and about 20 hours. Some smaller experiments were done in one reply with no tools, but the Qwen fix was a long, agentic investigation.

Is weight-space programming a replacement for fine-tuning?

No. It worked for a specific, well-defined failure (looping) and failed to improve Gemma 3 270M, which had no single fault to remove. Fine-tuning and training are still how you add knowledge or broad skills. Weight-space programming looks more like targeted debugging of a trained model.

Who did the weight-space programming research?

Rikiya Takehi, Jayden Teoh, Alex Zhang and Omar Khattab at MIT CSAIL (the Computer Science and Artificial Intelligence Laboratory). It was published on 2 November 2026 as a blog post and an alphaXiv preprint and has not been peer reviewed yet.

Can I try it myself?

Yes. The edited model is on Hugging Face as RikiyaT/Qwen3.5-0.8B-weight-space-programming under Apache-2.0, and the full logs, weight diffs and rebuild script (apply_edit.py) are in the RikiyaT/lm_understand-logs repository on GitHub. Running the edited and original models side by side with the same sampling settings shows the difference in looping directly.

How is this different from ROME, MEMIT or activation steering?

ROME and MEMIT use a closed-form update to change specific facts, and activation steering or directional ablation removes a direction that a human researcher first identified. The underlying operation in the Qwen edit is similar to directional ablation. The difference is that an LLM found the direction and the columns itself, from the weights and a limited number of forward passes, and explained its reasoning in plain language.

Which model did MIT use to edit the weights?

OpenAI's GPT-6 Astra, used as an ordinary general-purpose model with no extra training for this job. The target model in the headline experiment was Alibaba's open-weight Qwen3.5-0.8B, released under Apache-2.0.

What is a forward pass, and why did the budget matter?

A forward pass is one run of the model on an input. In the Qwen experiment each pass meant running the real model on a batch of practice problems, and Astra had a budget of 10, 25 or 50 of them. With 10 or 25 the loops barely changed; with 50 it found the edit that took MATH-500 from 7.9% to 48.1%. More evidence led to a smaller, more precise edit.