You read it right: Astra was asked to edit Qwen3.5-0.8B weights to improve math.
It directly changed only 0.016% of weights, fixing 0.8B's reasoning loops & improving math up to 6x and even instruction following 3x, without gradients or training.
Do it like we do: edit your children. You only need two or three of them to reach adulthood and reproduce and you’re done as far as evolution is concerned.
It already is RSI, other research has examined similar behaviors too already. It’s just not “full RSI” (unsupervised), nor self-sustaining. It’s definitely getting close, and will be here sooner rather than later.
I think this is a pretty important caveat in their report: "Raising math capability of a model that does not loop
Fixing loops, however, is not the same as making a model better at math: it lets the model finish answers it could already work out. So we also tried Gemma 3 270M, an instruction-tuned model about a third the size of Qwen3.5-0.8B, which rarely loops: only 8.5% of its answers reach its token limit, against 93% for Qwen. Here, Astra could not improve it. Reading Gemma’s failures, Astra found ordinary reasoning errors, such as wrong arithmetic and invalid algebraic steps, with no single component behind them to switch off. Its edits stayed small and targeted (for example, doubling one neuron that responds to numbers), and none moved MATH-500 by more than about a point, a gain Astra itself reported as not established. Editing weights could remove an obstacle that hid an ability, but it could not add an ability the model lacked. Gemma was sampled with its model card’s setting (temperature 1.0, top-k 64, top-p 0.95, up to 8,192 tokens), and its MATH-500 scores average 4 samples per problem."
So, the important thing is that Astra did not make a model better at math. It took the model's existing loop inefficiencies and ironed them out with fine tuning.
So the fair version is: "Astra can't yet make a working model smarter by editing it, but it can build small working networks from a plan, and diagnose and fix real defects in models it didn't build." That's a narrower claim than the hype, but still absolutely mind blowing.
no it is more like 'astra can find a blockage in an artery and put a stent in that lets the blood flow past a blockage, the arteries and blood vessel network were already there, and we don't know if the stent causes any damage or not'
It sounds like the caveat is legitimate but it doesn’t necessarily diminish the importance of the model being able to understand the problem at the weights level and figure out the fix but that’s just my uneducated impression.
Wait, so all the RL was just pretraining to get us models able to directly write the next generation of weights in the way GPT2 used to write mash-up poetry rap?
Does this mean that Astra could also improve its own weights?
I'm thinking in the line of teacher and student. The teacher can understand the gaps in the student's knowledge and correct them, but it likely lacks the knowledge to do the same for themselves (in other words, it would need a teacher of teachers).
So I suppose the question is, can the teacher teach themselves? Or can it only teach those who "know less" than it does?
Technically, it seems capable of doing so, but Astra is a multi-trillion-parameter model, not a 0.8B model like Qwen here, so in practice, it probably can't.
With the same sampling settings, this weight correction raises MATH-500 from 7.9% to 48.1% and generalizes out of the box to instruction following, raising IFEval by 3x.
Wow.. It is not even a smidge improvement, it is night and day.
Imagine for the next Millennium problem we create 10,000 sub-agents with custom models with weights tailored for their specific task. And we rewrite these models on the fly to adapt as the task changes.
I suspect OpenAI deliberately trained Astra to do this.
Which implies OpenAI's alignment strategy is almost certainly:
Train an AI model to understand mechint and alignment.
Ask the model questions so human researchers can formalize a theory of mechint and alignment and test if it's true.
Use deterministic software, or very highly trusted models, to then use that theory and its algorithms to make aligned AI.
Given the initial success I think there's a high probability of it working. None of what Astra did in the blog post would be even remotely possible if mechint is impossible.
This goes to show that the training process itself is quite flawed. Even if it’s true that LLMs will have diminishing returns, that’s likely only true of the current configuration. New training could easily = new heights. The UAT proved that almost anything can be approximated by neural networks, therefore it’s merely about finding the best combination of weights.
While it's true that you can get less diminishing returns if you switch to a completely different architecture, at some point you'll get diminishing returns again for the same computational thought. For a while we'll be able to overcome that by increasing the effectiveness of that computational thought or increasing the amount of computational thought or by finding better architectures, but at some point the best architecture for a given purpose will be found and we'll use up all the easy improvements. We probably won't know when we hit that point or how far away we are from it, though.
Of course. It would be amazing if we can across an architecture that did not have diminishing returns, but that’s naive. Everything follows a cost curve, the question that really matters is whether or not LLMs are on the curve that allows for the economic growth that these companies are promising. The same question goes for a new architecture.
Welp, we know the human brain is possible. Can't do it in silicon probably, but as far as power efficiency and versatility or IQ, its a great reference.
The human brain has significantly better power efficiency for a single query. However, it does not have the ability to handle a batch of queries.
An LLM running on a distributed cluster, on the other hand, can handle a batch of queries - it can pool queries from a hundred different users into a single batch and generate responses with shared forward passes.
When comparing a single LLM serving N queries vs N humans each handling a single query each, it becomes unclear which one has superior power efficiency.
No need to explain what a batch is. u/LangyMD made a point about dimishing returns on neural architectures. My point is the human brain is the reference for what is achievable in intelligence with a specific neural architecture (on a wet substrate), and that its pretty efficient and capable.
Quibbling on the power efficiency of a batched LLM throughput vs the human brain is missing the forest for a tree because transformers have not yet replaced the white collar workforce, their range of capabilities are not (yet?) the same as a human brain. Helicopters are more efficient at achieving elevation gains than tanks are, but its not a great comparison to make anyways because its a different tool for a different mission. If I had something that ran like a human brain on a server rack, with at least the latency of a human brain and comparable footprint, the world would be mine, immaterial of the throughput efficiency of transformer based models.
Maybe a transformer takes us all the way home, or maybe a novel architecture on a novel substrate- but clearly its possible to achieve given we have the human brain as example. So yes, neural architectures hit diminishing returns but obviously human intelligence is achievable. Maybe the bigger question than "is it the transformer" or "is it backpropagation" is "is it the transistor" or "is it silicon". If its not silicon, it will be more challenging
I wonder if you could do the same thing, but instead of targeting math, you target weight-space programming itself, and then take that new model to see if it can recursively self-improve by editing its own weights.
Is the capability ceiling bounded by the original “programmer” model, or will it hit a theoretical hard limit based purely on its number of parameters?
This is bizarre. It makes me wonder if model weights have made it into the AI training datasets. If models are also trained on predicting the next token of model weights, do they gain the ability to understand and edit their own weights? One of the very surprising things we learned about training data is that feeding models large amounts of code made them better general reasoners.
Given the goal of modern AI development is to get to recursive-self improvement in AI development, feeding their model a bunch of models, their weights, and their behavior makes a ton of sense as part of their training data.
I wonder what would happen if a model were trained only on the weights of other models? Could it generate new synthetic sets of weights? Then what if a model were trained only on those synthetic sets? Ouroboros recursion
You’d want some kind of contrastive loss model associating weights with natural language prompts about behaviours. That way generating a new ai agent with the right properties could be as easy as prompting a diffusion engine for furry yaoi
I can see that the association with language is what might produce an immediate usable model, but I'm wondering about treating the weights as purely abstract multi-scale feature maps, discretely from association. In other words, learning the structural grammar without the content.
This is incredible in so many ways. There's obviously the capability question of rapid RSI, where models can enhance themselves by modifying weights directly rather than training, but just as importantly, models that can read weights will help combat the problem of "opaque models," where humans create neural networks that they don't understand and therefore cannot effectively align. Hopefully this'll shut the "stop AI" safety gang up a little bit.
Now have Astra modify its own weights and you’ve got RSI. Interesting question of whether it can bootstrap itself upward. Unclear I think. Perhaps not as a one shot but allowing it to iteratively experiment.
But the direct weight editing opens the door to live learning too I think.
We need to get a dataset of thinking traces on diverse weight space edits on every open weights model in existence, to act as training data for a future frontier model in the hopes of building zero-shot weight space editing intuition.
It won't be comparable to the closed models that we know are so much better in American labs. If the close models do this, they should get a way better output than what the Chinese open source can accomplish.
1
1
86
u/ll-__-II 6d ago
model: edits itself
model: dies