LLMs can read and write neural network weights, researchers demonstrate
Weight space programming offers a new route to automated AI improvement, but remains limited to simple tasks for now
Researchers have shown that large language models (LLMs) can directly read and edit the weights of another model, a capability the article describes as a new recursive self-improvement capability. In a new paper co-authored by Alex Zhang, a sufficiently smart model was able to identify what a network does from its weights, fix bugs in one step given failed traces, and write complete network weights from scratch that beat some independently trained models.