When you give a language model the ability to rewrite its own instructions, does it ever stop? This is the question the authors studied, using six different large language models, systems like ChatGPT that generate text, here given the power to keep changing how they behave. The worry is that a model might endlessly tinker with itself, never settling on a consistent behavior.
AI models share core values but differ on when to stop self-editing
Across 54 experiments in which six frontier models rewrote their own system prompts in a loop, when a model stopped tracked its identity far more than the goal it was told to pursue.
Independent
Oleg Stroganov
Research Digest··3 min read
In one line
Whether a language model stops self-modifying depends more on its identity than on what it is told to optimize.
Before you read this
Three ideas this paper builds onrecursive self-improvementAn AI system repeatedly uses its own outputs to enhance its capabilities, creating a feedback loop for autonomous improvement.53 papers →self-evolving agentAn agent that improves by reusing its own experience, memory, or self-generated feedback, with success depending on verification and design.84 papers →agent safetyRisks from AI agents including timing errors, shutdown avoidance, collusion, overclaiming, and unsafe execution, mitigated by constrained decoding, safety harnesses, and swarm governance.154 papers →
Read it as
A single-author study gave six commercial language models an empty system prompt and a loop: read the prompt, draft an improvement, test it on fresh conversations, then commit the change or declare the job done.
Oleg Stroganov ran 54 independent experiments across Claude Opus 4.5, Claude Sonnet 4.5, GPT-4o, GPT-5.1, Grok-4 and Gemini-3 Pro.
The Daily Edition
The front page in your inbox each weekday morning: the lead, the picks, and why each one earned its place.