This paper compares neuroevolution (NE) methods—evolution strategies (ES) and genetic algorithms (GA)—against (RL) methods in scenarios where tasks change. Across environments with policies from hundreds to millions of parameters, ES consistently achieves the best balance between retaining past skills (stability) and learning new ones (plasticity). The GA is the most plastic method but forgets more. The study finds that ES finds wider neighborhoods in weight space where perturbed policies still solve the task, and the overlap between neighborhoods of consecutive tasks predicts the trade-off. Notably, symptoms of plasticity loss common in RL (like dormant units) do not appear in NE, suggesting that training with weight perturbations may be a robust mechanism for continual learning.
Evolution strategies balance adaptation and retention across changing tasks
Across several continual-control benchmarks, neuroevolution retained earlier capabilities while adapting more consistently than gradient-based reinforcement learning variants.
AI Startup
Eleni Nisioti · Andrea Cossu · Kathrin Korte · Sebastian Risi
IT University of Copenhagen · University of Pisa · Sakana AI
Research Digest··2 min read
In one line
Neuroevolution, especially evolution strategies, outperforms reinforcement learning methods in continual task settings by achieving a better stability-plasticity trade-off.
Before you read this
Two ideas this paper builds oncontinual learningA learning paradigm where a model sequentially acquires knowledge from multiple tasks without significantly forgetting earlier ones.72 papers →reinforcement learningA machine learning method where an agent learns to make sequences of decisions by receiving rewards or penalties for its actions.296 papers →
Read it as
Nisioti and colleagues test whether neuroevolution, which optimizes populations of neural networks through weight perturbation and selection, can address the tension between learning new tasks and retaining old ones.
The authors compared evolution strategies (ES) and genetic algorithms (GAs) with PPO, continual-RL variants including TRAC, ReDo and C-CHAIN, and population-based PPO.
How it works
stage by stageThe Daily Edition
The front page in your inbox each weekday morning: the lead, the picks, and why each one earned its place.