overfitting
Learning too well on the data or tasks you trained for is overfitting: performance looks great there but fails to transfer to new settings. Imagine practicing only one route until you could drive it blindfolded, but still cannot navigate anywhere else. In machine learning, the model picks up quirks or noise in the training set as if they were general rules.
The papers show overfitting wearing different masks. One finds evolved harnesses gain much by exploiting benchmark quirks, not general skills. Another shows repeated data hurts mixture-of-experts models earlier because extra parameters memorize, not generalize. A third shows agents editing their own skills accumulate overfit instructions, and regularization fixes it. All distinguish true generalization from artifacts of the training environment.
- Papers
- 3
- Released code
- 0
- First seen
- Sept 2026
- Latest
- Oct 2026
3 papers in the last two months, against 0 in the two before.
Start here
The paper to read first- 1Most harness-evolution gains come from overfitting or extra compute
Overfitting and test-time scaling explain most harness evolution gains, leaving little evidence of generalizable improvement.
Oct 2026
How fast it is moving
Often studied with
agent skillsLearned capabilities that enable AI agents to perform specialized tasks with improved effectiveness.1 togethercontinual learningA learning paradigm where a model sequentially acquires knowledge from multiple tasks without significantly forgetting earlier ones.1 togetherdata repetitionReusing the same training samples across epochs, affecting overfitting, model degradation, and scaling-law predictions in pretraining.1 togetherThe papers
Most central to this idea first, not most recent.
- Big Techcs.AI
Most harness-evolution gains come from overfitting or extra compute
Overfitting and test-time scaling explain most harness evolution gains, leaving little evidence of generalizable improvement.
Amazon, New York University · Oct 2026
- Chinese Techcs.AI
Regularization limits agent skill growth without sacrificing downstream performance
SkillEvoReg uses dropout, complexity regularization, and counterexample validation to prevent overfitting in agent skill evolution.
Huawei Noah’s Ark Lab · Sept 2026
- Top Universitycs.LG
Sparse mixture-of-experts models overfit repeated training data faster
MoE models degrade more rapidly than dense models under data repetition, with overfitting increasing with sparsity.
Stanford University, Paul G. Allen School of Computer Science, University of Washington · Sept 2026
Concepts are extracted from each paper and reused across the corpus, so this page grows on its own as the desk reads.