A common way to train language models uses fine-grained criteria called rubrics. During training, a large fraction of these rubrics become inactive because the model always fails or always passes them, giving no signal to improve.
The authors identify that such inactive rubrics lack either the required behavior (unreachable) or the ability to distinguish quality among responses (saturated). They propose RuACT, which adaptively turns unreachable rubrics into teachers and decomposes saturated rubrics into finer critics.
This recovers useful supervision. The trained policy satisfies far more previously unreachable criteria and its reward scores better align with human judgments, leading to higher performance on medical and scientific benchmarks.