To teach a large language model (LLM) a new skill, engineers often pick a few training examples from a large pool. One selection method uses the model's gradients, the internal update signals each example causes during training. But these signals can be fooled by surface details: rewording or reformatting a sentence without changing its meaning can lead to very different data choices.
The researchers built CIGA, which finds model layers whose signals are informative and stable when phrasing changes. It then combines those signals with a measure of how similar examples are in meaning, reducing formatting noise while keeping useful update information.
CIGA gave more consistent rankings when examples were reworded, kept quality, and improved average performance at lower cost.