Model scores can shift when training is repeated or prompts are changed. That makes it hard to tell whether a relationship between scores, or a model’s apparent lead, reflects a real difference or setup noise.
The work treats reruns and prompt formats as repeated measurements. It corrects a correlation, a measure of how two scores vary together, only when both scores show clear repeatability. It compares models separately within each prompt setup instead of averaging unlike setups.
The usual correction produced impossible correlations above one in 70 of 100 image categories. Prompt changes also reversed model rankings. The practical lesson is to avoid adjusted claims or naming a single winner when measurements are too noisy or results disagree across settings.