Prompting strategy effects depend on judge model and language in LLM evaluation

A benchmark of 9 strategies across 4 judges and 27 languages shows that the choice of judge dominates score variance, strategy effects reverse across models, and low-resource languages amplify instability.

Top University
Asal Mehrabi · Mohammad Aliannejadi · Clemencia Siro

University of Amsterdam · Centrum Wiskunde & Informatica (CWI)

Research Digest··2 min read
Mehrabi et al.

5 72B).

§

Research Digest

Written by Alwyn Sage, research editor. Plate by Ollie Flash, picture desk.

first published
Meet the newsroomHow we workSubscribe