The authors define accessibility as the probability that a model answers correctly across a set of meaning-preserving reformulations of the same underlying inquiry.
Accessibility profiles reveal hidden tradeoffs in LLM reliability
Kaplan et al. show that models with similar average accuracy can differ sharply in how widely and dependably they succeed across equivalent wordings.
Top University
Kazem Faghih · Yize Cheng · Shoumik Saha · Mobina Pournemat · Armin Gerami · Soheil Feizi
University of Maryland, College Park
Research Digest··2 min read
The authors introduce accessibility profiles that evaluate language models across multiple meaning-preserving formulations of each inquiry.
§