The authors construct ComFair by taking existing single-attribute fairness items and minimally introducing additional attribute values (e.g., adding age to a gender-based instance) while preserving the task and content, creating a task-equivalent counterpart.
Intersectional fairness benchmark exposes instability of single-attribute audits in LLMs
ComFair pairs 479,066 single- and multi-attribute instances to reveal hidden bias shifts and ranking changes across four tasks.
Top University
Zheng Zhang · Yifan Wu · Yuxiang Zhou · Qi Liu · Qingyang Mao · Ning Li · +4 more
University of Science and Technology of China · Nanyang Technological University
Research Digest··3 min read
The authors present ComFair, a configurable benchmark that systematically augments single-attribute fairness data with additional sensitive attributes to create 479,066 paired Dual- and Multi-attribute instances.
The Daily Edition
The front page in your inbox each weekday morning: the lead, the picks, and why each one earned its place.