The authors evaluated role-playing prompts across multiple language models and model scales using MMLU and MMLU-Redux, two broad collections of multiple-choice questions spanning academic and professional subjects.
Role prompts help language models only under the right conditions
Across multilingual knowledge benchmarks, persona prompting varied with model capacity, subject and language, while combining equivalent prompts across languages produced more consistent results.
Chinese Tech
Xingjie Zhuang · Jialong Tang · Chulun Zhou · Buchao Zhan · Zhirui Li · Junhui Li · +2 more
Xiamen University · Tongyi Lab · The Chinese University of Hong Kong · Soochow University
Research Digest··3 min read
Zhuang et al.
Why this paper
From Tongyi Lab and 3 others
In one line
Role-playing prompts do not universally improve large language model performance; gains depend on model capability, knowledge domain, and prompt language.
What we could check
- ·No code link found
- ·No weights link found
- ·No dataset link found
- ·No compute details found
- ·No stated limitations found
- ·No benchmark numbers found
Observed from the paper text and links we have. Absence here means we did not find it, not that it does not exist.
§