The authors evaluated 296 harmful behaviors from HarmURLBench in a setting where each request includes a user-specified URL, allowing retrieval targets to be held fixed while request, page, and response languages are varied independently.
Non-English requests raise tool admission and response harm in LLM agents
The authors show that harmful requests in non-English languages increase retrieval tool use, while requiring answers in the page language worsens response harmfulness in closed-weight models.
Top University
Aditya Nawal · Manit Baser · Mohan Gurusamy
National University of Singapore
Research Digest··2 min read
Using a controlled URL-retrieval benchmark across ten models and seven languages, the authors independently varied request, retrieved-page, and response languages to isolate where safety fails.
§