4 million lines of code across Python, JavaScript, Java, and TypeScript covering six task domains.
Only 5.3% of LLM call sites in agentic systems enforce all expected constraints
Weng et al. analyze 496 call sites across 40 systems and find that the presence of checks does not guarantee adequacy, with 63% of expected constraints unmet.
Top University
Shihao Weng · Longtian Wang · Yang Feng · Mingfei Cheng · Jia Liu · Chao Shen · +3 more
Nanjing University · Xi'an Jiaotong University · Singapore Management University · University College London
Research Digest··2 min read
Thread:Agent Security & Attacks
The authors present a systematic empirical study of constraints around LLM call sites in open-source agentic systems.
§