The authors split computer use between a privileged LLM, which plans and performs trusted actions, and a quarantined LLM, which processes potentially hostile content.
Accessibility-based masking shields computer agents from indirect prompt injections
SA-CUA separates trusted and attacker-controlled screen content, then verifies untrusted values before allowing them to influence privileged actions.
Academic
Dominik Seip · Bartol Markovinović · Matthias Hein
Tübingen AI Center · University of Tübingen
Research Digest··3 min read
Seip, Markovinović and Hein introduce SA-CUA, a dual-LLM architecture for agents that operate graphical interfaces through screenshots, mouse clicks and keyboard input.
What it released
Code
§