News without the noise
Alignment, red teaming, jailbreaks and attacks, guardrails, privacy, interpretability for safety
300 articles · page 7 of 7
Zhang et al. add intent-targeted tools to language-model agents, creating a dedicated channel through which models can signal commitment to particular behaviors. They report that tool-call probabilities produce fine-grained intent trajectories and help identify moments when intervention may prevent harmful action.
Jiang et al. train an agent safeguard to identify and invoke the safety policy relevant to a full execution trajectory. Initialized on their PolicyTraj-20K dataset and refined with reinforcement learning, RePolicy showed strong safety detection across six benchmarks and remained robust when policy contexts changed.
Chang et al. built a controlled air traffic control benchmark informed by aviation standards and feedback from 40 controllers across three countries. They report a consistent gap between standard semantic scores and evaluations that account for the unequal consequences of errors; risk-aware fine-tuning reduced, but did not eliminate, that gap.
The authors audited medical chain-of-thought by applying 30 clinically motivated perturbations to model rationales and questions across four medical QA benchmarks. They find that answers are frequently insensitive to meaningful changes in the displayed reasoning, suggesting that a correct answer does not necessarily make the visible rationale a faithful account of how it was produced.
Abdullah, Cambria and Živković coupled a clinical language model to a graph-based physiological world model that ranks candidate answers using homeostatic constraints, biomedical paths and drug-interaction penalties. On a 2,500-scenario safety benchmark, their training method raised the main safety score from 69.5% to 90.8% and reduced physician-evaluated harm from 14.1% to 5.1% on a blinded subset.
The authors trained a guard model to identify risky actions at individual steps in an agent’s trajectory, including before a tool call is executed. On AgentDojo and AgentDyn, StepGuard reduced mean attack success by 77.3% relative to unguarded agents while lowering mean utility by 2.8 percentage points.
The authors introduce Untrusted Content Masking (UCM), a defense that prevents prompt injection attacks on web agents by masking untrusted content in the Document Object Model (DOM) before the agent processes it. They show that the DOM structure alone can distinguish trusted from untrusted page regions without reading their content, enabling agents to observe and interact safely with web pages while remaining isolated from adversarial content.
The authors introduce WhisperBench, a 108-case benchmark for stealth memory injection, and MemGhost, a one-shot payload generation framework. In experiments, MemGhost achieves 87.5% end-to-end success on OpenClaw with GPT-5.4 and 71.4% on Claude Code SDK with Sonnet 4.6, revealing that persistent memory can be a practical pathway for long-term agent compromise.
Hills et al. introduce Iterative VibeCoding, a benchmark for AI control in persistent codebases, and demonstrate that gradual attacks—distributing a covert side task across multiple pull requests—achieve high evasion (≥65%) across several attack models. They find that no single monitor is robust against both gradual and non-gradual attack strategies, but a stateful link-tracker monitor combined with trajectory monitors in a four-monitor ensemble reduces gradual-attack evasion from 93% under the weakest diff monitor to 47%.
Feng et al. present Vera, an end-to-end automated safety testing framework for LLM agents that uses a three-stage pipeline to discover risks, compose executable safety cases, and execute them in sandboxed environments with evidence-grounded verification. Evaluating on four production agent frameworks (OpenClaw, Hermes, Codex, and Claude Code), they find average attack success rates of 93.9% under multi-channel attacks, and release Vera-Bench with 1600 safety cases spanning 124 risk categories.
Genewein et al. present a report examining how AI might develop beyond human-level AGI to artificial superintelligence (ASI). They characterize ASI as systems more cognitively capable than large organizations of humans, outline four possible pathways, and conclude that a single step-change event may be less likely than a series of transformative, AI-driven breakthroughs.
Zhou et al. conducted the first systematic measurement of authentication security in real-world remote Model Context Protocol (MCP) servers, scanning over 7,900 live servers and performing in-depth analysis on 119 OAuth-enabled servers. They found that 40.55% of servers require no authentication, and among authenticated servers, every tested server had at least one authentication flaw, with 96.6% vulnerable to dynamic client registration attacks.