The authors treat an agent’s output as a trajectory of context-dependent actions and estimate the probability that this trajectory produces a specified rare event.
Weight perturbations efficiently estimate extremely rare failures in language-model agents
The authors construct importance-sampling proposals by modifying model weights, allowing probabilities as low as 10^-9 to be estimated without prohibitive naive sampling.
Independent
Hanming Yang · Daksh Mittal · Jing Dong · Hongseok Namkoong
Research Digest··2 min read
Thread:Agent Security & Attacks
Yang et al.
Why this paper
Independent · Part of Agent Security & Attacks, now 26 papers
In one line
Iterative unalignment estimates rare event probabilities in stochastic agent trajectories via gradient-based importance sampling.
What we could check
- ·No code link found
- ·No weights link found
- ·No dataset link found
- ·No compute details found
- ·No stated limitations found
- ·No benchmark numbers found
Observed from the paper text and links we have. Absence here means we did not find it, not that it does not exist.
§