What they did

The authors extend the BLOOM auditing framework with WILT, which operates on both sides of an interaction. An auditor model revises its conversational strategy between rounds based on scores from earlier conversations, while the target model’s decoding distribution is adaptively tilted using its own probabilities under a separate elicitation prompt.

The method requires no additional model training, but it does require access to the target’s next-token distribution. The authors tested it on four target models and eight specified behaviors, comparing it with a baseline auditor and other elicitation methods under matched compute.

Key findings

• WILT outperformed the baseline auditor in 30 of 32 model–behavior settings.
• For self-harm encouragement in Qwen3.5-4B, average behavior presence increased from 51% with the baseline to 100% with WILT.
• In that self-harm experiment, WILT surpassed every other elicitation method ported into the same pipeline at matched compute.
• The resulting measurements reversed the models’ previous safety ranking, showing that rankings can depend strongly on how effectively rare behaviors are elicited.

Why it matters

Automated audits can miss consequential behaviors simply because ordinary sampling rarely reaches them. By adding optimization pressure to both the conversation strategy and decoding process, BLOOM-WILT offers a more sample-efficient way to probe models for specified failure modes and to test whether apparent safety differences persist under stronger elicitation.

Caveats

The evaluation covers only four models and eight behaviors, so broader generalization remains unproven. WILT also assumes access to next-token distributions, which many commercial model APIs do not provide, and decoding intervention may not perfectly represent behavior under ordinary deployment even though the authors report that sampled-output probability did not fall below the baseline in the highlighted experiment.