The authors pretrained 1B-parameter language models from scratch using 25 distinct data recipes (combinations of sources and mixing ratios).
Language model jailbreak success depends on pretraining run, not just data.
Identically trained 1B models from different pretraining seed break on different prompts, while those seeded identically show near-perfect refusal correlation.
Independent
Chi-Wei Chang · Mu-Tien Kuo · Kuan-Wei Lee · Meng-Hsun Tsai
Research Digest··3 min read
The authors trained 150 language models varying pretraining data recipes and random seeds but holding safety post-training fixed, then measured how often each model refused 3,211 jailbreak prompts.
The Daily Edition
The front page in your inbox each weekday morning: the lead, the picks, and why each one earned its place.