29 papers this week in Safety & security12 active threadsbusiest: Alignment Training Dynamics

Safety & security research

Evaluation & benchmarks

AI models now recite test answers from memory, inflating scores

Benchmark contamination has escalated from indirect optimization to explicit memorization. Qin et al. document that recent base models can reproduce entire test examples, questions, options, and reference answers, that were never in their input, with one model recalling 96% of a coding benchmark. They propose an Anti-Hack score that subtracts credit for leaked examples and show it moves a leading model to last place.

today