Search

6 results for “MMLU”

1. Research · Agents

   Joint harness search balances model accuracy, safety, and token use

   MoMHa: Multi-Objective Optimization of LLM Harnesses over Accuracy, Safety, and Tokens

   Across 17 domains and 12 models, jointly optimizing the code around language models outperformed sequential and accuracy-only approaches.

   3D hypervolume: 0.36 hypervolume indicator (as reported)

   28 Sept 2026Adobe Research

2. Research · Safety & security

   Method to permanently erase unwanted LLM personas via weight edits

   LLM Persona Unlearning

   A new benchmark and contrastive erasure technique suppress target personas while preserving utility across six models and five personas.

   PersonaUnlearnBench: 100% TF (as reported)

   1 Oct 2026University of Macau, Mila – Québec AI Institute, McGill University

3. Research · Safety & security

   Local evaluation reveals activation patching can cause benign regressions even when global metrics hold steady

   PatchBench: Measuring Collateral Damage in Activation Patching

   PatchBench curates model-specific jailbreak failures and introduces a protocol to measure harmful correction and benign preservation around each repair.

   MMLU: 57.88% accuracy (as reported)

   9 Oct 2026LIX (École Polytechnique, IP Paris, CNRS), AMIAD (Agence Ministérielle pour l’IA de Défense)

4. Research · Safety & security

   Language models favor suggested answers even when told the source is unreliable

   How Much Do You Trust A Dog? Quantifying Hint-Following in Reasoning Language Models

   Across three model families, the authors find structured shifts toward suggested answers, even when prompts explicitly warn that the source is unreliable.

   MMLU: 73.2% target rate (as reported)

   26 Oct 2026Northwestern University, NSF-Simons AI Institute for the Sky (SkAI), Stanford University

5. Research · Safety & security

   TRACE reduces multi-turn attack success while largely preserving model utility

   TRACE: Trajectory Return Attribution and Contrastive Erasure for Multi-Turn Safety

   A token-level safety objective outperformed alternatives across 35 model and attack combinations, with benchmark utility falling by no more than 1.23 points.

   MMLU and HellaSwag: 1.23 pts accuracy drop (as reported)

   2 Oct 2026PRADA Lab, King Abdullah University of Science and Technology, State Key Laboratory of Internet of Things for Smart City

6. Research · Safety & security

   Pretrained LLMs already contain refusal circuits that shift during alignment

   SafeEvo: Deciphering the Safety Alignment Mechanism and Evolution in Language Models

   SafeEvo identifies sparse refusal circuits in base models, traces their structural drift across safety checkpoints, and uses them to improve fine-tuning.

   HarmBench: 55.98% ASR increase (as reported)

   9 Oct 2026The University of Hong Kong, Chinese Academy of Sciences, Information Engineering University