Mistral releases open-weight ML4 model designed for long-horizon agent tasks
The 52B-active, 1T-total-parameter multimodal mixture-of-experts model treats agentic evaluations as a design constraint, pairing sparse attention with a 606 GB checkpoint deployable on four GB300 GPUs.
Mistral has released Mistral Large 4 (ML4), an open-weight multimodal mixture-of-experts model with 52 billion active parameters and 1 trillion total parameters, designed for reasoning and long-horizon agentic tasks. The model achieves competitive scores on code, cybersecurity and agentic benchmarks, including 19.7% on FrontierCode 2.0, 41% on Terminal-Bench 4.0 and 50.9% on the Artificial Analysis Cyber Index.
The model's development was shaped by a key observation: language-modeling loss can be an inadequate proxy for performance on SWE-bench Verified. The initial combination of sliding-window attention, cross-layer attention and YOCO reduced KV-cache size by a factor of 16; at the same training-token budget, SWE-bench Verified scores declined moving from full attention to sliding-window attention and declined further when cross-layer attention was added. The team behind ML4 performed “model surgery” on an existing checkpoint, removing the sliding-window limits and cross-layer attention while retaining some YOCO cache sharing after a larger-scale test found that keeping it performed better.
To control serving cost, ML4 adopts sparse attention, in which a lightweight indexer selects the top 2,048 key tokens for each query. In a compute-scale experiment on surgery and midtraining, the indexer was warmed up for about 15.1 billion tokens with the main model frozen and using full attention, then both were trained jointly under sparse attention for a further 198 billion tokens. The authors report that calibrated NVFP4 quantization reduces the checkpoint to 606 GB while matching the FP8 reference on evaluations, enabling deployment on four GB300 GPUs.
The model also introduces a post-training data pipeline in which agents create executable environments, task specifications, reference solutions and verifiers. For repository tasks, an authoring agent builds a runnable container and identifies tests that fail before a fix and pass after it. A refinement agent has several solvers attempt the task, then repairs ambiguous specifications or unreliable tests. The authors stress that agents need external verification and supervision, and some domains still require post hoc adjustments.
Multiple-teacher distillation provides high-, medium- and low-reasoning-effort modes, letting users control the trade-off between token consumption and performance.
On benchmarks, ML4 scores 41.0% on Terminal-Bench 4.0 against 40.0% for GLM-5.3 in a reported code evaluation table, and 50.9% on the Artificial Analysis Cyber Index, below DeepSeek V4.1-Flash's 56.8%. The comparisons are cross-model, not controlled tests of the surgery or task-authoring pipeline.
"Mistral Large 4"
paper published exclusively on alphaXiv
Their latest technical report shares that aggressively reducing KV-cache memory can hurt an LLM's agentic capabilities, even when its language modeling performance looks unchanged.
Mistral initially reduced KV-cache memory by 16x, but they found that this hurts long-context agentic tasks, where models need to retrieve information from much earlier context.
To fix this, Mistral restored full-context attention to more layers, then adopted DeepSeek Sparse Attention to select the 2,048 most relevant tokens across the context instead of attending to everything.
Their resulting 1T-parameter MoE model (w 52B active) scores 41% on Terminal-Bench 4.0 and 19.7% on FrontierCode 2.0, and its NVFP4-quantized checkpoint fits on four GB300 GPUs.
ML4's design prioritises agentic benchmarks over language-model loss, a shift that could influence how other labs evaluate and optimise long-context models.
The open-weight release allows researchers and developers to run a capable agentic model on four GB300 GPUs, lowering the hardware barrier for advanced agent applications.
The finding that apparently healthy language-model loss can mask poor agentic performance warns that standard evaluation regimes may miss capabilities critical for autonomous systems.
Background
Mistral Large 4 is a multimodal mixture-of-experts model with 52 billion active parameters out of 1 trillion total, developed by French AI company Mistral AI. The model is designed for tasks that require agents to work over long horizons, opening files, running tests, navigating codebases, where attention to earlier context is essential. The work builds on earlier sparse attention approaches such as DeepSeek Sparse Attention but departs by centring agentic evaluations as a design constraint rather than language-model loss alone.
Key Perspectives
Mistral's researchers: The architecture can penalise agentic capabilities even when language-model loss is similar; longer training can mitigate the penalty, but language-model loss is an inadequate proxy for agentic tasks.
Authors of the paper: Language-modeling loss did not ensure matching long-horizon agent results, and agentic evaluations guided architecture changes; this is a difference in what was measured and prioritised, not a claim that DeepSeek's conclusion is false.
What to Watch
Whether other labs adopt agentic benchmarks as a primary design constraint, and what effect that has on model architecture choices.
How ML4 performs in real-world deployments of autonomous coding and cybersecurity agents, beyond controlled benchmarks.
The extent to which the sparse attention approach scales to even longer contexts or larger agentic tasks.
12 claims in this story checked against 1 source: 12 supported.
Show each claim
SupportedMistral released an open-weight model called ML4.alphaXiv: “Mistral releases open-weight ML4 with technical report.”
SupportedML4 is a multimodal mixture-of-experts model with 52 billion active and 1 trillion total parameters, designed for reasoning and long-horizon agentic tasks.alphaXiv: “We introduce Mistral Large 4 (ML4), a 52B active, 1T total parameter multimodal mixture-of-experts model designed for reasoning and long-horizon agentic tasks.”
SupportedMistral treated agentic evaluations as a design constraint for ML4.alphaXiv: “The authors' lesson is to treat long-horizon agent evaluations, not language-model loss alone, as a design constraint”
SupportedCalibrated NVFP4 quantization reduces the checkpoint to 606 GB while matching the FP8 reference on evaluations, enabling deployment on four GB300 GPUs.alphaXiv: “Calibrated NVFP4 quantization reduces the checkpoint to 606 GB while matching the FP8 reference on evaluations, enabling deployment on four GB300 GPUs.”
SupportedML4 scores 19.7% on FrontierCode 2.0, 41% on Terminal-Bench 4.0 and 50.9% on the Artificial Analysis Cyber Index.alphaXiv: “Mistral Large 4 achieves competitive performance on code, cybersecurity, and held-out agentic benchmarks, including 19.7% on FrontierCode 2.0, 41% on Terminal-Bench 4.0, and 50.9% on the Artificial Analysis Cyber Index.”
SupportedThe authors observed that language-modeling loss can be an inadequate proxy for performance on SWE-bench Verified.alphaXiv: “These ablations support the reading that language-modeling loss is an inadequate proxy for this agentic task.”
SupportedThe initial attention design reduced KV-cache size by a factor of 16, while SWE-bench Verified scores declined from full attention to sliding-window attention and declined further when cross-layer attention was added, at the same training-token budget.alphaXiv: “Together these reduced KV-cache size by a factor of 16 versus a full-attention baseline. ... At the same training-token budget, SWE-bench Verified scores declined moving from full attention to SWA, and declined further when CLA was added.”
SupportedThe team performed “model surgery” on an existing checkpoint, removing sliding-window limits and cross-layer attention while retaining some YOCO cache sharing after larger-scale tests found keeping it performed better.alphaXiv: “Rather than start over, the authors performed "model surgery" on an existing checkpoint. ... At full model scale they kept YOCO, because a larger-scale test found that keeping it performed better.”
SupportedML4 uses sparse attention in which a lightweight indexer selects the top 2,048 key tokens for each query.alphaXiv: “To control cost, the model then adopts DSA, in which a lightweight indexer chooses the top 2,048 key tokens for each query.”
SupportedIn a compute-scale surgery and midtraining experiment, the indexer was warmed up for about 15.1 billion tokens with the main model frozen and using full attention, then the model and indexer were trained jointly under sparse attention for a further 198 billion tokens.alphaXiv: “The indexer is first warmed up while the main model stays frozen and uses full attention (about 15.1 billion tokens), and then the model and indexer are trained jointly under sparse attention for an additional 198 billion tokens.”
SupportedThe post-training pipeline uses agents to create executable environments, task specifications, reference solutions and verifiers; for repository tasks, an authoring agent builds a runnable container and identifies tests that fail before the reference fix and pass after it.alphaXiv: “Agents create executable environments, task specifications, reference solutions, and verifiers. For repository tasks, an authoring agent builds a runnable container and identifies tests that fail before the reference fix and pass after it.”
SupportedML4's 50.9% score on the Artificial Analysis Cyber Index is below DeepSeek V4.1-Flash's 56.8%.alphaXiv: “There is also a clear exception: on the Artificial Analysis Cyber Index, ML4 scores 50.9%, below DeepSeek V4.1-Flash's 56.8% in the cybersecurity table.”
