Mistral Large 4 Released: A 1 Trillion-Parameter Multimodal MoE Model

Hugging Face✍ TextPublished: Source: Transformers

#Mistral #Hugging Face #NVIDIA

Key point

Mistral has announced Mistral Large 4 (Le Chonk), a 1 trillion-parameter natively multimodal Mixture-of-Experts model. It supports over 160 languages and unifies reasoning, coding, and agentic workflows.

Mistral has announced Mistral Large 4 (Le Chonk), its largest and most capable model to date. This is a 1 trillion-parameter natively multimodal Mixture-of-Experts (MoE) model with 49 billion active parameters per token (52 billion including embeddings and output layers). It was trained from scratch on 3,800 NVIDIA Grace Blackwell GPUs in Mistral's own datacenters in Europe.

Mistral Large 4 unifies instruction following, reasoning, coding, and agentic workflows in a single set of weights. While a generalist model, the company claims it shows especially good performance on workloads such as cybersecurity, finance, and law, and surpasses frontier closed models on visual grounding. It is natively fluent in over 160 languages, including every official language of the European Union.

Architecturally, the model uses a granular MoE with 128 routed experts (4 active per token) and 1 shared expert, featuring a 1M context length. It accepts text and image inputs with text output. The vision encoder is Pixtral-style, trained to handle variable image sizes and aspect ratios. It also includes instruct and reasoning functionalities with function calls, where reasoning effort is configurable per request.

This item is a summary. The full, checkable version is at the original source:

Read the original at Transformers ↗