Atlas

Models

← All models

Mistral NeMo Instruct 2407

Mistral AI · 2024-07-18 · 12.2B parameters

Mistral NeMo Instruct 2407 is the instruction-fine-tuned checkpoint of the dense Mistral NeMo model, jointly trained by Mistral AI and NVIDIA and released on July 18, 2024. It is a text-only model with a 131,072-token context window and function-calling training, and uses the Tekken tokenizer with a 131,072-token vocabulary. Its downloadable weights are licensed under Apache-2.0; Mistral exposed the checkpoint as `open-mistral-nemo-2407` on la Plateforme.

Benchmark scores

BenchmarkScore
BALROG17.6
BALROG — BabaIsAI (LLM)20.8
BALROG — BabyAI (LLM)50.0
BALROG — Crafter (LLM)27.7
BALROG — MiniHack (LLM)2.5
BALROG — NetHack (LLM)0.3
BALROG — TextWorld (LLM)4.5
BBH (Open LLM Leaderboard v2)50.4
BFCL v4 Overall Accuracy27.6
BuseyBench SVG0.8
Capability109.1
Creative Writing v277.5
Creative Writing v3 Elo881
Creative Writing v3 Rubric Score9.5
CRED v0.92 - Data integrity0.0
CRED v0.92 - Execution65.0
CRED v0.92 - Paper–code1.1
CRED v0.92 - Prose–table4.0
DTBench48.6
Epoch Capabilities Index (ECI)119
EQ-Bench v277.1
EQ-Bench v2 + MAGI-Hard Combined60.4
FormationEval78.8
GPQA (Open LLM Leaderboard v2)29.0
IFEval (Open LLM Leaderboard v2)63.8
LisanBench Difficulty-Weighted11.5
LisanBench Path Length48.7
Longform Creative Writing28.7
MAGI-Hard43.6
MATH Level 5 (Open LLM Leaderboard v2)12.7
Loading Atlas data…