Atlas

Benchmarks

Conceptual Reasoning Index

The Conceptual Reasoning Index (CRI) is the composite of the maintainers' conceptualreasoning.ai leaderboard for reasoning about hard-to-verify conceptual questions: 60% LMCA (agreement with expert ratings of argument critiques), 20% ACCoRD (consistency of stated beliefs) and 20% DTBench (decision theory), each on its leaderboard 0-100 scale. DTBench enters as accuracy rescaled so the 40% guessing level is 0, unlike Atlas's raw-accuracy `dtbench`. A model gets a CRI only when it has all three scores; 0 is random guessing and 100 the maximum on every benchmark. The maintainers say the composition and weights will change as benchmarks are added or retired. Diagnostic, because it reweights LMCA, ACCoRD and DTBench, which Atlas imports separately.

Models
142
Top score
79.9
Leader
Claude Fable 5.2
Updated
—
Resolution
—
Saturation
—

No signal or difficulty estimate. Held out of the fit by design.

Organizations
137 dated models · 18 frontier modelsClosed weightsOpen weights
Data: Conceptual Reasoning Benchmark Authorsatlas.village.dev

Conceptual Reasoning Index leaderboard

#ModelSettingScoreSource
1Claude Fable 5.2x-high
79.9
2Claude Opus 5.5max
77.9
3Claude Sonnet 5.5max
77.1
4Claude Fable 5.1medium
74.9
5Claude Opus 5x-high
74.7
6GPT-6 Astrax-high
73.9
7GPT-6.1 Solmax
72.7
8Claude Fable 5max
72.7
9Gemini 4 Argonlow
72.4
10GPT-6 Solmax
71.0
11GPT-5.6 Solmax
68.7
12Opus 4.8max
68.3
13Kimi K3.1 Promax
67.8
14Muse Wondermax
67.7
15GPT-5.5x-high
67.3
16Opus 4.6max
66.6
17GPT-5.5 Prox-high
66.2
18Muse Spark 1.3max
66.2
19Gemini 3.8 Flashmedium
65.7
20GPT-5.6 Terramax
65.7
21Opus 4.7max
65.5
22Gemini 3.1 Pro Previewmedium
65.4
23GPT-5.4x-high
65.3
24Claude Haiku 5.5max
65.3
25Gemini 3.7 Flashhigh
64.6
26Muse Spark 1.1thinking
63.8
27GLM-5.3max
63.4
28Grok 4.7x-high
63.1
29Kimi K3max
62.8
30Muse Spark 1.2x-high
62.7
31Grok 4.6x-high
62.5
32Sonnet 5max
61.8
33Grok 4.5high
60.3
34Gemini 3.5 Flashhigh
59.8
35Gemini 3.6 Flashhigh
59.5
36Qwen3.8-Maxthinking
59.4
37Qwen3.8 2.4T A95Bx-high
59.4
38DeepSeek V4 Pro 0813max
59.3
39Sonnet 4.6max
59.2
40GPT-5.2x-high
59.2
41Opus 4.5high
59.1
42GPT-5.6 Lunax-high
58.7
43GLM-5.2max
58.6
44DeepSeek V4.1 Flashmax
58.4
45GPT-6 Lunamax
57.8
46GPT-5.1high
57.3
47Qwen3.7-Max (source unspecified)thinking
56.9
48Gemini 3 Flash Previewhigh
56.0
49DeepSeek-V4-Promax
55.8
50GPT-5high
55.5
51Qwen3.8 27Bx-high
55.5
52Qwen3.6-Max-Previewthinking
55.4
53DeepSeek-V4-Flash-0731max
54.4
54Grok 4.20thinking
53.7
55Kimi K2.6thinking
53.6
56o3-prohigh
53.5
57o3high
53.0
58Hy3high
52.2
59GPT-5.4 Minix-high
51.9
60Sonnet 4.5thinking
51.8
61GPT-5.2 Instant (source unspecified)not supported
51.0
62Mistral Large 4 Previewhigh
50.7
63Gemini 3.5 Flash-Litehigh
50.7
64Muse Glimmer 30Bx-high
50.5
65Gemma 4 31B ITthinking
50.2
66Qwen3.5 397B A17Bthinking
49.7
67Opus 4thinking
49.5
68Qwen3.7-Plus (2026-05-26)thinking
49.4
69GPT-5.4 Nanox-high
49.0
70DeepSeek-V4-Flashmax
48.5
71GPT-5 Minihigh
48.3
72Opus 4.1thinking
48.3
73GPT-5.3 Instant (source unspecified)not supported
48.1
74Gemini 2.5 Prothinking
47.5
75Qwen3.5-27Bthinking
46.6
76MiniMax M3thinking
46.4
77DeepSeek-V3.2-Expthinking
45.7
78Qwen3.5 122B A10Bthinking
45.7
79Qwen3.6 Plus (2026-04-02)thinking
45.7
80DeepSeek-V3.2thinking
45.4
81Qwen3.6 27Bthinking
45.2
82MiMo-V2.5-Prothinking
44.2
83Haiku 4.5thinking
43.1
84Qwen3 235B A22B Thinking 2507always on
42.4
85Qwen3.5-Flash (source unspecified)thinking
42.3
86Qwen3.5 35B A3Bthinking
42.1
87DeepSeek-V3.1-Terminusthinking
42.0
88o4-minihigh
41.2
89Sonnet 4thinking
41.1
90Gemma 4 26B A4B ITthinking
40.3
91DeepSeek-V3.1thinking
39.9
92Qwen3.6 35B A3Bthinking
39.7
93gpt-oss-120bhigh
38.9
94Qwen3 235B A22B Instruct 2507not supported
38.7
95o1 (2024-12-17)high
37.8
96Gemini 2.5 Flashthinking
37.7
97Qwen3-Next-80B-A3B-Thinkingalways on
36.7
98Mistral Medium 3.5high
36.6
99Qwen3-235B-A22Bthinking
36.3
100GPT-4.1not supported
36.0
101Qwen3.5-9Bthinking
35.1
102Qwen3 Next 80B A3B Instructnot supported
34.0
103Nemotron 3.5 Lightning 30B A3B (precision unspecified)thinking
33.6
104Qwen3 30B A3B Thinking 2507always on
32.9
105o3-minihigh
32.3
106Qwen3 30B A3B Instruct 2507not supported
31.6
107GPT-4.1 Mininot supported
31.0
108Mistral Small 4high
30.5
109gpt-oss-20bhigh
30.4
110Qwen3 32Bthinking
29.2
111Qwen3 14Bthinking
28.6
112Mistral Medium 3.1not supported
28.4
113Mistral Large 3 675B Instruct 2512not supported
28.1
114GPT-4o (source unspecified)not supported
27.1
115Llama 4 Maverick Instructnot supported
26.8
116DeepSeek-V3-0324not supported
26.2
117Llama 3.3 70B Instructnot supported
26.1
118Qwen3-30B-A3Bthinking
25.2
119DeepSeek-V3not supported
25.0
120Llama 3.1 70B Instructnot supported
24.2
121Qwen2.5 72B Instructnot supported
24.2
122GPT-5 Nanohigh
24.1
123Opus 3not supported
24.0
124Llama 4 Scout Instructnot supported
23.1
125GPT-4 Turbonot supported
21.5
126Command Anot supported
20.7
127Qwen3 8Bthinking
20.2
128Gemma 3 27B ITnot supported
19.7
129Ministral 3 14B Instruct 2512not supported
19.3
130GPT-4o Mininot supported
17.5
131Ministral 3 8B Instruct 2512not supported
17.4
132Llama 3.1 8B Instructnot supported
13.6
133Gemma 3n E4B Instructnot supported
13.3
134Gemma 3 12B ITnot supported
13.3
135GPT-4.1 Nanonot supported
12.3
136Qwen2.5 7B Instructnot supported
11.9
137Claude 3 Haikunot supported
11.6
138C4AI Command R+ 08-2024not supported
11.6
139C4AI Command R 08-2024not supported
10.3
140Ministral 3 3B Instruct 2512not supported
10.1
141Gemma 3 4B ITnot supported
10.1
142Command R7B (Dec 2024)not supported
7.1

Related benchmarks

Loading Atlas data…