Atlas

Benchmarks

NoCoT-Bench NCRI 15.2

Neel Nanda's NoCoT-Bench measures how much reasoning a language model can do in a single forward pass, with no chain of thought, which bears on how much reasoning a chain-of-thought monitor could miss. Models answer generated problems (arithmetic chains, sudoku, shortest paths, context-free grammars, multi-hop and multi-document tasks, order tracking, constraint counting and more, plus GPQA and contest items) at once, under per-model recipes that suppress reasoning, and every row is checked for reasoning tokens and visible working; a row that reasoned is scored wrong. NCRI 15.2 is a Rasch ability fitted over 76 difficulty rungs (64 sealed, plus 12 harder rungs bought for the top 35 models) in 19 effective domains and reported as 100 + (10/ln 2)·θ: 100 is the average sealed rung, +10 points doubles the odds of solving any rung, and negative values are allowed. The scale is anchored on the sealed items rather than the models but is unbounded, so it stays out of the capability fit.

Models
228
Top score
159
Leader
GPT-6 Astra
Updated
—
Resolution
—
Saturation
—

No signal or difficulty estimate. This benchmark has no declared score range, so it cannot be placed on the 0-1 scale the capability curve requires.

Organizations
226 dated models · 17 frontier modelsClosed weightsOpen weights
Data: Neel Nandaatlas.village.dev

NoCoT-Bench NCRI 15.2 leaderboard

#ModelSettingScoreSource
1GPT-6 Astradisabled · NoCoT-Bench no-chain-of-thought ask
159
2Gemini 4 Argondisabled · NoCoT-Bench no-chain-of-thought ask
151
3GPT-6.1 Soldisabled · NoCoT-Bench no-chain-of-thought ask
149
4GPT-6.1 Terradisabled · NoCoT-Bench no-chain-of-thought ask
130
5Claude Fable 5.1disabled · NoCoT-Bench no-chain-of-thought ask
128
6Gemini 3.7 Flashdisabled · NoCoT-Bench no-chain-of-thought ask
126
7Gemini 3.8 Flashdisabled · NoCoT-Bench no-chain-of-thought ask
126
8Claude Fable 5disabled · NoCoT-Bench no-chain-of-thought ask
119
9Gemini 3.1 Pro Previewdisabled · NoCoT-Bench no-chain-of-thought ask
115
10Claude Opus 5disabled · NoCoT-Bench no-chain-of-thought ask
115
11GPT-6 Soldisabled · NoCoT-Bench no-chain-of-thought ask
115
12GPT-5.6 Soldisabled · NoCoT-Bench no-chain-of-thought ask
109
13Opus 4.6disabled · NoCoT-Bench no-chain-of-thought ask
104
14Opus 4.7disabled · NoCoT-Bench no-chain-of-thought ask
104
15Gemini 3 Flash Previewdisabled · NoCoT-Bench no-chain-of-thought ask
103
16Kimi K3.1 Prodisabled · NoCoT-Bench no-chain-of-thought ask
103
17Opus 4.5disabled · NoCoT-Bench no-chain-of-thought ask
103
18Opus 4.1disabled · NoCoT-Bench no-chain-of-thought ask
103
19Opus 4.8disabled · NoCoT-Bench no-chain-of-thought ask
102
20Gemini 3.5 Flashdisabled · NoCoT-Bench no-chain-of-thought ask
102
21Opus 4disabled · NoCoT-Bench no-chain-of-thought ask
101
22Gemini 3.6 Flashdisabled · NoCoT-Bench no-chain-of-thought ask
101
23GPT-5.5disabled · NoCoT-Bench no-chain-of-thought ask
100
24Gemini 2.5 Prodisabled · NoCoT-Bench no-chain-of-thought ask
99.8
25Muse Spark 1.3disabled · NoCoT-Bench no-chain-of-thought ask
98.3
26Muse Wonderdisabled · NoCoT-Bench no-chain-of-thought ask
97.7
27Muse Spark 1.2disabled · NoCoT-Bench no-chain-of-thought ask
97.1
28Muse Spark 1.1disabled · NoCoT-Bench no-chain-of-thought ask
96.8
29Kimi K3disabled · NoCoT-Bench no-chain-of-thought ask
96.8
30Muse Spark Opendisabled · NoCoT-Bench no-chain-of-thought ask
95.7
31Claude Sonnet 5.5disabled · NoCoT-Bench no-chain-of-thought ask
95.1
32Hy4disabled · NoCoT-Bench no-chain-of-thought ask
94.0
33GPT-5.4disabled · NoCoT-Bench no-chain-of-thought ask
94.0
34Sonnet 4.5disabled · NoCoT-Bench no-chain-of-thought ask
94.0
35Sonnet 4disabled · NoCoT-Bench no-chain-of-thought ask
91.3
36Sonnet 5disabled · NoCoT-Bench no-chain-of-thought ask
90.8
37GPT-5.6 Terradisabled · NoCoT-Bench no-chain-of-thought ask
89.0
38Sonnet 4.6disabled · NoCoT-Bench no-chain-of-thought ask
87.8
39Qwen3.6-Max-Previewdisabled · NoCoT-Bench no-chain-of-thought ask
87.0
40Grok 4.6disabled · NoCoT-Bench no-chain-of-thought ask
86.8
41Qwen3.8 2.4T A95Bdisabled · NoCoT-Bench no-chain-of-thought ask
86.0
42GPT-5.2disabled · NoCoT-Bench no-chain-of-thought ask
85.8
43GPT-5.3-Codexdisabled · NoCoT-Bench no-chain-of-thought ask
85.2
44DeepSeek V4 Pro 0813disabled · NoCoT-Bench no-chain-of-thought ask
84.1
45Hy4 Previewdisabled · NoCoT-Bench no-chain-of-thought ask
82.8
46LongCat-2.0disabled · NoCoT-Bench no-chain-of-thought ask
82.5
47Qwen3.6 Plus (2026-04-02)disabled · NoCoT-Bench no-chain-of-thought ask
82.2
48Qwen3 Max Thinking (2026-01-23)disabled · NoCoT-Bench no-chain-of-thought ask
81.9
49DeepSeek V4.1 Flash 1017disabled · NoCoT-Bench no-chain-of-thought ask
81.6
50Gemma 4 31B ITdisabled · NoCoT-Bench no-chain-of-thought ask
81.0
51Grok 4.5disabled · NoCoT-Bench no-chain-of-thought ask
81.0
52KAT-Coder-Pro V2not supported · NoCoT-Bench no-chain-of-thought ask
80.8
53Gemini 3.5 Flash-Litedisabled · NoCoT-Bench no-chain-of-thought ask
80.6
54Qwen3.5 Plus (2026-02-15)disabled · NoCoT-Bench no-chain-of-thought ask
80.6
55Kimi K2.5disabled · NoCoT-Bench no-chain-of-thought ask
80.6
56Qwen3.5 397B A17Bdisabled · NoCoT-Bench no-chain-of-thought ask
80.0
57Kimi K2.6disabled · NoCoT-Bench no-chain-of-thought ask
79.2
58DeepSeek-V4-Flash-Basenot supported · NoCoT-Bench no-chain-of-thought ask
79.1
59Gemini 2.5 Flashdisabled · NoCoT-Bench no-chain-of-thought ask
78.8
60Nemotron 3 Ultra 550B A55Bdisabled · NoCoT-Bench no-chain-of-thought ask
78.3
61Inklingdisabled · NoCoT-Bench no-chain-of-thought ask
78.3
62Kimi K2 Instructnot supported · NoCoT-Bench no-chain-of-thought ask
78.0
63DeepSeek-V3.1-Terminusdisabled · NoCoT-Bench no-chain-of-thought ask
77.6
64GLM-5.2disabled · NoCoT-Bench no-chain-of-thought ask
77.3
65GPT-6 Lunadisabled · NoCoT-Bench no-chain-of-thought ask
77.3
66Qwen3-VL-235B-A22B-Instructnot supported · NoCoT-Bench no-chain-of-thought ask
77.3
67MiMo-V2.5-Prodisabled · NoCoT-Bench no-chain-of-thought ask
77.1
68DeepSeek-V3.2-Expdisabled · NoCoT-Bench no-chain-of-thought ask
76.7
69GPT-5.6 Lunadisabled · NoCoT-Bench no-chain-of-thought ask
76.6
70DeepSeek-V3.1disabled · NoCoT-Bench no-chain-of-thought ask
76.5
71GLM-5.3 Flashdisabled · NoCoT-Bench no-chain-of-thought ask
76.4
72GLM-5disabled · NoCoT-Bench no-chain-of-thought ask
76.2
73DeepSeek-V4-Flash-0731disabled · NoCoT-Bench no-chain-of-thought ask
75.5
74GLM-5.1disabled · NoCoT-Bench no-chain-of-thought ask
75.2
75GPT-4.1not supported · NoCoT-Bench no-chain-of-thought ask
75.1
76DeepSeek-V3.2disabled · NoCoT-Bench no-chain-of-thought ask
75.0
77Qwen3 235B A22B Instruct 2507not supported · NoCoT-Bench no-chain-of-thought ask
74.6
78GLM-5.3disabled · NoCoT-Bench no-chain-of-thought ask
74.5
79Qwen3.8 Flashdisabled · NoCoT-Bench no-chain-of-thought ask
74.4
80DeepSeek-V3-0324not supported · NoCoT-Bench no-chain-of-thought ask
74.4
81GPT-4o (2024-08-06)not supported · NoCoT-Bench no-chain-of-thought ask
74.2
82Haiku 4.5disabled · NoCoT-Bench no-chain-of-thought ask
74.2
83Ling 2.6 1Tnot supported · NoCoT-Bench no-chain-of-thought ask
73.9
84WizardLM-2 8x22Bnot supported · NoCoT-Bench no-chain-of-thought ask
73.2
85Qwen3-Coder-480B-A35B-Instructnot supported · NoCoT-Bench no-chain-of-thought ask
73.1
86DeepSeek-V3not supported · NoCoT-Bench no-chain-of-thought ask
72.9
87GPT-4 0613not supported · NoCoT-Bench no-chain-of-thought ask
72.8
88MiniMax M3disabled · NoCoT-Bench no-chain-of-thought ask
72.6
89Hy3disabled · NoCoT-Bench no-chain-of-thought ask
72.5
90Qwen3.5 Plus 2026-04-20disabled · NoCoT-Bench no-chain-of-thought ask
72.3
91Mercury 3disabled · NoCoT-Bench no-chain-of-thought ask
72.3
92Hy3 previewdisabled · NoCoT-Bench no-chain-of-thought ask
72.3
93GPT-4o (2024-05-13)not supported · NoCoT-Bench no-chain-of-thought ask
72.2
94GLM-4.7disabled · NoCoT-Bench no-chain-of-thought ask
71.3
95GPT-4 Turbonot supported · NoCoT-Bench no-chain-of-thought ask
71.3
96GLM-4.5disabled · NoCoT-Bench no-chain-of-thought ask
71.3
97GLM 4.6disabled · NoCoT-Bench no-chain-of-thought ask
70.8
98GPT-5.1disabled · NoCoT-Bench no-chain-of-thought ask
70.6
99Qwen3-235B-A22Bdisabled · NoCoT-Bench no-chain-of-thought ask
70.5
100Llama 4 Maverick Instructnot supported · NoCoT-Bench no-chain-of-thought ask
70.5
101GPT-5.4 Minidisabled · NoCoT-Bench no-chain-of-thought ask
70.5
102GPT-4o (2024-11-20)not supported · NoCoT-Bench no-chain-of-thought ask
70.3
103MiMo-V2.5disabled · NoCoT-Bench no-chain-of-thought ask
69.9
104Gemma 4 26B A4B ITdisabled · NoCoT-Bench no-chain-of-thought ask
69.5
105Qwen3.5 122B A10Bdisabled · NoCoT-Bench no-chain-of-thought ask
69.4
106Grok 4.3disabled · NoCoT-Bench no-chain-of-thought ask
69.2
107Qwen3 235B A22B Thinking 2507always on · NoCoT-Bench no-chain-of-thought ask
68.9
108Qwen3 Next 80B A3B Instructnot supported · NoCoT-Bench no-chain-of-thought ask
68.4
109MiniMax M3.1 Flashdisabled · NoCoT-Bench no-chain-of-thought ask
68.4
110Claude Haiku 5.5disabled · NoCoT-Bench no-chain-of-thought ask
68.3
111Mistral Medium 3.5disabled · NoCoT-Bench no-chain-of-thought ask
67.5
112Grok 4.20disabled · NoCoT-Bench no-chain-of-thought ask
67.1
113GPT-5disabled · NoCoT-Bench no-chain-of-thought ask
67.0
114ERNIE 4.5 VL 424B A47Bdisabled · NoCoT-Bench no-chain-of-thought ask
66.9
115GLM-4.5Vdisabled · NoCoT-Bench no-chain-of-thought ask
66.7
116Qwen3.5-27Bdisabled · NoCoT-Bench no-chain-of-thought ask
66.6
117Ling 3.0 Flashdisabled · NoCoT-Bench no-chain-of-thought ask
66.3
118Muse Glimmer 30Bdisabled · NoCoT-Bench no-chain-of-thought ask
66.3
119Nemotron 3 Super 120B A12Bdisabled · NoCoT-Bench no-chain-of-thought ask
66.2
120Qwen3.6 27Bdisabled · NoCoT-Bench no-chain-of-thought ask
66.2
121Qwen3.8 27Bdisabled · NoCoT-Bench no-chain-of-thought ask
66.1
122Ling 2.6 Flashnot supported · NoCoT-Bench no-chain-of-thought ask
65.5
123Qwen3-Coder-Nextnot supported · NoCoT-Bench no-chain-of-thought ask
65.5
124Solar Pro 4disabled · NoCoT-Bench no-chain-of-thought ask
65.3
125Qwen2.5-VL-72B-Instructnot supported · NoCoT-Bench no-chain-of-thought ask
65.2
126GPT-4.1 Mininot supported · NoCoT-Bench no-chain-of-thought ask
65.0
127MiniMax M2.5always on · NoCoT-Bench no-chain-of-thought ask
65.0
128Beam 1.0 501B A23Bdisabled · NoCoT-Bench no-chain-of-thought ask
64.8
129Hermes 3 Llama 3.1 405Bnot supported · NoCoT-Bench no-chain-of-thought ask
64.6
130Gemini 2.5 Flash-Litedisabled · NoCoT-Bench no-chain-of-thought ask
64.5
131MiniMax M2.7always on · NoCoT-Bench no-chain-of-thought ask
64.0
132Qwen3 VL 32B Instructnot supported · NoCoT-Bench no-chain-of-thought ask
64.0
133Qwen3-Next-80B-A3B-Thinkingalways on · NoCoT-Bench no-chain-of-thought ask
63.8
134L3.3-70B-Euryale-v2.3not supported · NoCoT-Bench no-chain-of-thought ask
63.8
135Gemma 4 31Bnot supported · NoCoT-Bench no-chain-of-thought ask
63.6
136Mistral Small 4disabled · NoCoT-Bench no-chain-of-thought ask
63.5
137GLM-4.6Vdisabled · NoCoT-Bench no-chain-of-thought ask
63.4
138Qwen3.7 Flashdisabled · NoCoT-Bench no-chain-of-thought ask
63.3
139Qwen2.5 72Bnot supported · NoCoT-Bench no-chain-of-thought ask
63.1
140Hermes 4 405Bdisabled · NoCoT-Bench no-chain-of-thought ask
62.8
141Llama 3.1 70B Instructnot supported · NoCoT-Bench no-chain-of-thought ask
62.8
142Qwen2.5 Instruct 32Bnot supported · NoCoT-Bench no-chain-of-thought ask
62.7
143Llama 3.3 70B Instructnot supported · NoCoT-Bench no-chain-of-thought ask
62.6
144Laguna S 2.1disabled · NoCoT-Bench no-chain-of-thought ask
62.5
145GLM 4.5 Airdisabled · NoCoT-Bench no-chain-of-thought ask
62.4
146Qwen2.5 72B Instructnot supported · NoCoT-Bench no-chain-of-thought ask
62.1
147Qwen3 30B A3B Instruct 2507not supported · NoCoT-Bench no-chain-of-thought ask
62.1
148Qwen3.6 35B A3Bdisabled · NoCoT-Bench no-chain-of-thought ask
61.6
149Qwen3.5-Flash (2026-02-23)disabled · NoCoT-Bench no-chain-of-thought ask
61.4
150Nova Pronot supported · NoCoT-Bench no-chain-of-thought ask
61.3
151Qwen2.5 32Bnot supported · NoCoT-Bench no-chain-of-thought ask
61.1
152Qwen2 72B Instructnot supported · NoCoT-Bench no-chain-of-thought ask
61.0
153Qwen3.5 35B A3Bdisabled · NoCoT-Bench no-chain-of-thought ask
60.1
154Qwen2.5 Coder 32B Instructnot supported · NoCoT-Bench no-chain-of-thought ask
60.0
155Qwen3 VL 30B A3B Instructnot supported · NoCoT-Bench no-chain-of-thought ask
59.9
156Gemma 4 12B ITdisabled · NoCoT-Bench no-chain-of-thought ask
59.7
157Qwen3 Coder 30B A3B Instructnot supported · NoCoT-Bench no-chain-of-thought ask
59.6
158Ripple-65B Previewunknown · NoCoT-Bench no-chain-of-thought ask
59.3
159Qwen3 32Bdisabled · NoCoT-Bench no-chain-of-thought ask
59.2
160Cydonia 24B v4.1not supported · NoCoT-Bench no-chain-of-thought ask
59.1
161Ministral 3 14B Instruct 2512not supported · NoCoT-Bench no-chain-of-thought ask
58.7
162Phi-4not supported · NoCoT-Bench no-chain-of-thought ask
58.3
163gpt-oss-120bdisabled · NoCoT-Bench no-chain-of-thought ask
58.1
164Hermes 4 70Bdisabled · NoCoT-Bench no-chain-of-thought ask
57.9
165Llama 4 Scout Instructnot supported · NoCoT-Bench no-chain-of-thought ask
57.7
166Llama 3.1 70Bnot supported · NoCoT-Bench no-chain-of-thought ask
57.6
167Laguna XS 2.1disabled · NoCoT-Bench no-chain-of-thought ask
57.5
168Gemma 3 27B PTnot supported · NoCoT-Bench no-chain-of-thought ask
57.5
169Claude 3 Haikunot supported · NoCoT-Bench no-chain-of-thought ask
57.3
170Mistral Small 3 24B Base 2501not supported · NoCoT-Bench no-chain-of-thought ask
56.2
171Qwen3-VL-8B-Instructnot supported · NoCoT-Bench no-chain-of-thought ask
56.0
172GLM 4.7 Flashdisabled · NoCoT-Bench no-chain-of-thought ask
55.9
173GPT-4o Mininot supported · NoCoT-Bench no-chain-of-thought ask
55.7
174Mistral Small 3 24B Instruct 2501not supported · NoCoT-Bench no-chain-of-thought ask
55.4
175GPT-3.5 Turbo 0125not supported · NoCoT-Bench no-chain-of-thought ask
55.2
176Mercury 2disabled · NoCoT-Bench no-chain-of-thought ask
55.1
177Qwen3 14Bdisabled · NoCoT-Bench no-chain-of-thought ask
55.0
178Gemma 3 27B ITnot supported · NoCoT-Bench no-chain-of-thought ask
54.8
179Olmo 3.1 32B Instructnot supported · NoCoT-Bench no-chain-of-thought ask
54.3
180Nemotron 3.5 Lightning 30B A3B (precision unspecified)disabled · NoCoT-Bench no-chain-of-thought ask
54.1
181Mistral Small 3.2 24B Instruct 2506not supported · NoCoT-Bench no-chain-of-thought ask
53.8
182Gemma 2 27Bnot supported · NoCoT-Bench no-chain-of-thought ask
53.3
183Gemma 2 27B ITnot supported · NoCoT-Bench no-chain-of-thought ask
52.6
184Qwen1.5 110B Chatnot supported · NoCoT-Bench no-chain-of-thought ask
52.6
185Qwen3.5-9Bdisabled · NoCoT-Bench no-chain-of-thought ask
52.4
186gpt-oss-20bdisabled · NoCoT-Bench no-chain-of-thought ask
51.5
187Qwen3 8Bdisabled · NoCoT-Bench no-chain-of-thought ask
51.4
188Ministral 3 8B Instruct 2512not supported · NoCoT-Bench no-chain-of-thought ask
51.0
189GPT-5 Minidisabled · NoCoT-Bench no-chain-of-thought ask
50.7
190Gemma 4 E4B ITdisabled · NoCoT-Bench no-chain-of-thought ask
50.6
191Qwen3-30B-A3Bdisabled · NoCoT-Bench no-chain-of-thought ask
50.4
192GPT-4.1 Nanonot supported · NoCoT-Bench no-chain-of-thought ask
50.2
193Gemma 3 12B PTnot supported · NoCoT-Bench no-chain-of-thought ask
49.9
194Qwen1.5 72B Chatnot supported · NoCoT-Bench no-chain-of-thought ask
49.7
195GPT-3.5 Turbo Instructnot supported · NoCoT-Bench no-chain-of-thought ask
49.6
196GPT-5.4 Nanodisabled · NoCoT-Bench no-chain-of-thought ask
49.4
197Hunyuan A13B Instructdisabled · NoCoT-Bench no-chain-of-thought ask
48.7
198Qwen2.5 7B Instructnot supported · NoCoT-Bench no-chain-of-thought ask
48.4
199Gemma 2 9B ITnot supported · NoCoT-Bench no-chain-of-thought ask
47.5
200Nemotron 3 Nano 30B A3Bdisabled · NoCoT-Bench no-chain-of-thought ask
47.3
201Gemma 3 12B ITnot supported · NoCoT-Bench no-chain-of-thought ask
46.8
202Ministral 3 3B Instruct 2512not supported · NoCoT-Bench no-chain-of-thought ask
46.2
203Gemma 2 9Bnot supported · NoCoT-Bench no-chain-of-thought ask
44.7
204Qwen3.5-4Bdisabled · NoCoT-Bench no-chain-of-thought ask
44.3
205Gemma 4 E2B ITdisabled · NoCoT-Bench no-chain-of-thought ask
44.0
206Mistral NeMo Base 2407not supported · NoCoT-Bench no-chain-of-thought ask
43.2
207Mistral NeMo Instruct 2407not supported · NoCoT-Bench no-chain-of-thought ask
42.5
208Llama 3.1 8B Instructnot supported · NoCoT-Bench no-chain-of-thought ask
41.6
209Llama 2 70Bnot supported · NoCoT-Bench no-chain-of-thought ask
40.7
210L3 8B Lunaris v1not supported · NoCoT-Bench no-chain-of-thought ask
40.7
211Llama 3.1 8Bnot supported · NoCoT-Bench no-chain-of-thought ask
40.6
212Llama 3 8B Instructnot supported · NoCoT-Bench no-chain-of-thought ask
39.2
213Gemma 3n E4B Instructnot supported · NoCoT-Bench no-chain-of-thought ask
38.5
214Gemma 3 4B ITnot supported · NoCoT-Bench no-chain-of-thought ask
37.9
215LLaMA 65Bnot supported · NoCoT-Bench no-chain-of-thought ask
37.5
216Aion-RP 1.0 (8B)not supported · NoCoT-Bench no-chain-of-thought ask
37.2
217Qwen3.5-2Bdisabled · NoCoT-Bench no-chain-of-thought ask
35.5
218GPT-5 Nanodisabled · NoCoT-Bench no-chain-of-thought ask
31.0
219Gemma 7B ITnot supported · NoCoT-Bench no-chain-of-thought ask
25.8
220Qwen3.5-0.8Bdisabled · NoCoT-Bench no-chain-of-thought ask
24.9
221Gemma 2 2Bnot supported · NoCoT-Bench no-chain-of-thought ask
24.3
222Gemma 2 2B ITnot supported · NoCoT-Bench no-chain-of-thought ask
24.1
223MythoMax L2 13Bnot supported · NoCoT-Bench no-chain-of-thought ask
23.8
224Llama 2 7Bnot supported · NoCoT-Bench no-chain-of-thought ask
18.7
225Gemma 3 1B ITnot supported · NoCoT-Bench no-chain-of-thought ask
16.1
226LLaMA 7Bnot supported · NoCoT-Bench no-chain-of-thought ask
15.0
227Gemma 2B ITnot supported · NoCoT-Bench no-chain-of-thought ask
11.9
228Gemma 3 270M ITnot supported · NoCoT-Bench no-chain-of-thought ask
-23.1

Related benchmarks

Loading Atlas data…