Atlas

Benchmarks

NoCoT-Bench NCKI (kspine v3)

The knowledge half of Neel Nanda's NoCoT-Bench: NCKI is a Rasch ability over the kspine_v3 knowledge rungs, built from seven long-tail knowledge bank files (1,508 scored items on obscure Wikipedia subjects, arXiv first authors, Python-ecosystem internals, scientific constants and US case law) that models must recall with no chain of thought, under the same witnessed recipes as NCRI. The authors re-anchored its gauge (106.789 + 14.161401·θ) so that the sealed models' mean and spread match the previous spine's, which ties the scale to that cohort of models; it is unbounded and diagnostic.

Models
228
Top score
131
Leader
GPT-6 Astra
Updated
—
Resolution
—
Saturation
—

No signal or difficulty estimate. Held out of the fit by design.

Organizations
226 dated models · 14 frontier modelsClosed weightsOpen weights
Data: Neel Nandaatlas.village.dev

NoCoT-Bench NCKI (kspine v3) leaderboard

#ModelSettingScoreSource
1GPT-6 Astradisabled · NoCoT-Bench no-chain-of-thought ask
131
2GPT-6.1 Soldisabled · NoCoT-Bench no-chain-of-thought ask
130
3Gemini 4 Argondisabled · NoCoT-Bench no-chain-of-thought ask
128
4Claude Fable 5.1disabled · NoCoT-Bench no-chain-of-thought ask
128
5Claude Fable 5disabled · NoCoT-Bench no-chain-of-thought ask
128
6Gemini 3.1 Pro Previewdisabled · NoCoT-Bench no-chain-of-thought ask
121
7Gemini 3.7 Flashdisabled · NoCoT-Bench no-chain-of-thought ask
121
8Gemini 3.8 Flashdisabled · NoCoT-Bench no-chain-of-thought ask
121
9GPT-5.6 Soldisabled · NoCoT-Bench no-chain-of-thought ask
121
10Claude Opus 5disabled · NoCoT-Bench no-chain-of-thought ask
119
11GPT-6 Soldisabled · NoCoT-Bench no-chain-of-thought ask
119
12Opus 4.7disabled · NoCoT-Bench no-chain-of-thought ask
118
13Opus 4.8disabled · NoCoT-Bench no-chain-of-thought ask
117
14Kimi K3.1 Prodisabled · NoCoT-Bench no-chain-of-thought ask
116
15Gemini 3.5 Flashdisabled · NoCoT-Bench no-chain-of-thought ask
116
16Gemini 3 Flash Previewdisabled · NoCoT-Bench no-chain-of-thought ask
115
17Gemini 3.6 Flashdisabled · NoCoT-Bench no-chain-of-thought ask
115
18DeepSeek V4 Pro 0813disabled · NoCoT-Bench no-chain-of-thought ask
114
19Muse Spark 1.2disabled · NoCoT-Bench no-chain-of-thought ask
114
20Muse Spark 1.1disabled · NoCoT-Bench no-chain-of-thought ask
113
21Muse Spark 1.3disabled · NoCoT-Bench no-chain-of-thought ask
113
22Opus 4.5disabled · NoCoT-Bench no-chain-of-thought ask
112
23Opus 4.1disabled · NoCoT-Bench no-chain-of-thought ask
112
24Grok 4.6disabled · NoCoT-Bench no-chain-of-thought ask
112
25Muse Spark Opendisabled · NoCoT-Bench no-chain-of-thought ask
111
26Hy4disabled · NoCoT-Bench no-chain-of-thought ask
111
27Opus 4disabled · NoCoT-Bench no-chain-of-thought ask
111
28GPT-5.5disabled · NoCoT-Bench no-chain-of-thought ask
111
29Kimi K3disabled · NoCoT-Bench no-chain-of-thought ask
110
30GPT-6.1 Terradisabled · NoCoT-Bench no-chain-of-thought ask
110
31Muse Wonderdisabled · NoCoT-Bench no-chain-of-thought ask
109
32Opus 4.6disabled · NoCoT-Bench no-chain-of-thought ask
109
33GPT-5.4disabled · NoCoT-Bench no-chain-of-thought ask
108
34Grok 4.5disabled · NoCoT-Bench no-chain-of-thought ask
107
35Claude Sonnet 5.5disabled · NoCoT-Bench no-chain-of-thought ask
107
36Hermes 3 Llama 3.1 405Bnot supported · NoCoT-Bench no-chain-of-thought ask
106
37Gemini 2.5 Prodisabled · NoCoT-Bench no-chain-of-thought ask
105
38Hy4 Previewdisabled · NoCoT-Bench no-chain-of-thought ask
104
39DeepSeek-V4-Flash-Basenot supported · NoCoT-Bench no-chain-of-thought ask
104
40GPT-5.6 Terradisabled · NoCoT-Bench no-chain-of-thought ask
104
41Hermes 4 405Bdisabled · NoCoT-Bench no-chain-of-thought ask
103
42Grok 4.3disabled · NoCoT-Bench no-chain-of-thought ask
103
43Sonnet 4.5disabled · NoCoT-Bench no-chain-of-thought ask
103
44GPT-4.1not supported · NoCoT-Bench no-chain-of-thought ask
103
45Sonnet 4.6disabled · NoCoT-Bench no-chain-of-thought ask
102
46DeepSeek V4.1 Flash 1017disabled · NoCoT-Bench no-chain-of-thought ask
102
47GPT-5.1disabled · NoCoT-Bench no-chain-of-thought ask
101
48Sonnet 5disabled · NoCoT-Bench no-chain-of-thought ask
101
49LongCat-2.0disabled · NoCoT-Bench no-chain-of-thought ask
101
50GPT-5.2disabled · NoCoT-Bench no-chain-of-thought ask
99.9
51Sonnet 4disabled · NoCoT-Bench no-chain-of-thought ask
99.4
52Inklingdisabled · NoCoT-Bench no-chain-of-thought ask
99.1
53Kimi K2.5disabled · NoCoT-Bench no-chain-of-thought ask
98.7
54Gemini 3.5 Flash-Litedisabled · NoCoT-Bench no-chain-of-thought ask
98.6
55GPT-4o (2024-05-13)not supported · NoCoT-Bench no-chain-of-thought ask
97.6
56DeepSeek-V3.1-Terminusdisabled · NoCoT-Bench no-chain-of-thought ask
97.6
57GPT-4o (2024-11-20)not supported · NoCoT-Bench no-chain-of-thought ask
97.6
58Qwen3.6-Max-Previewdisabled · NoCoT-Bench no-chain-of-thought ask
97.4
59DeepSeek-V3.2-Expdisabled · NoCoT-Bench no-chain-of-thought ask
97.3
60GPT-5disabled · NoCoT-Bench no-chain-of-thought ask
97.2
61DeepSeek-V3.1disabled · NoCoT-Bench no-chain-of-thought ask
97.1
62Qwen3.8 2.4T A95Bdisabled · NoCoT-Bench no-chain-of-thought ask
96.8
63GPT-4o (2024-08-06)not supported · NoCoT-Bench no-chain-of-thought ask
96.6
64MiMo-V2.5-Prodisabled · NoCoT-Bench no-chain-of-thought ask
96.5
65Grok 4.20disabled · NoCoT-Bench no-chain-of-thought ask
96.0
66WizardLM-2 8x22Bnot supported · NoCoT-Bench no-chain-of-thought ask
96.0
67Kimi K2 Instructnot supported · NoCoT-Bench no-chain-of-thought ask
95.9
68GPT-5.3-Codexdisabled · NoCoT-Bench no-chain-of-thought ask
95.8
69Nemotron 3 Ultra 550B A55Bdisabled · NoCoT-Bench no-chain-of-thought ask
95.3
70Llama 3.1 70Bnot supported · NoCoT-Bench no-chain-of-thought ask
95.2
71Beam 1.0 501B A23Bdisabled · NoCoT-Bench no-chain-of-thought ask
95.2
72Gemini 2.5 Flashdisabled · NoCoT-Bench no-chain-of-thought ask
95.1
73DeepSeek-V3.2disabled · NoCoT-Bench no-chain-of-thought ask
95.0
74Kimi K2.6disabled · NoCoT-Bench no-chain-of-thought ask
94.7
75Hy3 previewdisabled · NoCoT-Bench no-chain-of-thought ask
94.6
76GLM-5.2disabled · NoCoT-Bench no-chain-of-thought ask
94.3
77Llama 4 Maverick Instructnot supported · NoCoT-Bench no-chain-of-thought ask
94.2
78Qwen3 Max Thinking (2026-01-23)disabled · NoCoT-Bench no-chain-of-thought ask
94.2
79DeepSeek-V3-0324not supported · NoCoT-Bench no-chain-of-thought ask
94.0
80GPT-6 Lunadisabled · NoCoT-Bench no-chain-of-thought ask
93.8
81GLM-5disabled · NoCoT-Bench no-chain-of-thought ask
93.3
82Hy3disabled · NoCoT-Bench no-chain-of-thought ask
93.2
83DeepSeek-V3not supported · NoCoT-Bench no-chain-of-thought ask
93.1
84DeepSeek-V4-Flash-0731disabled · NoCoT-Bench no-chain-of-thought ask
92.7
85GPT-4.1 Mininot supported · NoCoT-Bench no-chain-of-thought ask
92.7
86GLM-5.3 Flashdisabled · NoCoT-Bench no-chain-of-thought ask
92.4
87GPT-4 Turbonot supported · NoCoT-Bench no-chain-of-thought ask
92.4
88GLM-5.3disabled · NoCoT-Bench no-chain-of-thought ask
92.4
89Qwen3.5 397B A17Bdisabled · NoCoT-Bench no-chain-of-thought ask
91.7
90Qwen3.6 Plus (2026-04-02)disabled · NoCoT-Bench no-chain-of-thought ask
91.7
91GLM-4.5disabled · NoCoT-Bench no-chain-of-thought ask
91.6
92Mercury 3disabled · NoCoT-Bench no-chain-of-thought ask
91.5
93Qwen3.5 Plus (2026-02-15)disabled · NoCoT-Bench no-chain-of-thought ask
91.1
94GPT-4 0613not supported · NoCoT-Bench no-chain-of-thought ask
90.4
95Ling 2.6 1Tnot supported · NoCoT-Bench no-chain-of-thought ask
89.7
96ERNIE 4.5 VL 424B A47Bdisabled · NoCoT-Bench no-chain-of-thought ask
89.6
97GLM-5.1disabled · NoCoT-Bench no-chain-of-thought ask
89.0
98GLM-4.7disabled · NoCoT-Bench no-chain-of-thought ask
88.8
99Mistral Small 4disabled · NoCoT-Bench no-chain-of-thought ask
88.0
100MiniMax M3disabled · NoCoT-Bench no-chain-of-thought ask
87.9
101GPT-5.6 Lunadisabled · NoCoT-Bench no-chain-of-thought ask
87.7
102KAT-Coder-Pro V2not supported · NoCoT-Bench no-chain-of-thought ask
87.2
103L3.3-70B-Euryale-v2.3not supported · NoCoT-Bench no-chain-of-thought ask
87.2
104Claude Haiku 5.5disabled · NoCoT-Bench no-chain-of-thought ask
85.9
105Hermes 4 70Bdisabled · NoCoT-Bench no-chain-of-thought ask
85.8
106Llama 3.3 70B Instructnot supported · NoCoT-Bench no-chain-of-thought ask
85.8
107GLM 4.6disabled · NoCoT-Bench no-chain-of-thought ask
85.8
108Haiku 4.5disabled · NoCoT-Bench no-chain-of-thought ask
84.7
109Llama 3.1 70B Instructnot supported · NoCoT-Bench no-chain-of-thought ask
84.6
110Mistral Medium 3.5disabled · NoCoT-Bench no-chain-of-thought ask
83.8
111Qwen3 235B A22B Instruct 2507not supported · NoCoT-Bench no-chain-of-thought ask
83.4
112Qwen3-VL-235B-A22B-Instructnot supported · NoCoT-Bench no-chain-of-thought ask
83.3
113Ling 3.0 Flashdisabled · NoCoT-Bench no-chain-of-thought ask
83.2
114MiMo-V2.5disabled · NoCoT-Bench no-chain-of-thought ask
82.8
115Claude 3 Haikunot supported · NoCoT-Bench no-chain-of-thought ask
82.3
116GPT-5.4 Minidisabled · NoCoT-Bench no-chain-of-thought ask
82.2
117Qwen3-Coder-480B-A35B-Instructnot supported · NoCoT-Bench no-chain-of-thought ask
80.0
118Qwen3.8 Flashdisabled · NoCoT-Bench no-chain-of-thought ask
79.8
119MiniMax M2.5always on · NoCoT-Bench no-chain-of-thought ask
79.4
120Qwen3.5 Plus 2026-04-20disabled · NoCoT-Bench no-chain-of-thought ask
79.0
121Qwen3-235B-A22Bdisabled · NoCoT-Bench no-chain-of-thought ask
78.7
122Nemotron 3 Super 120B A12Bdisabled · NoCoT-Bench no-chain-of-thought ask
78.3
123Qwen3.5 122B A10Bdisabled · NoCoT-Bench no-chain-of-thought ask
78.1
124GLM-4.5Vdisabled · NoCoT-Bench no-chain-of-thought ask
77.9
125GPT-3.5 Turbo 0125not supported · NoCoT-Bench no-chain-of-thought ask
77.7
126Gemma 4 31Bnot supported · NoCoT-Bench no-chain-of-thought ask
77.7
127Qwen2.5 72Bnot supported · NoCoT-Bench no-chain-of-thought ask
77.5
128Mistral Small 3 24B Base 2501not supported · NoCoT-Bench no-chain-of-thought ask
77.2
129GLM-4.6Vdisabled · NoCoT-Bench no-chain-of-thought ask
76.8
130Qwen3 235B A22B Thinking 2507always on · NoCoT-Bench no-chain-of-thought ask
76.8
131Qwen3 Next 80B A3B Instructnot supported · NoCoT-Bench no-chain-of-thought ask
76.7
132Qwen3-Next-80B-A3B-Thinkingalways on · NoCoT-Bench no-chain-of-thought ask
76.5
133Muse Glimmer 30Bdisabled · NoCoT-Bench no-chain-of-thought ask
76.2
134GLM 4.5 Airdisabled · NoCoT-Bench no-chain-of-thought ask
76.0
135Qwen2.5 72B Instructnot supported · NoCoT-Bench no-chain-of-thought ask
75.9
136Ling 2.6 Flashnot supported · NoCoT-Bench no-chain-of-thought ask
75.8
137MiniMax M2.7always on · NoCoT-Bench no-chain-of-thought ask
75.6
138Nova Pronot supported · NoCoT-Bench no-chain-of-thought ask
75.0
139Qwen2.5-VL-72B-Instructnot supported · NoCoT-Bench no-chain-of-thought ask
74.6
140Gemini 2.5 Flash-Litedisabled · NoCoT-Bench no-chain-of-thought ask
74.6
141gpt-oss-120bdisabled · NoCoT-Bench no-chain-of-thought ask
73.7
142Qwen1.5 110B Chatnot supported · NoCoT-Bench no-chain-of-thought ask
73.2
143Mistral Small 3 24B Instruct 2501not supported · NoCoT-Bench no-chain-of-thought ask
73.2
144Gemma 3 27B PTnot supported · NoCoT-Bench no-chain-of-thought ask
73.0
145Solar Pro 4disabled · NoCoT-Bench no-chain-of-thought ask
73.0
146GPT-5 Minidisabled · NoCoT-Bench no-chain-of-thought ask
72.4
147Llama 2 70Bnot supported · NoCoT-Bench no-chain-of-thought ask
71.3
148Gemma 2 27Bnot supported · NoCoT-Bench no-chain-of-thought ask
71.1
149Qwen3.6 27Bdisabled · NoCoT-Bench no-chain-of-thought ask
70.3
150Qwen3-Coder-Nextnot supported · NoCoT-Bench no-chain-of-thought ask
70.2
151Mistral Small 3.2 24B Instruct 2506not supported · NoCoT-Bench no-chain-of-thought ask
70.1
152GPT-4.1 Nanonot supported · NoCoT-Bench no-chain-of-thought ask
70.1
153GPT-4o Mininot supported · NoCoT-Bench no-chain-of-thought ask
70.0
154Qwen3.5-27Bdisabled · NoCoT-Bench no-chain-of-thought ask
69.8
155Cydonia 24B v4.1not supported · NoCoT-Bench no-chain-of-thought ask
69.6
156Qwen2 72B Instructnot supported · NoCoT-Bench no-chain-of-thought ask
69.4
157Qwen3.6 35B A3Bdisabled · NoCoT-Bench no-chain-of-thought ask
68.7
158Llama 4 Scout Instructnot supported · NoCoT-Bench no-chain-of-thought ask
68.3
159GPT-3.5 Turbo Instructnot supported · NoCoT-Bench no-chain-of-thought ask
68.2
160LLaMA 65Bnot supported · NoCoT-Bench no-chain-of-thought ask
68.0
161Nemotron 3.5 Lightning 30B A3B (precision unspecified)disabled · NoCoT-Bench no-chain-of-thought ask
67.7
162Qwen2.5 32Bnot supported · NoCoT-Bench no-chain-of-thought ask
67.6
163Qwen3.5 35B A3Bdisabled · NoCoT-Bench no-chain-of-thought ask
67.6
164Qwen3.7 Flashdisabled · NoCoT-Bench no-chain-of-thought ask
67.3
165GLM 4.7 Flashdisabled · NoCoT-Bench no-chain-of-thought ask
66.8
166Qwen3.5-Flash (2026-02-23)disabled · NoCoT-Bench no-chain-of-thought ask
66.8
167Gemma 3 12B PTnot supported · NoCoT-Bench no-chain-of-thought ask
66.8
168Laguna S 2.1disabled · NoCoT-Bench no-chain-of-thought ask
66.7
169Qwen3 VL 32B Instructnot supported · NoCoT-Bench no-chain-of-thought ask
66.6
170Qwen2.5 Instruct 32Bnot supported · NoCoT-Bench no-chain-of-thought ask
66.6
171GPT-5.4 Nanodisabled · NoCoT-Bench no-chain-of-thought ask
66.5
172Llama 3.1 8Bnot supported · NoCoT-Bench no-chain-of-thought ask
66.4
173Qwen1.5 72B Chatnot supported · NoCoT-Bench no-chain-of-thought ask
66.4
174Qwen3.8 27Bdisabled · NoCoT-Bench no-chain-of-thought ask
66.3
175Mercury 2disabled · NoCoT-Bench no-chain-of-thought ask
66.2
176Mistral NeMo Base 2407not supported · NoCoT-Bench no-chain-of-thought ask
66.1
177MiniMax M3.1 Flashdisabled · NoCoT-Bench no-chain-of-thought ask
65.9
178Qwen3 30B A3B Instruct 2507not supported · NoCoT-Bench no-chain-of-thought ask
65.7
179Qwen3 VL 30B A3B Instructnot supported · NoCoT-Bench no-chain-of-thought ask
65.4
180Ripple-65B Previewunknown · NoCoT-Bench no-chain-of-thought ask
65.1
181Phi-4not supported · NoCoT-Bench no-chain-of-thought ask
64.5
182Llama 3.1 8B Instructnot supported · NoCoT-Bench no-chain-of-thought ask
64.1
183Ministral 3 14B Instruct 2512not supported · NoCoT-Bench no-chain-of-thought ask
64.1
184Qwen2.5 Coder 32B Instructnot supported · NoCoT-Bench no-chain-of-thought ask
63.5
185Qwen3 32Bdisabled · NoCoT-Bench no-chain-of-thought ask
63.3
186Gemma 2 27B ITnot supported · NoCoT-Bench no-chain-of-thought ask
63.2
187Gemma 3 27B ITnot supported · NoCoT-Bench no-chain-of-thought ask
62.6
188Aion-RP 1.0 (8B)not supported · NoCoT-Bench no-chain-of-thought ask
62.3
189Qwen3 Coder 30B A3B Instructnot supported · NoCoT-Bench no-chain-of-thought ask
61.9
190Llama 3 8B Instructnot supported · NoCoT-Bench no-chain-of-thought ask
61.7
191Gemma 4 31B ITdisabled · NoCoT-Bench no-chain-of-thought ask
61.6
192Mistral NeMo Instruct 2407not supported · NoCoT-Bench no-chain-of-thought ask
61.5
193Laguna XS 2.1disabled · NoCoT-Bench no-chain-of-thought ask
61.0
194Gemma 2 9Bnot supported · NoCoT-Bench no-chain-of-thought ask
60.3
195gpt-oss-20bdisabled · NoCoT-Bench no-chain-of-thought ask
60.2
196Qwen3-30B-A3Bdisabled · NoCoT-Bench no-chain-of-thought ask
60.0
197L3 8B Lunaris v1not supported · NoCoT-Bench no-chain-of-thought ask
59.8
198Gemma 4 26B A4B ITdisabled · NoCoT-Bench no-chain-of-thought ask
59.3
199Ministral 3 8B Instruct 2512not supported · NoCoT-Bench no-chain-of-thought ask
58.3
200Olmo 3.1 32B Instructnot supported · NoCoT-Bench no-chain-of-thought ask
57.5
201Qwen3.5-9Bdisabled · NoCoT-Bench no-chain-of-thought ask
56.6
202Gemma 2 9B ITnot supported · NoCoT-Bench no-chain-of-thought ask
56.0
203Qwen3-VL-8B-Instructnot supported · NoCoT-Bench no-chain-of-thought ask
55.9
204Qwen3 14Bdisabled · NoCoT-Bench no-chain-of-thought ask
54.7
205Qwen3 8Bdisabled · NoCoT-Bench no-chain-of-thought ask
53.8
206Nemotron 3 Nano 30B A3Bdisabled · NoCoT-Bench no-chain-of-thought ask
53.5
207Qwen3.5-4Bdisabled · NoCoT-Bench no-chain-of-thought ask
51.3
208Qwen2.5 7B Instructnot supported · NoCoT-Bench no-chain-of-thought ask
51.0
209MythoMax L2 13Bnot supported · NoCoT-Bench no-chain-of-thought ask
50.1
210Gemma 4 12B ITdisabled · NoCoT-Bench no-chain-of-thought ask
49.8
211Gemma 3 12B ITnot supported · NoCoT-Bench no-chain-of-thought ask
49.6
212GPT-5 Nanodisabled · NoCoT-Bench no-chain-of-thought ask
49.2
213LLaMA 7Bnot supported · NoCoT-Bench no-chain-of-thought ask
47.3
214Llama 2 7Bnot supported · NoCoT-Bench no-chain-of-thought ask
46.5
215Ministral 3 3B Instruct 2512not supported · NoCoT-Bench no-chain-of-thought ask
46.2
216Gemma 4 E4B ITdisabled · NoCoT-Bench no-chain-of-thought ask
45.8
217Hunyuan A13B Instructdisabled · NoCoT-Bench no-chain-of-thought ask
43.6
218Gemma 3 4B ITnot supported · NoCoT-Bench no-chain-of-thought ask
39.9
219Gemma 2 2Bnot supported · NoCoT-Bench no-chain-of-thought ask
39.1
220Gemma 3n E4B Instructnot supported · NoCoT-Bench no-chain-of-thought ask
37.6
221Gemma 2 2B ITnot supported · NoCoT-Bench no-chain-of-thought ask
36.9
222Gemma 4 E2B ITdisabled · NoCoT-Bench no-chain-of-thought ask
36.4
223Gemma 7B ITnot supported · NoCoT-Bench no-chain-of-thought ask
35.0
224Qwen3.5-2Bdisabled · NoCoT-Bench no-chain-of-thought ask
31.3
225Qwen3.5-0.8Bdisabled · NoCoT-Bench no-chain-of-thought ask
8.7
226Gemma 2B ITnot supported · NoCoT-Bench no-chain-of-thought ask
1.0
227Gemma 3 1B ITnot supported · NoCoT-Bench no-chain-of-thought ask
-4.5
228Gemma 3 270M ITnot supported · NoCoT-Bench no-chain-of-thought ask
-13.3

Related benchmarks

Loading Atlas data…