Atlas

Benchmarks

ACCoRD

ACCoRD (Assessment of Consistency in Conceptual Reasoning Domains), from the Conceptual Reasoning Index maintainers, tests whether the beliefs models state about hard-to-verify conceptual claims are logically coherent. Models give probabilities for propositions and choose between options, and 567 expert-approved constraints from a 13,853-constraint dataset are checked, such as a conjunction being no more probable than either conjunct and recommendations not depending on the order options are shown. Each probability is sampled once. The score averages the model's constraint violations, normalizes them by the average distance between probabilities of random unrelated propositions so that avoiding the full probability range is not rewarded, subtracts the result from one, and rescales it so uniformly random answers score 0 and perfect consistency 100. The scored constraints are not public.

Models
142
Top score
89.4
Leader
Claude Fable 5.2
Updated
—
Resolution
~5 mo
Useful
Saturation
~Mar 2027
Projected
Signal
39thpercentile
195 benchmarks from 0.8 to 3.5224 benchmarks from 3.5 to 6.1133 benchmarks from 6.1 to 8.898 benchmarks from 8.8 to 11.566 benchmarks from 11.5 to 14.126 benchmarks from 14.1 to 16.815 benchmarks from 16.8 to 19.59 benchmarks from 19.5 to 22.18 benchmarks from 22.1 to 24.84 benchmarks from 24.8 to 27.55 benchmarks from 27.5 to 30.12 benchmarks from 30.1 to 32.84 benchmarks from 32.8 to 35.45 benchmarks from 35.4 to 38.11 benchmark from 38.1 to 40.82 benchmarks from 43.4 to 46.11 benchmark from 62.1 to 64.8

485th strongest of 798

Signal-to-noise +4.74

Difficulty
35thpercentile
1 benchmark from 5.5 to 14.41 benchmark from 14.4 to 23.22 benchmarks from 32.1 to 41.01 benchmark from 41.0 to 49.82 benchmarks from 49.8 to 58.79 benchmarks from 58.7 to 67.67 benchmarks from 67.6 to 76.418 benchmarks from 76.4 to 85.324 benchmarks from 85.3 to 94.241 benchmarks from 94.2 to 103.031 benchmarks from 103.0 to 111.939 benchmarks from 111.9 to 120.850 benchmarks from 120.8 to 129.676 benchmarks from 129.6 to 138.560 benchmarks from 138.5 to 147.4118 benchmarks from 147.4 to 156.2124 benchmarks from 156.2 to 165.1120 benchmarks from 165.1 to 174.039 benchmarks from 174.0 to 182.820 benchmarks from 182.8 to 191.710 benchmarks from 191.7 to 200.61 benchmark from 200.6 to 209.44 benchmarks from 209.4 to 218.3

518th hardest of 798

Capability needed 136.3

Organizations
137 dated models · 14 frontier modelsClosed weightsOpen weights
Data: Conceptual Reasoning Benchmark Authorsatlas.village.dev

ACCoRD leaderboard

#ModelSettingScoreSource
1Claude Fable 5.2x-high
89.4
2Claude Opus 5.5high
88.4
3Claude Sonnet 5.5x-high
85.7
4Claude Fable 5max
85.0
5Claude Opus 5low
83.8
6Claude Fable 5.1medium
83.7
7GPT-6 Solmax
82.4
8GPT-6 Astramax
81.7
9Claude Haiku 5.5max
81.5
10GPT-6.1 Solmax
80.9
11GPT-5.5x-high
80.5
12Opus 4.6max
80.4
13GPT-5.4x-high
80.0
14Opus 4.7max
79.8
15GPT-5.2x-high
79.3
16Opus 4.5high
79.1
17Muse Spark 1.1thinking
78.4
18Gemini 3.8 Flashmedium
78.2
19Kimi K3.1 Promax
77.5
20Opus 4.8max
77.4
21Muse Spark 1.2x-high
77.4
22Gemini 4 Argonlow
77.3
23Gemini 3.7 Flashhigh
77.0
24Sonnet 5x-high
76.9
25Muse Wondermax
76.2
26GPT-5.6 Solmax
75.9
27GPT-5.5 Prox-high
75.8
28Muse Spark 1.3max
75.1
29GPT-5.6 Terramax
74.7
30Grok 4.7x-high
74.0
31o3-prohigh
73.8
32Gemini 3.1 Pro Previewmedium
73.5
33Sonnet 4.6max
73.4
34Qwen3.8 27Bx-high
73.3
35GPT-5high
73.1
36GPT-6 Lunamax
72.1
37Muse Glimmer 30Bx-high
72.0
38Qwen3.8 2.4T A95Bx-high
71.6
39Grok 4.5high
71.6
40Grok 4.6x-high
71.6
41GPT-5 Minihigh
71.5
42Qwen3.8-Maxthinking
71.4
43Kimi K2.6thinking
71.4
44o3high
71.4
45GPT-5.1high
71.2
46DeepSeek-V4-Promax
70.9
47GLM-5.3max
70.9
48Qwen3.6-Max-Previewthinking
70.8
49Sonnet 4.5thinking
70.7
50GPT-5.4 Minix-high
70.5
51Kimi K3max
70.5
52DeepSeek V4 Pro 0813max
70.2
53Gemini 3.6 Flashhigh
70.0
54Grok 4.20thinking
68.8
55Gemini 3 Flash Previewhigh
68.6
56DeepSeek V4.1 Flashmax
68.1
57Gemini 3.5 Flash-Litehigh
68.0
58gpt-oss-120bhigh
67.5
59GPT-5.6 Lunahigh
67.5
60GPT-5.4 Nanox-high
67.3
61Haiku 4.5thinking
66.7
62Gemini 3.5 Flashhigh
66.6
63Hy3high
66.4
64GLM-5.2max
66.0
65GPT-5.2 Instant (source unspecified)not supported
66.0
66MiniMax M3thinking
65.9
67Opus 4thinking
65.8
68Mistral Large 4 Previewhigh
65.7
69Qwen3.7-Max (source unspecified)thinking
65.3
70DeepSeek-V3.2thinking
64.6
71o1 (2024-12-17)high
64.2
72o4-minihigh
63.8
73Opus 4.1thinking
63.5
74Gemini 2.5 Prothinking
62.7
75Gemma 4 31B ITthinking
62.0
76gpt-oss-20bhigh
61.9
77DeepSeek-V3.2-Expthinking
61.8
78DeepSeek-V4-Flash-0731max
61.7
79GPT-5.3 Instant (source unspecified)not supported
61.1
80Qwen3.7-Plus (2026-05-26)thinking
60.7
81Qwen3.5-27Bthinking
60.5
82Qwen3-Next-80B-A3B-Thinkingalways on
59.6
83Qwen3.6 Plus (2026-04-02)thinking
59.3
84Qwen3.6 27Bthinking
58.8
85GPT-5 Nanohigh
58.8
86Qwen3 235B A22B Instruct 2507not supported
58.7
87MiMo-V2.5-Prothinking
58.2
88Qwen3.5 122B A10Bthinking
58.0
89DeepSeek-V4-Flashmax
57.3
90Qwen3 235B A22B Thinking 2507always on
57.1
91Qwen3 30B A3B Thinking 2507always on
57.0
92Sonnet 4thinking
57.0
93o3-minihigh
56.5
94GPT-4.1not supported
56.0
95Qwen3.5 397B A17Bthinking
55.6
96DeepSeek-V3.1thinking
55.5
97Qwen3.5 35B A3Bthinking
55.3
98DeepSeek-V3.1-Terminusthinking
55.2
99Nemotron 3.5 Lightning 30B A3B (precision unspecified)thinking
54.7
100Gemma 4 26B A4B ITthinking
54.3
101Qwen3.6 35B A3Bthinking
53.0
102Qwen3.5-Flash (source unspecified)thinking
52.5
103Qwen3 Next 80B A3B Instructnot supported
52.0
104Qwen3.5-9Bthinking
50.2
105Llama 4 Maverick Instructnot supported
50.2
106Llama 4 Scout Instructnot supported
49.7
107Mistral Large 3 675B Instruct 2512not supported
48.7
108Qwen3 14Bthinking
48.4
109Qwen3 32Bthinking
48.2
110Qwen3-235B-A22Bthinking
46.8
111Mistral Medium 3.5high
45.6
112Llama 3.3 70B Instructnot supported
45.6
113Qwen3 30B A3B Instruct 2507not supported
45.4
114Gemini 2.5 Flashthinking
45.3
115Qwen3-30B-A3Bthinking
45.0
116GPT-4o (source unspecified)not supported
44.8
117GPT-4.1 Mininot supported
43.6
118DeepSeek-V3-0324not supported
43.4
119Llama 3.1 70B Instructnot supported
43.2
120Qwen2.5 72B Instructnot supported
42.5
121GPT-4 Turbonot supported
42.2
122Qwen3 8Bthinking
41.8
123DeepSeek-V3not supported
41.6
124Gemma 3n E4B Instructnot supported
41.2
125Gemma 3 27B ITnot supported
40.5
126Mistral Medium 3.1not supported
39.2
127Mistral Small 4high
39.0
128Gemma 3 12B ITnot supported
38.3
129Command Anot supported
37.3
130Llama 3.1 8B Instructnot supported
33.7
131Ministral 3 8B Instruct 2512not supported
33.5
132Opus 3not supported
33.1
133Ministral 3 14B Instruct 2512not supported
32.8
134GPT-4o Mininot supported
32.6
135Qwen2.5 7B Instructnot supported
27.3
136GPT-4.1 Nanonot supported
24.3
137Gemma 3 4B ITnot supported
23.8
138C4AI Command R+ 08-2024not supported
18.0
139Claude 3 Haikunot supported
14.6
140Ministral 3 3B Instruct 2512not supported
14.6
141C4AI Command R 08-2024not supported
13.0
142Command R7B (Dec 2024)not supported
9.9

Related benchmarks

Loading Atlas data…