Atlas

Benchmarks

← All benchmarks

IOI (Vals v2)

Code

Vals IOI v2 evaluates eighteen official IOI problems from 2024, 2025 and 2026 using OpenCode in an offline C++20 sandbox. Agents receive no submission or grading feedback during their attempts; only the final solution is graded. The score averages the three six-problem yearly scores. This protocol is explicitly incomparable with the earlier interactive-grading IOI board.

Top models (higher is better)

ModelScore
Gemini 4 Argon100.0
GPT-6 Astra100.0
GPT-6.1 Sol96.9
GPT-5.6 Sol91.2
GPT-5.6 Terra87.6
Claude Opus 584.3
Claude Sonnet 5.583.1
GPT-6 Sol82.6
Kimi K3.1 Pro81.9
Muse Wonder Full79.8
Qwen3.8-Max68.9
GLM-5.368.4
Gemini 3.7 Flash67.8
Muse Wonder66.9
GPT-5.6 Luna61.8
Hy4 Preview59.3
Grok 4.757.7
Gemini 3.8 Flash56.9
Muse Spark 1.356.6
GPT-6 Luna55.6
GPT-5.3-Codex53.8
GLM-5.3 Flash52.5
Gemini 3.1 Pro Preview51.8
DeepSeek V4 Pro 081351.6
Kimi K348.9
MiMo-V2.6-Flash47.7
Grok 4.647.6
Claude Haiku 5.547.3
Ember-146.3
Mistral Large 4 Preview45.3
Sonnet 545.0
Grok 4.540.6
DeepSeek V4.1 Flash40.3
MiMo-V2.6-Pro39.3
Qwen3.8 27B39.1
Step 5 Preview38.1
Gemini 3.6 Flash35.1
DeepSeek-V4-Flash-073132.7
Beam 1.0 501B A23B22.8
Muse Spark 1.221.8
Loading Atlas data…