Benchmarks
347 sources · 1,258 models × 3,038 benchmarks · 158,912 scoresSingularity-Bench (CMU Infini-AI Lab and others) asks a coding agent to build six AI-development systems from a minimal starting codebase: a pre-training run, a mid-training data mixture, post-training, an inference engine, an agent harness and a serverless inference-cluster orchestrator. Each system is scored on identical hardware against a mature human-built reference such as OLMo 2, and the Singularity-Index is the mean over the six tasks of the agent's score divided by the reference's (inverted for perplexity), so 1.0 matches the human references on average. A failed task scores 0 and ratios are not capped. Runs use the Breakwater extension of Harbor to block reward hacking.
No signal or difficulty estimate. This benchmark has no declared score range, so it cannot be placed on the 0-1 scale the capability curve requires.
| # | Model | Setting | Score | Source |
|---|---|---|---|---|
| 1 | Claude Opus 5.5 | max · Claude Code | 1.3 | |
| 2 | GPT-6.1 Sol | max · Codex | 1.0 | |
| 3 | Claude Opus 5 | not reported · Claude Code | 0.8 | |
| 4 | GLM-5.3 | not reported · Claude Code | 0.7 | |
| 5 | MiniMax M3 | not reported · Claude Code | 0.6 | |
| 6 | Kimi K3 | not reported · Claude Code | 0.6 |