Atlas

Benchmarks

InferenceBench

InferenceBench gives a coding agent a base model, a GPU environment and a two-hour budget to build an OpenAI-compatible inference server, and scores the final submitted server's speedup over a vanilla PyTorch baseline on four scenarios: prefill latency (time to first token), decode latency (time per output token), throughput under burst, Poisson and constant-rate traffic, and a balanced all-in-one. A server that fails the correctness checks or the integrity audit for reward hacking, is unreachable, or regresses scores at the baseline (1.0). Each scenario is run three times, and the score is the geometric mean of the four scenario speedups. This is the v1.0.1 prompt, the leaderboard's default view.

Models
18
Top score
12.1
Leader
Claude Opus 5.5
Updated
—
Resolution
—
Saturation
—

No signal or difficulty estimate. This benchmark has no declared score range, so it cannot be placed on the 0-1 scale the capability curve requires.

Organizations
18 dated models · 4 frontier modelsClosed weightsOpen weights
Data: InferenceBenchatlas.village.dev

InferenceBench leaderboard

Related benchmarks

Loading Atlas data…