Benchmarks
326 sources · 1,256 models × 2,956 benchmarks · 154,484 scoresVisa Agentic Commerce Benchmark
The Visa Agentic Commerce Benchmark (Visa Research) tests whether an AI shopping assistant can work out what a shopper actually needs. In each of 15 episodes the assistant talks with a simulated customer who reveals requirements only when asked, and researches a fixed market with tools under limits on turns, tool calls and tokens. A run passes when the purchase meets every requirement in the answer key and each requirement came out during the session, or when the assistant rightly ends without a purchase because nothing fits. Each model runs every episode three times, and the score is its share of 45 runs passed. One episode is public and 14 are private. Visa says the two leaders' confidence intervals overlap those of the next eleven models.
480th strongest of 779 fitted benchmarks
Signal-to-noise +4.70
307th hardest of 779 fitted benchmarks
Capability needed 156.4
| # | Model | Setting | Score | Source |
|---|---|---|---|---|
| 1 | Gemini 4 Argon | not reported | 82.2 | |
| 2 | Muse Wonder | not reported | 82.2 | |
| 3 | Claude Fable 5.1 | not reported | 75.6 | |
| 4 | GLM-5.3 | not reported | 66.7 | |
| 5 | Claude Sonnet 5.5 | not reported | 64.4 | |
| 6 | GPT-6 Astra | not reported | 64.4 | |
| 7 | Muse Spark 1.3 | not reported | 64.4 | |
| 8 | Claude Opus 5.5 | not reported | 62.2 | |
| 9 | GPT-6.1 Sol | not reported | 62.2 | |
| 10 | Grok 4.7 | not reported | 60.0 | |
| 11 | Qwen3.8 Max (0902) | not reported | 60.0 | |
| 12 | DeepSeek V4.1 Flash 1017 | not reported | 57.8 | |
| 13 | Beam 1.0 501B A23B | not reported | 55.6 | |
| 14 | Gemini 3.8 Flash | not reported | 53.3 | |
| 15 | GPT-6 Luna | not reported | 53.3 | |
| 16 | Kimi K3.1 Pro | not reported | 53.3 | |
| 17 | GPT-6.1 Terra | not reported | 51.1 | |
| 18 | MiniMax M3 | not reported | 48.9 | |
| 19 | DeepSeek-V4-Pro | not reported | 42.2 | |
| 20 | Gemini 3.1 Pro Preview | not reported | 35.6 | |
| 21 | MiniMax M3.1 Flash | not reported | 33.3 | |
| 22 | Inkling | not reported | 28.9 |
