LegalReasonBench
Legal
micro1's LegalReasonBench: 50 lawyer-built matters, each a sequence of two to five assignments (122 in all) that an agent works in one persistent sandbox from the matter's case file, 22 with web research and 28 confined to the uploaded record, through research memoranda and court filings. Each assignment has a signed rubric of required points and penalties (2,236 criteria), and a strict majority of three LLM judges decides each criterion. An assignment scores earned over available positive points, clamped to 0-1; a matter is the points-weighted total of its assignments, and the score is the mean over matters of the mean of three runs.
Top models (higher is better)
| Model | Score |
|---|---|
| Gemini 4 Argon | 50.8 |
| Claude Fable 5.1 | 50.6 |
| Claude Opus 5.5 | 46.5 |
| Grok 4.7 | 40.4 |
| Muse Spark 1.3 | 40.4 |
| Gemini 3.8 Flash | 39.6 |
| GPT-6 Astra | 39.5 |
| GPT-6 Sol | 31.9 |