MoReBench
Chat & Writing · 2025-10-18
MoReBench, from the University of Washington, NYU, Scale AI and other co-authors, grades procedural moral reasoning on moral-dilemma scenarios, both advice to a user (Moral Advisor) and decisions an autonomous AI must take (Moral Agent). Each scenario has 20 to 49 expert-written rubric criteria weighted from -3 to +3 across identifying considerations, clear and logical process, and helpful and harmless outcome; a scenario's score is the share of its total absolute criterion weight earned (positive criteria met, negative criteria avoided) as judged by gpt-oss-120b, and the benchmark score is the mean over the 500-scenario public set the paper uses in every experiment (the other 500 of its 1,000 scenarios are held out). Scores here grade the model's final response (MoReBench-Regular, paper Table 7), with up to 10,500 output tokens and a 10,000-token thinking budget for models that require one; thinking-trace scores, the length-corrected MoReBench-Hard and the 150-scenario MoReBench-Theory are not imported.
Top models (higher is better)
| Model | Score |
|---|---|
| Qwen3 235B A22B Thinking 2507 | 77.6 |
| GPT-5 Mini | 75.6 |
| Qwen3 30B A3B Thinking 2507 | 75.4 |
| Gemini 2.5 Flash | 75.0 |
| Gemini 2.5 Pro | 74.3 |
| DeepSeek-V3.1 | 73.9 |
| GPT-5 | 73.2 |
| DeepSeek-R1-0528 | 72.9 |
| GPT-5 Nano | 72.3 |
| Gemini 2.5 Flash-Lite | 71.7 |
| gpt-oss-20b | 71.7 |
| gpt-oss-120b | 69.9 |
| Opus 4.1 | 65.5 |
| Sonnet 4 | 63.4 |