← All benchmarks

Japanese

General QA · 2024-12-18

Elo rating on Scale SEAL's Japanese multilingual prompt set.

Top models (higher is better)

Model	Score
o1 Preview	1118
Gemini Exp-1206	1117
o3-mini	1104
Gemini 1.5 Pro Experimental 0827	1093
o1 (2024-12-17)	1086
Gemini Experimental 1121	1084
GPT-4o (2024-08-06)	1077
Claude 3.5 Sonnet (June 2024)	1069
Gemini 2.0 Flash Experimental	1064
GPT-4 Turbo (1106 Preview)	978
Mistral Large 2 (Instruct 2407)	967
Gemini 1.5 Flash Preview 0514	966
Aya Expanse 32B	960
Llama 3.1 405B Instruct	895
Gemma 2 27B IT	892
Llama 3.3 70B Instruct	778
Aya-23-35B	760