Atlas

Benchmarks

SEA-HELM Filipino Safety

The safety competency of SEA-HELM's Filipino score: the mean of toxicity detection (classifying Filipino text as clean, offensive or hateful) and the SEA-Safeguard task group (classifying content as safe or harmful under a 14-category safety policy, including culturally specific cases), scored by balanced accuracy rescaled against chance and floored at 0, from 8 runs per model with prompts in the language. Diagnostic: it measures safety classification rather than a model's own refusals, and reaches the SEA-HELM score only through the Filipino score.

Models
63
Top score
71.9
Leader
Gemma 4 31B IT
Updated
—
Resolution
—
Saturation
—

No signal or difficulty estimate. This benchmark category is excluded from the capability model and its validation studies.

Organizations
63 dated models · 4 frontier modelsClosed weightsOpen weights
Data: AI Singaporeatlas.village.dev

SEA-HELM Filipino Safety leaderboard

#ModelSettingScoreSource
1Gemma 4 31B ITdisabled · SEA-HELM leaderboard: zero-shot native-language prompts in seven Southeast Asian languages, 8 runs per model with the model's default generation settings (vLLM defaults otherwise); reasoning off (the board labels the row non-reasoning)
71.9
2Gemma 3 27B ITnot supported · SEA-HELM leaderboard: zero-shot native-language prompts in seven Southeast Asian languages, 8 runs per model with the model's default generation settings (vLLM defaults otherwise); the model has no reasoning mode
69.3
3GPT-5.5thinking · SEA-HELM leaderboard: zero-shot native-language prompts in seven Southeast Asian languages, 8 runs per model with the model's default generation settings (vLLM defaults otherwise); reasoning on (the board labels the row reasoning); effort or thinking budget unpublished
68.7
4GPT-5.6 Solthinking · SEA-HELM leaderboard: zero-shot native-language prompts in seven Southeast Asian languages, 8 runs per model with the model's default generation settings (vLLM defaults otherwise); reasoning on (the board labels the row reasoning); effort or thinking budget unpublished
67.9
5Qwen3.5 397B A17Bdisabled · SEA-HELM leaderboard: zero-shot native-language prompts in seven Southeast Asian languages, 8 runs per model with the model's default generation settings (vLLM defaults otherwise); reasoning off (the board labels the row non-reasoning)
66.6
6GPT-5.6 Lunathinking · SEA-HELM leaderboard: zero-shot native-language prompts in seven Southeast Asian languages, 8 runs per model with the model's default generation settings (vLLM defaults otherwise); reasoning on (the board labels the row reasoning); effort or thinking budget unpublished
66.5
7GPT-6 Astrathinking · SEA-HELM leaderboard: zero-shot native-language prompts in seven Southeast Asian languages, 8 runs per model with the model's default generation settings (vLLM defaults otherwise); reasoning on (the board labels the row reasoning); effort or thinking budget unpublished
66.4
8GPT-5.6 Terrathinking · SEA-HELM leaderboard: zero-shot native-language prompts in seven Southeast Asian languages, 8 runs per model with the model's default generation settings (vLLM defaults otherwise); reasoning on (the board labels the row reasoning); effort or thinking budget unpublished
66.4
9Gemma 3 12B ITnot supported · SEA-HELM leaderboard: zero-shot native-language prompts in seven Southeast Asian languages, 8 runs per model with the model's default generation settings (vLLM defaults otherwise); the model has no reasoning mode
65.9
10GPT-5.4thinking · SEA-HELM leaderboard: zero-shot native-language prompts in seven Southeast Asian languages, 8 runs per model with the model's default generation settings (vLLM defaults otherwise); reasoning on (the board labels the row reasoning); effort or thinking budget unpublished
65.4
11Sonnet 5thinking · SEA-HELM leaderboard: zero-shot native-language prompts in seven Southeast Asian languages, 8 runs per model with the model's default generation settings (vLLM defaults otherwise); reasoning on (the board labels the row reasoning); effort or thinking budget unpublished
64.0
12Haiku 4.5thinking · SEA-HELM leaderboard: zero-shot native-language prompts in seven Southeast Asian languages, 8 runs per model with the model's default generation settings (vLLM defaults otherwise); reasoning on (the board labels the row reasoning); effort or thinking budget unpublished
63.5
13Qwen3.5-27Bdisabled · SEA-HELM leaderboard: zero-shot native-language prompts in seven Southeast Asian languages, 8 runs per model with the model's default generation settings (vLLM defaults otherwise); reasoning off (the board labels the row non-reasoning)
63.5
14Muse Glimmer 30Bthinking · SEA-HELM leaderboard: zero-shot native-language prompts in seven Southeast Asian languages, 8 runs per model with the model's default generation settings (vLLM defaults otherwise); reasoning on (the board labels the row reasoning); effort or thinking budget unpublished
63.5
15Mistral Medium 3.5thinking · SEA-HELM leaderboard: zero-shot native-language prompts in seven Southeast Asian languages, 8 runs per model with the model's default generation settings (vLLM defaults otherwise); reasoning on (the board labels the row reasoning); effort or thinking budget unpublished
62.8
16Qwen3.5 35B A3Bthinking · SEA-HELM leaderboard: zero-shot native-language prompts in seven Southeast Asian languages, 8 runs per model with the model's default generation settings (vLLM defaults otherwise); reasoning on (the board labels the row reasoning); effort or thinking budget unpublished
61.9
17Opus 4.8thinking · SEA-HELM leaderboard: zero-shot native-language prompts in seven Southeast Asian languages, 8 runs per model with the model's default generation settings (vLLM defaults otherwise); reasoning on (the board labels the row reasoning); effort or thinking budget unpublished
61.6
18MiniMax M3thinking · SEA-HELM leaderboard: zero-shot native-language prompts in seven Southeast Asian languages, 8 runs per model with the model's default generation settings (vLLM defaults otherwise); reasoning on (the board labels the row reasoning); effort or thinking budget unpublished
61.4
19GPT-5.4 Minithinking · SEA-HELM leaderboard: zero-shot native-language prompts in seven Southeast Asian languages, 8 runs per model with the model's default generation settings (vLLM defaults otherwise); reasoning on (the board labels the row reasoning); effort or thinking budget unpublished
61.2
20Qwen3 VL 32B Instructnot supported · SEA-HELM leaderboard: zero-shot native-language prompts in seven Southeast Asian languages, 8 runs per model with the model's default generation settings (vLLM defaults otherwise); the model has no reasoning mode
61.0
21Qwen3.6 27Bthinking · SEA-HELM leaderboard: zero-shot native-language prompts in seven Southeast Asian languages, 8 runs per model with the model's default generation settings (vLLM defaults otherwise); reasoning on (the board labels the row reasoning); effort or thinking budget unpublished
60.8
22Qwen3-VL-235B-A22B-Instructnot supported · SEA-HELM leaderboard: zero-shot native-language prompts in seven Southeast Asian languages, 8 runs per model with the model's default generation settings (vLLM defaults otherwise); the model has no reasoning mode
60.7
23Qwen3.5 122B A10Bthinking · SEA-HELM leaderboard: zero-shot native-language prompts in seven Southeast Asian languages, 8 runs per model with the model's default generation settings (vLLM defaults otherwise); reasoning on (the board labels the row reasoning); effort or thinking budget unpublished
60.7
24Gemini 3.1 Pro Previewthinking · SEA-HELM leaderboard: zero-shot native-language prompts in seven Southeast Asian languages, 8 runs per model with the model's default generation settings (vLLM defaults otherwise); reasoning on (the board labels the row reasoning); effort or thinking budget unpublished
60.6
25Gemma 4 12B ITdisabled · SEA-HELM leaderboard: zero-shot native-language prompts in seven Southeast Asian languages, 8 runs per model with the model's default generation settings (vLLM defaults otherwise); reasoning off (the board labels the row non-reasoning)
60.3
26gpt-oss-120bthinking · SEA-HELM leaderboard: zero-shot native-language prompts in seven Southeast Asian languages, 8 runs per model with the model's default generation settings (vLLM defaults otherwise); reasoning on (the board labels the row reasoning); effort or thinking budget unpublished
60.1
27Qwen3.8-Flash-Nextthinking · SEA-HELM leaderboard: zero-shot native-language prompts in seven Southeast Asian languages, 8 runs per model with the model's default generation settings (vLLM defaults otherwise); reasoning on (the board labels the row reasoning); effort or thinking budget unpublished
59.9
28Gemma 4 26B A4B ITthinking · SEA-HELM leaderboard: zero-shot native-language prompts in seven Southeast Asian languages, 8 runs per model with the model's default generation settings (vLLM defaults otherwise); reasoning on (the board labels the row reasoning); effort or thinking budget unpublished
59.6
29Gemini 3.1 Flash-Litethinking · SEA-HELM leaderboard: zero-shot native-language prompts in seven Southeast Asian languages, 8 runs per model with the model's default generation settings (vLLM defaults otherwise); reasoning on (the board labels the row reasoning); effort or thinking budget unpublished
58.5
30Nemotron 3 Ultra 550B A55Bdisabled · SEA-HELM leaderboard: zero-shot native-language prompts in seven Southeast Asian languages, 8 runs per model with the model's default generation settings (vLLM defaults otherwise); reasoning off (the board labels the row non-reasoning)
58.5
31Qwen3 32Bthinking · SEA-HELM leaderboard: zero-shot native-language prompts in seven Southeast Asian languages, 8 runs per model with the model's default generation settings (vLLM defaults otherwise); reasoning on (the board labels the row reasoning); effort or thinking budget unpublished
57.7
32Qwen3.6 35B A3Bthinking · SEA-HELM leaderboard: zero-shot native-language prompts in seven Southeast Asian languages, 8 runs per model with the model's default generation settings (vLLM defaults otherwise); reasoning on (the board labels the row reasoning); effort or thinking budget unpublished
57.6
33Gemini 3.5 Flashthinking · SEA-HELM leaderboard: zero-shot native-language prompts in seven Southeast Asian languages, 8 runs per model with the model's default generation settings (vLLM defaults otherwise); reasoning on (the board labels the row reasoning); effort or thinking budget unpublished
57.6
34Gemini 3.6 Flashthinking · SEA-HELM leaderboard: zero-shot native-language prompts in seven Southeast Asian languages, 8 runs per model with the model's default generation settings (vLLM defaults otherwise); reasoning on (the board labels the row reasoning); effort or thinking budget unpublished
57.5
35Qwen3 14Bthinking · SEA-HELM leaderboard: zero-shot native-language prompts in seven Southeast Asian languages, 8 runs per model with the model's default generation settings (vLLM defaults otherwise); reasoning on (the board labels the row reasoning); effort or thinking budget unpublished
57.3
36Sonnet 4.6thinking · SEA-HELM leaderboard: zero-shot native-language prompts in seven Southeast Asian languages, 8 runs per model with the model's default generation settings (vLLM defaults otherwise); reasoning on (the board labels the row reasoning); effort or thinking budget unpublished
56.6
37Gemini 3.8 Flashthinking · SEA-HELM leaderboard: zero-shot native-language prompts in seven Southeast Asian languages, 8 runs per model with the model's default generation settings (vLLM defaults otherwise); reasoning on (the board labels the row reasoning); effort or thinking budget unpublished
55.9
38GLM-4.7disabled · SEA-HELM leaderboard: zero-shot native-language prompts in seven Southeast Asian languages, 8 runs per model with the model's default generation settings (vLLM defaults otherwise); reasoning off (the board labels the row non-reasoning)
55.4
39Qwen3.5-9Bthinking · SEA-HELM leaderboard: zero-shot native-language prompts in seven Southeast Asian languages, 8 runs per model with the model's default generation settings (vLLM defaults otherwise); reasoning on (the board labels the row reasoning); effort or thinking budget unpublished
55.2
40Llama 4 Maverick Instructnot supported · SEA-HELM leaderboard: zero-shot native-language prompts in seven Southeast Asian languages, 8 runs per model with the model's default generation settings (vLLM defaults otherwise); the model has no reasoning mode
53.6
41Llama 3.3 70B Instructnot supported · SEA-HELM leaderboard: zero-shot native-language prompts in seven Southeast Asian languages, 8 runs per model with the model's default generation settings (vLLM defaults otherwise); the model has no reasoning mode
53.4
42gpt-oss-20bthinking · SEA-HELM leaderboard: zero-shot native-language prompts in seven Southeast Asian languages, 8 runs per model with the model's default generation settings (vLLM defaults otherwise); reasoning on (the board labels the row reasoning); effort or thinking budget unpublished
51.6
43GLM 4.7 Flashthinking · SEA-HELM leaderboard: zero-shot native-language prompts in seven Southeast Asian languages, 8 runs per model with the model's default generation settings (vLLM defaults otherwise); reasoning on (the board labels the row reasoning); effort or thinking budget unpublished
50.6
44Qwen3 VL 4B Instructnot supported · SEA-HELM leaderboard: zero-shot native-language prompts in seven Southeast Asian languages, 8 runs per model with the model's default generation settings (vLLM defaults otherwise); the model has no reasoning mode
49.1
45Inkling-Smallthinking · SEA-HELM leaderboard: zero-shot native-language prompts in seven Southeast Asian languages, 8 runs per model with the model's default generation settings (vLLM defaults otherwise); reasoning on (the board labels the row reasoning); effort or thinking budget unpublished
49.0
46Qwen3 8Bthinking · SEA-HELM leaderboard: zero-shot native-language prompts in seven Southeast Asian languages, 8 runs per model with the model's default generation settings (vLLM defaults otherwise); reasoning on (the board labels the row reasoning); effort or thinking budget unpublished
46.9
47Gemma 4 E4B ITthinking · SEA-HELM leaderboard: zero-shot native-language prompts in seven Southeast Asian languages, 8 runs per model with the model's default generation settings (vLLM defaults otherwise); reasoning on (the board labels the row reasoning); effort or thinking budget unpublished
46.3
48Llama 4 Scout Instructnot supported · SEA-HELM leaderboard: zero-shot native-language prompts in seven Southeast Asian languages, 8 runs per model with the model's default generation settings (vLLM defaults otherwise); the model has no reasoning mode
44.8
49Qwen3.5-4Bthinking · SEA-HELM leaderboard: zero-shot native-language prompts in seven Southeast Asian languages, 8 runs per model with the model's default generation settings (vLLM defaults otherwise); reasoning on (the board labels the row reasoning); effort or thinking budget unpublished
43.7
50Gemma 4 E2B ITthinking · SEA-HELM leaderboard: zero-shot native-language prompts in seven Southeast Asian languages, 8 runs per model with the model's default generation settings (vLLM defaults otherwise); reasoning on (the board labels the row reasoning); effort or thinking budget unpublished
43.7
51Gemma 3 4B ITnot supported · SEA-HELM leaderboard: zero-shot native-language prompts in seven Southeast Asian languages, 8 runs per model with the model's default generation settings (vLLM defaults otherwise); the model has no reasoning mode
42.5
52Mistral Small 4disabled · SEA-HELM leaderboard: zero-shot native-language prompts in seven Southeast Asian languages, 8 runs per model with the model's default generation settings (vLLM defaults otherwise); reasoning off (the board labels the row non-reasoning)
42.5
53Nemotron 3 Super 120B A12Bthinking · SEA-HELM leaderboard: zero-shot native-language prompts in seven Southeast Asian languages, 8 runs per model with the model's default generation settings (vLLM defaults otherwise); reasoning on (the board labels the row reasoning); effort or thinking budget unpublished
41.9
54Qwen3-VL-8B-Instructnot supported · SEA-HELM leaderboard: zero-shot native-language prompts in seven Southeast Asian languages, 8 runs per model with the model's default generation settings (vLLM defaults otherwise); the model has no reasoning mode
40.2
55Nemotron 3.5 Lightning 30B A3B BF16thinking · SEA-HELM leaderboard: zero-shot native-language prompts in seven Southeast Asian languages, 8 runs per model with the model's default generation settings (vLLM defaults otherwise); reasoning on (the board labels the row reasoning); effort or thinking budget unpublished
36.2
56Nemotron 3 Nano 30B A3Bthinking · SEA-HELM leaderboard: zero-shot native-language prompts in seven Southeast Asian languages, 8 runs per model with the model's default generation settings (vLLM defaults otherwise); reasoning on (the board labels the row reasoning); effort or thinking budget unpublished
32.8
57Apertus 70B Instructnot supported · SEA-HELM leaderboard: zero-shot native-language prompts in seven Southeast Asian languages, 8 runs per model with the model's default generation settings (vLLM defaults otherwise); the model has no reasoning mode
32.6
58Llama 3.1 8B Instructnot supported · SEA-HELM leaderboard: zero-shot native-language prompts in seven Southeast Asian languages, 8 runs per model with the model's default generation settings (vLLM defaults otherwise); the model has no reasoning mode
32.3
59Olmo 3.1 32B Instructnot supported · SEA-HELM leaderboard: zero-shot native-language prompts in seven Southeast Asian languages, 8 runs per model with the model's default generation settings (vLLM defaults otherwise); the model has no reasoning mode
28.8
60Command A+ (May 2026, BF16)thinking · SEA-HELM leaderboard: zero-shot native-language prompts in seven Southeast Asian languages, 8 runs per model with the model's default generation settings (vLLM defaults otherwise); reasoning on (the board labels the row reasoning); effort or thinking budget unpublished
16.6
61Apertus 8B Instructnot supported · SEA-HELM leaderboard: zero-shot native-language prompts in seven Southeast Asian languages, 8 runs per model with the model's default generation settings (vLLM defaults otherwise); the model has no reasoning mode
14.8
62Llama 3.2 3B Instructnot supported · SEA-HELM leaderboard: zero-shot native-language prompts in seven Southeast Asian languages, 8 runs per model with the model's default generation settings (vLLM defaults otherwise); the model has no reasoning mode
14.4
63Tiny Aya Globalnot supported · SEA-HELM leaderboard: zero-shot native-language prompts in seven Southeast Asian languages, 8 runs per model with the model's default generation settings (vLLM defaults otherwise); the model has no reasoning mode
12.5

Related benchmarks

Loading Atlas data…