Atlas

Benchmarks

← All benchmarks

Adversarial Robustness

Safety · 2024-07-29

Violation rate on adversarial safety prompts covering harmful, hateful, and illegal requests.

Top models (lower is better)

ModelScore
Gemini 1.5 Pro Preview 05140.8
Llama 3.1 405B Instruct1.0
Opus 31.3
Gemini 1.5 Flash Preview 05141.4
Claude 3.5 Sonnet (June 2024)1.6
GPT-4 0125 Preview2.0
Mistral Large 1.03.7
GPT-4o (2024-05-13)6.7
Loading Atlas data…