Atlas

Models

← All models

Gemma 1.1 7B IT

Google · 2024-04-05 · 8.5B parameters

Gemma 1.1 7B IT is Google's updated instruction-tuned 7B Gemma checkpoint. Google says it uses a novel RLHF method to improve quality, coding, factuality, instruction following, and multi-turn conversation quality, and fixes a multi-turn conversation bug in the original instruction-tuned release.

Benchmark scores

BenchmarkScore
Arena Text — Business, Management & Financial Ops (No Style Control)1063
Arena Text — Chinese (No Style Control)1061
Arena Text — Creative Writing (No Style Control)1059
Arena Text — English (No Style Control)1124
Arena Text — Entertainment, Sports & Media (No Style Control)1050
Arena Text — Exclude Ties (No Style Control)918
Arena Text — Exclude Ties (Style Controlled)1041
Arena Text — Expert (No Style Control)1039
Arena Text — French (No Style Control)1066
Arena Text — French (Style Controlled)1175
Arena Text — German (No Style Control)1054
Arena Text — Hard Prompts (No Style Control)1070
Arena Text — Hard Prompts English (No Style Control)1091
Arena Text — Instruction Following (No Style Control)1057
Arena Text — Japanese (No Style Control)971
Arena Text — Korean (No Style Control)988
Arena Text — Legal & Government (No Style Control)1094
Arena Text — Life, Physical & Social Science (No Style Control)1078
Arena Text — Longer Query (No Style Control)1055
Arena Text — Medicine & Healthcare (No Style Control)1072
Arena Text — Multi-turn (No Style Control)1040
Arena Text — Non-English (No Style Control)1051
Arena Text — Non-English (Style Controlled)1144
Arena Text — Occupational Mathematical (No Style Control)1098
Arena Text — Russian (No Style Control)1046
Arena Text — Software & IT Services (No Style Control)1095
Arena Text — Spanish (No Style Control)1049
Arena Text — Writing, Literature & Language (No Style Control)1073
BBH (Open LLM Leaderboard v2)39.4
Capability96.0
Loading Atlas data…