← All benchmarks

Linguini

General QA

Linguini tests language-agnostic linguistic reasoning with 894 questions grouped into 160 problems from the International Linguistics Olympiad, covering 75 mostly low-resource languages. Each problem supplies parallel examples in context, and the model must induce the language's morphology, phonology or numeral system to translate, fill blanks, match or convert numerals. The score is whole-problem exact match: a problem counts only when all of its answers are correct.

Top models (higher is better)