BIRD-SQL (Dev)
Code · 2023-05-04
BIRD is a large-scale text-to-SQL benchmark over 95 real databases from 37 professional domains, stressing dirty values and the external knowledge needed to write correct, efficient queries. This row is the 1,534-question development split scored by execution accuracy, the share of generated queries whose results match the gold query's results; higher is better.
Top models (higher is better)
| Model | Score |
|---|---|
| Gemini 2.0 Pro Experimental 02-05 | 59.3 |
| Gemini 2.0 Flash | 58.7 |
| Gemini 2.0 Flash-Lite Preview 02-05 | 57.4 |
| Gemini 2.0 Flash Experimental | 56.9 |
| Gemini 1.5 Pro 002 | 54.4 |
| Gemini 1.5 Flash 002 | 45.6 |