PubMedQA
Health & Medicine · 2019-09-13
PubMedQA asks yes, no or maybe research questions about PubMed abstracts, so a model must reason over biomedical study results. This row is the 500-question expert-labeled PQA-L test split in the reasoning-required setting, scored by answer accuracy; the Open Medical-LLM leaderboard's harness-specific results are kept separately. Higher is better.
Top models (higher is better)
| Model | Score |
|---|---|
| o3 | 80.0 |
| GPT-4o (source unspecified) | 78.4 |
| MedGemma 27B IT | 77.2 |
| MedGemma 27B Text IT | 76.8 |
| Gemini 2.5 Flash | 76.2 |
| Gemini 2.5 Pro | 75.8 |
| MedGemma 4B IT | 73.4 |
| Gemma 3 27B IT | 73.4 |
| Gemma 3 4B IT | 68.4 |
| MedGemma 1.5 4B IT | 68.2 |