CaseReportBench - release-reported, metric unspecified
Health & Medicine · 2025-05-22
CaseReportBench is an expert-annotated dense information-extraction benchmark of 138 clinical case reports with a rare-disease (inborn errors of metabolism) focus, whose findings are organized into 14 body-system, laboratory and history categories and scored by Levenshtein similarity, exact match, token set ratio, BLEU, ROUGE-L and a hallucination rate. Release tables that print a single CaseReportBench number without naming the metric, category aggregation or prompting strategy are kept as a diagnostic.
Top models (higher is better)
| Model | Score |
|---|---|
| LFM2.5-350M | 32.5 |
| LFM2.5-230M | 22.5 |
| Qwen3.5-0.8B | 13.8 |
| Granite 4.0 H 350M | 12.4 |
| LFM2-350M | 11.7 |
| Gemma 3 1B IT | 2.3 |
| Granite 4.0 350M | 0.8 |