BioSimBench 1.0: Immune signals
Science
Perturbation-specific F1 on 27 cases predicting gene expression changes when a cell receives an immune signal, with the donor held out. Frontier models run with two prompt wordings are scored on the better one.
Benchmark source: Celltype
Top models (higher is better)
| Model | Score | Source |
|---|---|---|
| Gemini 3.8 Flash | 16.5 | BioSimBench 1.0: Gemini 3.8 Flash · CellType |
| Gemini 3.1 Pro Preview | 16.1 | BioSimBench 1.0: Gemini 3.1 Pro · CellType |
| Gemini 4 Argon | 16.0 | BioSimBench 1.0: Gemini 4 Argon · CellType |
| GPT-6.1 Sol | 15.7 | BioSimBench 1.0: GPT-6.1 Sol · CellType |
| Claude Sonnet 5.5 | 15.6 | BioSimBench 1.0: Claude Sonnet 5.5 · CellType |
| GPT-6 Astra | 15.4 | BioSimBench 1.0: GPT-6 Astra · CellType |
| Claude Opus 5.5 | 14.0 | BioSimBench 1.0: Claude Opus 5.5 · CellType |
| GPT-6 Sol | 13.8 | BioSimBench 1.0: GPT-6 Sol · CellType |
| Claude Haiku 5.5 | 13.1 | BioSimBench 1.0: Claude Haiku 5.5 · CellType |
| GPT-6 Luna | 10.7 | BioSimBench 1.0: GPT-6 Luna · CellType |
| Grok 4.7 | 10.1 | BioSimBench 1.0: Grok 4.7 · CellType |