BioSimBench 1.0
Science
CellType's benchmark of language models predicting which genes respond when human cells receive a drug, a gene knockdown or an immune signal, on held-out experimental conditions. The score is the mean perturbation-specific F1 over four tracks: predicting the average response scores zero, and two lab replicates agree at 0.462 where they exist. Source experiments are public, but test answers stay private on CellType's scoring infrastructure.
Benchmark source: Celltype
Top models (higher is better)
| Model | Score | Source |
|---|---|---|
| Gemini 4 Argon | 18.4 | BioSimBench 1.0: Gemini 4 Argon · CellType |
| Claude Sonnet 5.5 | 16.0 | BioSimBench 1.0: Claude Sonnet 5.5 · CellType |
| GPT-6 Astra | 15.9 | BioSimBench 1.0: GPT-6 Astra · CellType |
| Gemini 3.1 Pro Preview | 15.3 | BioSimBench 1.0: Gemini 3.1 Pro · CellType |
| Gemini 3.8 Flash | 15.3 | BioSimBench 1.0: Gemini 3.8 Flash · CellType |
| GPT-6.1 Sol | 15.3 | BioSimBench 1.0: GPT-6.1 Sol · CellType |
| Claude Opus 5.5 | 14.5 | BioSimBench 1.0: Claude Opus 5.5 · CellType |
| GPT-6 Sol | 13.6 | BioSimBench 1.0: GPT-6 Sol · CellType |
| Claude Haiku 5.5 | 13.0 | BioSimBench 1.0: Claude Haiku 5.5 · CellType |
| GPT-6 Luna | 12.4 | BioSimBench 1.0: GPT-6 Luna · CellType |
| Grok 4.7 | 12.3 | BioSimBench 1.0: Grok 4.7 · CellType |