MultiFileTest - JavaScript correctness rate
Code · 2025-02-10
The JavaScript track of MultiFileTest (first released as ProjectTest), from the University of Illinois Chicago, Salesforce Research and Scale AI: the model writes unit tests for each of 20 multi-file JavaScript projects supplied whole in the prompt, run with Jest, zero-shot at temperature 0 and scored as generated. The score is the correctness rate: for each project, the share of generated unit tests that pass (zero when the suite does not run), averaged over the 20 projects; executability, coverage and the fixing scenarios are not imported.
Top models (higher is better)
| Model | Score |
|---|---|
| o1 (2024-12-17) | 87.0 |
| GPT-5 Mini | 73.0 |
| Gemini 3 Pro Preview | 68.0 |
| Claude 3.5 Sonnet (Oct 2024) | 65.0 |
| DeepSeek-Coder-Instruct 6.7B | 62.0 |
| Code Llama Instruct 7B | 26.0 |
| CodeQwen1.5 7B Chat | 23.0 |