Benchmarks
347 sources · 1,258 models × 3,038 benchmarks · 158,912 scoresRamp SWE-Bench is a private, production-grounded coding benchmark built from real Ramp backend engineering tasks. Its canonical manifest contains 80 human-accepted tasks evaluated using mini-swe-agent, with pass rate reported as the percentage of tasks resolved. Published runs omitting task IDs 35 and 75 belong to the separate ramp-swe-bench-observed-78 cohort.
No signal or difficulty estimate. No usable scores.
This is a benchmark-family page. Scores are recorded on the component benchmarks below rather than directly on the parent, because the components use different score scales and directions.
| Metric | Models | Leader | Direction | Date |
|---|---|---|---|---|
| Ramp SWE-Bench (78 published tasks) | 26 | Claude Fable 588.5 | higher is better | — |