OSWorld

Name: OSWorld
Published: 2024-04-11

Agents · 2024-04-11

A computer-use benchmark where agents complete real desktop and web tasks in reproducible OS environments using keyboard/mouse actions and structured UI observations.

Top models (higher is better)

Model	Score
Opus 4.5	66.3
o3	45.2
Gemini 2.5 Pro Preview 03-25	41.4
Computer Use Preview	38.1
Claude 3.7 Sonnet	34.5
GPT-4o (source unspecified)	27.0
UI-TARS 72B DPO	24.6
Claude 3.5 Sonnet (Oct 2024)	22.0
UI-TARS 72B SFT	18.8
UI-TARS 7B DPO	18.7
UI-TARS 7B SFT	17.7
Aguvis-72B	10.3
Qwen2.5-VL-72B-Instruct	8.8
Kimi-VL-A3B-Instruct	8.2
Gemini 1.0 Pro Vision	5.8
GPT-4 Turbo	5.4
Gemini 1.5 Pro (source unspecified)	5.4
GPT-4o Mini	3.8
Opus 3	2.4
LLaVA-OneVision 72B	2.4
Qwen VL Max 0809	2.4
MiniCPM-V 2.6	1.9
CogAgent (source unspecified)	1.1