o3 Crushes Coding Benchmarks

Software Engineering (SWE-bench Verified) Competition Code (Codeforces) 100 80 60 40 20 0 accuracy o1 preview 41.3 o1 48.9 o3 71.7 3000 2000 1000 0 Elo o1 preview 1258 o1 1891 o3 2727

Two bar charts compare o1 preview, o1, and o3 on software engineering and competitive programming benchmarks. The joke is that o3 dramatically outperforms the earlier models, making the uploader’s “where’s the catch?” reaction sarcastic because access is restricted to paying or privileged users.

• @dev_meme / 6469 2024-12-20