Claude Models Playing Pokémon

Claude models playing Pokémon* Milestone progress over time MILESTONE REACHED Get Surge's Badge Reach Vermilion City Get Misty's Badge Reach Cerulean City Reach Mt. Moon Get Brock's Badge Reach Viridian Forest Get Oak's Parcel Reach Viridian City Get Starter Leave House Start NUMBER OF ACTIONS 3.0 Sonnet 3.5 Sonnet 3.5 Sonnet (new) 3.7 Sonnet * Publication is a registered trademark of Nintendo of America Inc. This chart reflects tendency likely to reflect gains in milestones reached by Claude models. No affiliation, sponsorship, or endorsement by Nintendo of America Inc. is implied or intended. Claude 3.7 Sonnet demonstrates that it is the very best of all of the Sonnet models so far at playing Pokémon Red. On the x-axis is the number of interactions Claude completes as it plays the game; on the y-axis are important milestones in the game involving collecting certain items, navigating to certain areas, and defeating certain game bosses.

A line chart compares Claude Sonnet models as they play Pokémon Red, tracking how many actions each model needs to reach game milestones. The joke is that newer models appear to make much more progress, while the chart humorously treats AI benchmark performance like a Pokémon speedrun and claims Claude 3.7 Sonnet is the best.

• @dev_meme / 6578 2025-03-20