Llama 4 Scout Benchmark Table

Llama 4 Scout instruction-tuned benchmarks Category Benchmark Llama 4 Scout Llama 3.3 70B Llama 3.1 405B Gemma 3 27B Mistral 3.1 24B Gemini 2.0 Flash-Lite Image Reasoning MMMU 69.4 64.9 62.8 68.0 MathVista 70.7 67.6 68.9 57.6 Image Understanding ChartQA 88.8 No multimodal support No multimodal support 76.3 86.2 73.0 DocVQA (test) 94.4 90.4 94.1 91.2 Coding LiveCodeBench (10/01/2024-02/01/2025) 32.8 33.3 27.7 29.7 — 28.9 Reasoning & Knowledge MMLU Pro 74.3 68.9 73.4 67.5 66.8 71.6 GPQA Diamond 57.2 50.5 49.0 42.4 46.0 51.5 Long Context MTOB (half book) eng → kgv/kvq → eng 42.2/36.6 Context window is 128K Context window is 128K Context window is 128K Context window is 128K 42.3/35.1¹ MTOB (full book) eng → kgv/kvq → eng 39.7/36.3 35.1/30.0³ 1. For Llama model results, we report 0-shot evaluation with temperature = 0 and no majority voting or parallel test time compute. For high-variance benchmarks (GPQA Diamond, LiveCodeBench), we average over multiple generations to reduce uncertainty. 2. For non-Llama models, we source the highest available self-reported eval results unless otherwise specified. We only include results from models that have reproducible evals (via API or open weights), and we only include non-thinking models. 3. Specialized to long context evals are not traditionally reported for general models, so we use internal runs to showcase Llama's frontier performance.

A clean benchmark comparison table ranks Llama 4 Scout against other large language and multimodal models across reasoning, image understanding, coding, knowledge, and long-context tasks. The joke comes from presenting an extremely technical evaluation chart as a meme, often used to celebrate an impressive model launch or an unexpectedly strong start to the year. The uploader’s caption, “Incredible start of the year,” frames the benchmark results as exciting news.

• @dev_meme / 6632 2025-04-05