Skip to content
localllm
日本語

Performance ranking (third-party evaluations)

Performance ranking by GPQA Diamond (Epoch AI)

Accuracy on GPQA Diamond (198 graduate-level biology, physics and chemistry multiple-choice questions) run by Epoch AI. Random guessing scores about 25%. The range is the 95% interval (from the standard error).

Top 39 of 39 models listed on this site

Values as of Aug 30, 2026

  1. 1
    Kimi-K2.6

    moonshotai 1,026.4B kimi-k2.6

    Accuracy (95% interval) 90.8% 87.4%–94.1%

    UD-Q2_K_XL 342 GB

  2. 2
    Kimi-K2.7-Code

    moonshotai 1,026.4B kimi-k2.7-code

    Accuracy (95% interval) 87.9% 83.3%–92.4%

    UD-IQ1_M 305 GB

  3. 3
    Qwen3.5-397B-A17B

    Qwen 396.3B qwen3.5-397b-a17b

    Accuracy (95% interval) 85.9% 81.0%–90.7%

    Q4_K_M 246 GB

  4. 4
    Qwen3.6-27B

    Qwen 26.9B qwen3.6-27b

    Accuracy (95% interval) 85.9% 81.0%–90.7%

    Q4_K_M 18.6 GB

  5. 5
    Qwen3.6-35B-A3B

    Qwen 34.7B qwen3.6-35b-a3b

    Accuracy (95% interval) 83.8% 78.7%–89.0%

    UD-Q4_K_M 23.4 GB

  6. 6
    Qwen3.5-35B-A3B

    Qwen 34.7B qwen3.5-35b-a3b

    Accuracy (95% interval) 83.5% 79.2%–87.7%

    Q4_K_M 23.3 GB

  7. 7
    Qwen3.5-9B

    Qwen 9B qwen3.5-9B

    Accuracy (95% interval) 79.0% 74.3%–83.6%

    Q4_K_M 7.05 GB

  8. 8
    DeepSeek-R1-0528

    deepseek-ai 671B DeepSeek-R1-0528

    Accuracy (95% interval) 76.3% 71.6%–81.0%

    Q4_K_M 411 GB

  9. 9
    Qwen3-30B-A3B-Thinking-2507

    Qwen 30.5B qwen3-30b-a3b-thinking-2507

    Accuracy (95% interval) 70.1% 64.5%–75.6%

    Q4_K_M 20.2 GB

  10. 10
    DeepSeek-R1

    deepseek-ai 671B DeepSeek-R1

    Accuracy (95% interval) 69.2% 63.2%–75.2%

    Q4_K_M 446 GB

  11. 11
    DeepSeek-V3-0324

    deepseek-ai 671B DeepSeek-V3-0324

    Accuracy (95% interval) 67.6% 62.4%–72.9%

    IQ1_S 175 GB

  12. 12
    Qwen3-32B

    Qwen 32.8B Qwen3-32B

    Accuracy (95% interval) 65.7% 60.3%–71.2%

    Q4_K_M 22.8 GB

  13. 13
    QwQ-32B

    Qwen 32.8B QwQ-32B

    Accuracy (95% interval) 65.3% 59.8%–70.9%

    Q4_K_M 22.8 GB

  14. 14
    DeepSeek-R1-Distill-Qwen-32B

    deepseek-ai 32.8B DeepSeek-R1-Distill-Qwen-32B

    Accuracy (95% interval) 64.1% 58.7%–69.5%

    Q4_K_M 22.8 GB

  15. 15
    Qwen3-14B

    Qwen 14.8B qwen3-14b

    Accuracy (95% interval) 63.8% 58.0%–69.5%

    Q4_K_M 11.2 GB

  16. 16
    Qwen3-30B-A3B

    Qwen 30.5B Qwen3-30B-A3B

    Accuracy (95% interval) 61.7% 55.9%–67.5%

    Q4_K_M 20.2 GB

  17. 17
    gpt-oss-20b

    openai 20.9B gpt-oss-20b_medium

    Accuracy (95% interval) 60.8% 55.9%–65.7%

    Q4_K_M 12.8 GB

  18. 18
    Qwen3-8B

    Qwen 8.2B qwen3-8b

    Accuracy (95% interval) 56.8% 51.0%–62.5%

    Q4_K_M 7.08 GB

  19. 19
    phi-4

    microsoft 14.7B phi-4

    Accuracy (95% interval) 56.1% 51.0%–61.1%

    Q4_K_M 11.5 GB

  20. 20
    DeepSeek-R1-Distill-Llama-70B

    deepseek-ai 70.6B DeepSeek-R1-Distill-Llama-70B

    Accuracy (95% interval) 55.7% 49.9%–61.6%

    Q4_K_M 46.0 GB

  21. 21
    Qwen3-30B-A3B-Instruct-2507

    Qwen 30.5B qwen3-30b-a3b-instruct-2507

    Accuracy (95% interval) 55.6% 50.0%–61.3%

    Q4_K_M 20.2 GB

  22. 22
    Qwen3-4B

    Qwen 4B qwen3-4b

    Accuracy (95% interval) 52.3% 46.4%–58.2%

    Q4_K_M 4.55 GB

  23. 23
    Llama-4-Scout-17B-16E-Instruct

    meta-llama 107.8B Llama-4-Scout-17B-16E-Instruct

    Accuracy (95% interval) 51.8% 45.6%–58.0%

    Q4_K_M 67.9 GB

  24. 24
    Qwen2.5-72B-Instruct

    Qwen 72.7B qwen2.5-72b-instruct

    Accuracy (95% interval) 49.1% 43.9%–54.4%

    Q4_K_M 50.9 GB

  25. 25
    Mistral-Small-3.2-24B-Instruct-2506

    mistralai 23.6B mistral-small-3.2-2506

    Accuracy (95% interval) 49.1% 43.6%–54.5%

    Q4_K_M 16.5 GB

  26. 26
    gemma-3-27b-it

    google 27B gemma-3-27b-it

    Accuracy (95% interval) 47.7% 42.4%–53.0%

    Q4_K_M 18.9 GB

  27. 27
    Llama-3.3-70B-Instruct

    meta-llama 70.6B Llama-3.3-70B-Instruct

    Accuracy (95% interval) 47.4% 41.9%–53.0%

    Q4_K_M 46.0 GB

  28. 28
    Qwen2.5-32B-Instruct

    Qwen 32.8B qwen2.5-32b-instruct

    Accuracy (95% interval) 46.1% 41.0%–51.2%

    Q4_K_M 22.8 GB

  29. 29
    DeepSeek-R1-Distill-Qwen-14B

    deepseek-ai 14.8B DeepSeek-R1-Distill-Qwen-14B

    Accuracy (95% interval) 44.7% 39.1%–50.3%

    Q4_K_M 11.4 GB

  30. 30
    Meta-Llama-3-70B-Instruct

    NousResearch 70.6B Meta-Llama-3-70B-Instruct

    Accuracy (95% interval) 40.6% 35.4%–45.7%

    Q4_K_M 46.0 GB

  31. 31
    gemma-3-12b-it

    google 11.8B gemma-3-12b-it

    Accuracy (95% interval) 39.5% 34.5%–44.4%

    Q4_K_M 9.41 GB

  32. 32
    Qwen3-1.7B

    Qwen 1.7B qwen3-1.7b

    Accuracy (95% interval) 38.0% 32.6%–43.4%

    Q4_K_M 2.89 GB

  33. 33
    gemma-2-27b-it

    google 27.2B gemma-2-27b-it

    Accuracy (95% interval) 36.5% 31.8%–41.2%

    Q4_K_M 20.1 GB

  34. 34
    Qwen2.5-7B-Instruct

    Qwen 7.6B qwen2.5-7b-instruct

    Accuracy (95% interval) 35.5% 31.0%–40.0%

    Q4_K_M 6.00 GB

  35. 35
    DeepSeek-R1-Distill-Qwen-1.5B

    deepseek-ai 1.8B DeepSeek-R1-Distill-Qwen-1.5B

    Accuracy (95% interval) 33.6% 29.5%–37.6%

    Q4_K_M 2.20 GB

  36. 36
    gemma-2-9b-it

    google 9.2B gemma-2-9b-it

    Accuracy (95% interval) 27.5% 22.9%–32.0%

    Q4_K_M 9.02 GB

  37. 37
    Llama-3.2-1B-Instruct

    meta-llama 1.2B Llama-3.2-1B-Instruct

    Accuracy (95% interval) 23.9% 20.7%–27.1%

    Q4_K_M 1.88 GB

  38. 38
    gemma-3-4b-it

    google 3.9B gemma-3-4b-it

    Accuracy (95% interval) 23.2% 19.6%–26.9%

    Q4_K_M 3.91 GB

  39. 39
    gemma-3-1b-it

    google 1B gemma-3-1b-it

    Accuracy (95% interval) 19.9% 17.0%–22.9%

    Q4_K_M 1.94 GB

Estimated Memory at 8K: Estimated with a formula. Actual usage depends on your setup and settings.

About this ranking

Values are from GPQA Diamond (Epoch AI) data (CC BY 4.0), shown unchanged (sorting and filtering by this site).