Performance ranking (third-party evaluations)
Performance ranking of models that run in 8 GB of VRAM (GPQA Diamond (Epoch AI))
Accuracy on GPQA Diamond (198 graduate-level biology, physics and chemistry multiple-choice questions) run by Epoch AI. Random guessing scores about 25%. The range is the 95% interval (from the standard error).
Only models with a quantization whose estimated memory fits in 8 GB at an 8,192-token context.
Top 15 of 15 models listed on this site
Values as of Aug 30, 2026
-
1Qwen3.5-9B
Qwen 9B qwen3.5-9B
Accuracy (95% interval) 79.0% 74.3%–83.6%
Q5_K_M 8.48 GB
-
2Qwen3-14B
Qwen 14.8B qwen3-14b
Accuracy (95% interval) 63.8% 58.0%–69.5%
IQ3_XS 8.57 GB
-
3Qwen3-8B
Qwen 8.2B qwen3-8b
Accuracy (95% interval) 56.8% 51.0%–62.5%
Q5_K_L 8.29 GB
-
4phi-4
microsoft 14.7B phi-4
Accuracy (95% interval) 56.1% 51.0%–61.1%
Q2_K 7.97 GB
-
5Qwen3-4B
Qwen 4B qwen3-4b
Accuracy (95% interval) 52.3% 46.4%–58.2%
UD-Q8_K_XL 7.11 GB
-
6Mistral-Small-3.2-24B-Instruct-2506
mistralai 23.6B mistral-small-3.2-2506
Accuracy (95% interval) 49.1% 43.6%–54.5%
UD-IQ1_M 8.16 GB
-
7DeepSeek-R1-Distill-Qwen-14B
deepseek-ai 14.8B DeepSeek-R1-Distill-Qwen-14B
Accuracy (95% interval) 44.7% 39.1%–50.3%
Q2_K_L 8.41 GB
-
8gemma-3-12b-it
google 11.8B gemma-3-12b-it
Accuracy (95% interval) 39.5% 34.5%–44.4%
UD-Q3_K_XL 8.26 GB
-
9Qwen3-1.7B
Qwen 1.7B qwen3-1.7b
Accuracy (95% interval) 38.0% 32.6%–43.4%
BF16 5.86 GB
-
10Qwen2.5-7B-Instruct
Qwen 7.6B qwen2.5-7b-instruct
Accuracy (95% interval) 35.5% 31.0%–40.0%
Q6_K_L 7.84 GB
-
11DeepSeek-R1-Distill-Qwen-1.5B
deepseek-ai 1.8B DeepSeek-R1-Distill-Qwen-1.5B
Accuracy (95% interval) 33.6% 29.5%–37.6%
F32 8.20 GB
-
12gemma-2-9b-it
google 9.2B gemma-2-9b-it
Accuracy (95% interval) 27.5% 22.9%–32.0%
IQ4_XS 8.45 GB
-
13Llama-3.2-1B-Instruct
meta-llama 1.2B Llama-3.2-1B-Instruct
Accuracy (95% interval) 23.9% 20.7%–27.1%
F16 3.55 GB
-
14gemma-3-4b-it
google 3.9B gemma-3-4b-it
Accuracy (95% interval) 23.2% 19.6%–26.9%
UD-Q8_K_XL 6.59 GB
-
15gemma-3-1b-it
google 1B gemma-3-1b-it
Accuracy (95% interval) 19.9% 17.0%–22.9%
F16 3.14 GB
Estimated Memory at 8K: Estimated with a formula. Actual usage depends on your setup and settings.
About this ranking
- Values for open-weight models come from Epoch AI's data (GPQA Diamond). When a model was measured through several providers, only the most recent measurement is used (not the best one). Updated weekly.
- Values are for the original model before quantization. GGUF quantization (especially small quantizations) can lower performance.
- Values are attached only when the model name in the source matches a model listed on this site (its original Hugging Face repository). Fine-tunes and modified versions do not get them.
- Each source is a separate ranking. We do not mix values from different sources or assign our own scores.
- Small and new models are often not evaluated by the source, so they do not appear here.
Values are from GPQA Diamond (Epoch AI) data (CC BY 4.0), shown unchanged (sorting and filtering by this site).