Performance ranking (third-party evaluations)

Performance ranking of models that run in 24 GB of VRAM (GPQA Diamond (Epoch AI))

Accuracy on GPQA Diamond (198 graduate-level biology, physics and chemistry multiple-choice questions) run by Epoch AI. Random guessing scores about 25%. The range is the 95% interval (from the standard error).

Only models with a quantization whose estimated memory fits in 24 GB at an 8,192-token context.

Top 31 of 31 models listed on this site

Values as of Aug 30, 2026

Rank Model Accuracy (95% interval) Quantization that fits 24 GB
1

Qwen3.6-27B
Qwen 26.9B qwen3.6-27b

Accuracy (95% interval) 85.9% 81.0%–90.7%

Q6_K 25.4 GB
2

Qwen3.6-35B-A3B
Qwen 34.7B qwen3.6-35b-a3b

Accuracy (95% interval) 83.8% 78.7%–89.0%

Q4_1 24.1 GB
3

Qwen3.5-35B-A3B
Qwen 34.7B qwen3.5-35b-a3b

Accuracy (95% interval) 83.5% 79.2%–87.7%

Q4_1 24.1 GB
4

Qwen3.5-9B
Qwen 9B qwen3.5-9B

Accuracy (95% interval) 79.0% 74.3%–83.6%

BF16 19.8 GB
5

Qwen3-30B-A3B-Thinking-2507
Qwen 30.5B qwen3-30b-a3b-thinking-2507

Accuracy (95% interval) 70.1% 64.5%–75.6%

UD-Q5_K_XL 23.4 GB
6

Qwen3-32B
Qwen 32.8B Qwen3-32B

Accuracy (95% interval) 65.7% 60.3%–71.2%

Q5_K_S 25.6 GB
7

QwQ-32B
Qwen 32.8B QwQ-32B

Accuracy (95% interval) 65.3% 59.8%–70.9%

Q5_K_S 25.6 GB
8

DeepSeek-R1-Distill-Qwen-32B
deepseek-ai 32.8B DeepSeek-R1-Distill-Qwen-32B

Accuracy (95% interval) 64.1% 58.7%–69.5%

Q5_K_S 25.6 GB
9

Qwen3-14B
Qwen 14.8B qwen3-14b

Accuracy (95% interval) 63.8% 58.0%–69.5%

UD-Q8_K_XL 20.9 GB
10

Qwen3-30B-A3B
Qwen 30.5B Qwen3-30B-A3B

Accuracy (95% interval) 61.7% 55.9%–67.5%

UD-Q5_K_XL 23.4 GB
11

gpt-oss-20b
openai 20.9B gpt-oss-20b_medium

Accuracy (95% interval) 60.8% 55.9%–65.7%

F16 15.0 GB
12

Qwen3-8B
Qwen 8.2B qwen3-8b

Accuracy (95% interval) 56.8% 51.0%–62.5%

BF16 18.4 GB
13

phi-4
microsoft 14.7B phi-4

Accuracy (95% interval) 56.1% 51.0%–61.1%

Q8_0 18.0 GB
14

DeepSeek-R1-Distill-Llama-70B
deepseek-ai 70.6B DeepSeek-R1-Distill-Llama-70B

Accuracy (95% interval) 55.7% 49.9%–61.6%

UD-IQ2_XXS 22.9 GB
15

Qwen3-30B-A3B-Instruct-2507
Qwen 30.5B qwen3-30b-a3b-instruct-2507

Accuracy (95% interval) 55.6% 50.0%–61.3%

Q5_K_L 23.6 GB
16

Qwen3-4B
Qwen 4B qwen3-4b

Accuracy (95% interval) 52.3% 46.4%–58.2%

BF16 10.1 GB
17

Mistral-Small-3.2-24B-Instruct-2506
mistralai 23.6B mistral-small-3.2-2506

Accuracy (95% interval) 49.1% 43.6%–54.5%

UD-Q6_K_XL 22.9 GB
18

gemma-3-27b-it
google 27B gemma-3-27b-it

Accuracy (95% interval) 47.7% 42.4%–53.0%

Q6_K 24.6 GB
19

Llama-3.3-70B-Instruct
meta-llama 70.6B Llama-3.3-70B-Instruct

Accuracy (95% interval) 47.4% 41.9%–53.0%

IQ2_S 25.7 GB
20

Qwen2.5-32B-Instruct
Qwen 32.8B qwen2.5-32b-instruct

Accuracy (95% interval) 46.1% 41.0%–51.2%

Q5_K_S 25.6 GB
21

DeepSeek-R1-Distill-Qwen-14B
deepseek-ai 14.8B DeepSeek-R1-Distill-Qwen-14B

Accuracy (95% interval) 44.7% 39.1%–50.3%

Q8_0 18.2 GB
22

Meta-Llama-3-70B-Instruct
NousResearch 70.6B Meta-Llama-3-70B-Instruct

Accuracy (95% interval) 40.6% 35.4%–45.7%

IQ2_S 25.7 GB
23

gemma-3-12b-it
google 11.8B gemma-3-12b-it

Accuracy (95% interval) 39.5% 34.5%–44.4%

BF16 25.7 GB
24

Qwen3-1.7B
Qwen 1.7B qwen3-1.7b

Accuracy (95% interval) 38.0% 32.6%–43.4%

BF16 5.86 GB
25

gemma-2-27b-it
google 27.2B gemma-2-27b-it

Accuracy (95% interval) 36.5% 31.8%–41.2%

Q5_K_L 23.2 GB
26

Qwen2.5-7B-Instruct
Qwen 7.6B qwen2.5-7b-instruct

Accuracy (95% interval) 35.5% 31.0%–40.0%

F16 16.6 GB
27

DeepSeek-R1-Distill-Qwen-1.5B
deepseek-ai 1.8B DeepSeek-R1-Distill-Qwen-1.5B

Accuracy (95% interval) 33.6% 29.5%–37.6%

F32 8.20 GB
28

gemma-2-9b-it
google 9.2B gemma-2-9b-it

Accuracy (95% interval) 27.5% 22.9%–32.0%

Q8_0 14.0 GB
29

Llama-3.2-1B-Instruct
meta-llama 1.2B Llama-3.2-1B-Instruct

Accuracy (95% interval) 23.9% 20.7%–27.1%

F16 3.55 GB
30

gemma-3-4b-it
google 3.9B gemma-3-4b-it

Accuracy (95% interval) 23.2% 19.6%–26.9%

BF16 9.19 GB
31

gemma-3-1b-it
google 1B gemma-3-1b-it

Accuracy (95% interval) 19.9% 17.0%–22.9%

F16 3.14 GB

Estimated Memory at 8K: Estimated with a formula. Actual usage depends on your setup and settings.

About this ranking

Values for open-weight models come from Epoch AI's data (GPQA Diamond). When a model was measured through several providers, only the most recent measurement is used (not the best one). Updated weekly.
Values are for the original model before quantization. GGUF quantization (especially small quantizations) can lower performance.
Values are attached only when the model name in the source matches a model listed on this site (its original Hugging Face repository). Fine-tunes and modified versions do not get them.
Each source is a separate ranking. We do not mix values from different sources or assign our own scores.
Small and new models are often not evaluated by the source, so they do not appear here.

Values are from GPQA Diamond (Epoch AI) data (CC BY 4.0), shown unchanged (sorting and filtering by this site).