Skip to content
localllm
日本語

Find by VRAM / unified memory

Local LLMs that run in 32 GB of VRAM

Of the 1,276 models listed on this site, 965 have a quantization whose estimated memory fits in 32 GB at an 8,192-token context. For each model we show the largest (most precise) quantization that fits.

Data as of

Models that fit
965
Largest model that fits
Mixtral-8x22B-Instruct-v0.1 140.6B
32 GB devices (listed here)
8

Sorted by Hugging Face downloads (last 30 days, summed across sources). Not a performance ranking.

See the full download ranking →
  1. 1 Qwen3.8-27B 27.3B UD-Q8_K_XL 33.2 GB
  2. 2 Ornith-1.5-9B 9.2B BF16 19.8 GB
  3. 3 Huihui-Qwen3.6-35B-A3B-abliterated-MTP 35.5B Q6_K 30.5 GB
  4. 4 Ornith-1.5-35B-A3B 35.5B Q6_K_L 32.1 GB
  5. 5 Huihui-Qwen3.8-27B-abliterated 27.3B UD-Q8_K_XL 33.2 GB
  6. 6 OTel-2.0-Decision-26B-A4B-4bit 25.2B Q8_0 28.4 GB
  7. 7 Qwen3.8-27B-iMatrix-NVFP4-MTP NVFP4 18.9 GB
  8. 8 Tiel-Coder-35B-A3B-GGUF-MTP UD-Q6_K_XL 33.5 GB
  9. 9 gemma-4-12B-it 11.9B BF16 25.5 GB
  10. 10 Qwen3.5-9B 9B BF16 19.8 GB

Estimated Memory is estimated at 8,192 tokens with an f16 KV cache.

Third-party evaluations of models that run in 32 GB

Top values from other organizations for the original models before quantization (not this site's rating).

Human preference (LMArena)

See all →
  1. 1.Qwen3.8-27B 1,433
  2. 2.Qwen3.5-27B 1,409
  3. 3.Qwen3-Next-80B-A3B-Instruct 1,399
  4. 4.Qwen3.5-35B-A3B 1,394
  5. 5.Qwen3-30B-A3B-Instruct-2507 1,382

Source: LMArena (CC BY 4.0).

Capabilities (Epoch AI)

See all →
  1. 1.Qwen3.8-27B 152.7
  2. 2.Qwen3.6-27B 146.7
  3. 3.gemma-4-31B-it 144.2
  4. 4.Qwen3.6-35B-A3B 144.1
  5. 5.Qwen3.5-35B-A3B 144.0

Source: Epoch Capabilities Index (Epoch AI) (CC BY 4.0).

Devices with 32 GB of VRAM or unified memory

Choose a device to see the models that run on it and estimated generation speed.

All models that fit in 32 GB of VRAM

One entry per model, up to 30, sorted by popularity. For each model we show the largest (most precise) quantization that fits in this memory with a context of at least 8K tokens. The context length is the longest one that quantization fits (for models whose maximum is under 8K, such as embedding models, that maximum). Models that are not for chat or text generation are labelled with their use. "Left" is the memory minus the estimate; under 5% of the memory is marked "tight" (not a guarantee that it fits).

Frequently asked questions

Which local LLMs run in 32 GB of VRAM?
On this site, 965 models have a quantization that fits in 32 GB at 8,192 tokens (estimated). Popular ones include Qwen3.8-27B, Ornith-1.5-9B, Huihui-Qwen3.6-35B-A3B-abliterated-MTP, Ornith-1.5-35B-A3B, Huihui-Qwen3.8-27B-abliterated.
What is the largest model that runs in 32 GB of VRAM?
Among the models listed here, Mixtral-8x22B-Instruct-v0.1 (about 140.6B). With IQ1_S it needs an estimated 32.1 GB at 8,192 tokens. Smaller quantizations lower quality.
Is it the same for a Mac with 32 GB of unified memory?
macOS limits how much memory the GPU can use, so not all 32 GB is available to the model. On a Mac, leave headroom over the required memory.
What happens with a longer context?
The KV cache grows, so the required memory grows too. Each model page shows estimated memory for each context length.