GeForce RTX 4060

With 8 GB

One entry per model, up to 30, sorted by popularity. For each model we show the largest (most precise) quantization that fits in this memory with a context of at least 8K tokens. The context length is the longest one that quantization fits (for models whose maximum is under 8K, such as embedding models, that maximum). Models that are not for chat or text generation are labelled with their use. "Left" is the memory minus the estimate; under 5% of the memory is marked "tight" (not a guarantee that it fits).

• Qwen3.8-27B UD-IQ1_M
  Context length 8,192 Estimated 8.47 GB 0.12 GB left (tight)
• Ornith-1.5-9B Q5_K_M
  Context length 16,384 Estimated 8.49 GB 0.10 GB left (tight)
• gemma-4-12B-it IQ4_XS Image input
  Context length 8,192 Estimated 8.49 GB 0.10 GB left (tight)
• Ornith-1.0-9B Q5_K_M
  Context length 16,384 Estimated 8.13 GB 0.46 GB left
• mxbai-embed-large-v1 F32 Embedding (not for chat)
  Context length 512 Estimated 2.41 GB 6.18 GB left
• Qwen3.5-9B-The-Defiant-Fable-Uncensored-Heretic-NEO-IMATRIX-MAX-MTP Q4_K_M Image input
  Context length 8,192 Estimated 8.35 GB 0.24 GB left (tight)
• MOSS-Music-8B-Instruct Q4_K_M
  Context length 16,384 Estimated 8.45 GB 0.14 GB left (tight)
• Qwen3-VL-8B-Instruct UD-Q5_K_XL Image input
  Context length 8,192 Estimated 7.94 GB 0.65 GB left
• Qwen3.5-9B Q5_K_M Image input
  Context length 8,192 Estimated 8.48 GB 0.11 GB left (tight)
• Ornith-1.5-9B-uncensored Q5_K_M
  Context length 16,384 Estimated 8.10 GB 0.49 GB left
• Qwen3.5-4B UD-Q8_K_XL Image input
  Context length 32,768 Estimated 8.12 GB 0.47 GB left
• Wan2.1 Q4_K_M
  Context length 16,384 Estimated 8.08 GB 0.51 GB left
• Qwen3-ASR-1.7B F16 Speech / audio
  Context length 16,384 Estimated 6.82 GB 1.77 GB left
• gemma-4-E4B-it-qat-q4_0 Q4_0 Image input
  Context length 131,072 Estimated 8.42 GB 0.17 GB left (tight)
• Ternary-Bonsai-2-9B Q8_0 Image input
  Context length 131,072 Estimated 8.07 GB 0.52 GB left
• HyperCLOVAX-SEED-Text-Instruct-1.5B-Q4_K_M Q4_K_M
  Context length 65,536 Estimated 8.34 GB 0.25 GB left (tight)
• Hy-MT2-7B Q6_K
  Context length 8,192 Estimated 8.04 GB 0.55 GB left
• Ternary-Bonsai-2-2B Q2_0 Image input
  Context length 262,144 Estimated 4.83 GB 3.76 GB left
• MiMo-V2.6-Distill-Qwen-9B Q5_K_M Image input
  Context length 16,384 Estimated 8.51 GB 0.08 GB left (tight)
• Cyber-Ornith-1.5-9B-OBLITERATED Q5_K_M
  Context length 16,384 Estimated 8.10 GB 0.49 GB left
• Sharp-Spark-X2.5-4B Q6_K_XL
  Context length 65,536 Estimated 6.95 GB 1.64 GB left
• AdaVaR-3B F16
  Context length 32,768 Estimated 8.23 GB 0.36 GB left (tight)
• Qwen3-8B Q5_K_L
  Context length 8,192 Estimated 8.29 GB 0.30 GB left (tight)
• Qwen3.5-9B-Kimi-k3-Distilled Q5_K_M
  Context length 16,384 Estimated 8.28 GB 0.31 GB left (tight)
• Qwen3-4B-Instruct-2507 UD-Q8_K_XL
  Context length 16,384 Estimated 8.32 GB 0.27 GB left (tight)
• B1-9B Q5_K_M
  Context length 16,384 Estimated 8.28 GB 0.31 GB left (tight)
• Qwen3-VL-4B-Instruct UD-Q8_K_XL Image input
  Context length 16,384 Estimated 8.32 GB 0.27 GB left (tight)
• Qwen2.5-7B-Instruct Q6_K_L
  Context length 16,384 Estimated 8.31 GB 0.28 GB left (tight)
• Qwen3-4B UD-Q8_K_XL
  Context length 16,384 Estimated 8.32 GB 0.27 GB left (tight)
• Qwen-Image-2.1-Turbo-Uncensored Q5_K Image / video generation
  Context length 8,192 Estimated 8.34 GB 0.25 GB left (tight)