GeForce RTX 4090

With 24 GB

One entry per model, up to 30, sorted by popularity. For each model we show the largest (most precise) quantization that fits in this memory with a context of at least 8K tokens. The context length is the longest one that quantization fits (for models whose maximum is under 8K, such as embedding models, that maximum). Models that are not for chat or text generation are labelled with their use. "Left" is the memory minus the estimate; under 5% of the memory is marked "tight" (not a guarantee that it fits).

• Qwen3.8-27B Q6_K Image input
  Context length 8,192 Estimated 25.6 GB 0.17 GB left (tight) Speed 25–34
• Ornith-1.5-9B BF16
  Context length 131,072 Estimated 23.8 GB 1.94 GB left Speed 33–44
• Huihui-Qwen3.6-35B-A3B-abliterated-MTP Q4_K Image input
  Context length 131,072 Estimated 25.5 GB 0.26 GB left (tight) Speed not estimated (MoE)
• Ornith-1.5-35B-A3B Q4_1 Image input
  Context length 65,536 Estimated 24.9 GB 0.87 GB left (tight) Speed not estimated (MoE)
• Huihui-Qwen3.8-27B-abliterated Q6_K_L Image input
  Context length 8,192 Estimated 25.7 GB 0.09 GB left (tight) Speed 25–34
• OTel-2.0-Decision-26B-A4B-4bit Q4_K_M
  Context length 262,144 Estimated 23.6 GB 2.22 GB left Speed not estimated (MoE)
• Qwen3.8-27B-iMatrix-NVFP4-MTP NVFP4 Image input
  Context length 65,536 Estimated 22.6 GB 3.15 GB left Speed 35–47
• Tiel-Coder-35B-A3B-GGUF-MTP UD-Q4_K_XL Image input
  Context length 65,536 Estimated 25.2 GB 0.57 GB left (tight) Speed not estimated (MoE)
• gemma-4-12B-it BF16 Image input
  Context length 16,384 Estimated 25.7 GB 0.09 GB left (tight) Speed 25–34
• Qwen3.8-27B-Uncensored Q6_K
  Context length 16,384 Estimated 24.7 GB 1.06 GB left (tight) Speed 27–36
• Ornith-1.0-9B BF16
  Context length 131,072 Estimated 23.3 GB 2.43 GB left Speed 34–45
• mxbai-embed-large-v1 F32 Embedding (not for chat)
  Context length 512 Estimated 2.41 GB 23.4 GB left Speed 452–603
• Qwen3.6-35B-A3B-MTP UD-Q4_K_XL Image input
  Context length 65,536 Estimated 25.3 GB 0.46 GB left (tight) Speed not estimated (MoE)
• Qwen3.5-9B-The-Defiant-Fable-Uncensored-Heretic-NEO-IMATRIX-MAX-MTP BF16 Image input
  Context length 131,072 Estimated 23.8 GB 1.97 GB left Speed 33–44
• MOSS-Music-8B-Instruct F16 Speech / audio
  Context length 40,960 Estimated 23.3 GB 2.49 GB left Speed 37–49
• Qwen3.8-27B-Uncensored-HauhauCS-Aggressive-MTP Q5_K Image input
  Context length 65,536 Estimated 25.7 GB 0.05 GB left (tight) Speed 30–40
• gemma-4-26B-A4B-it UD-Q6_K_XL Image input
  Context length 32,768 Estimated 25.4 GB 0.42 GB left (tight) Speed not estimated (MoE)
• Qwen3-VL-8B-Instruct BF16 Image input
  Context length 32,768 Estimated 22.1 GB 3.70 GB left Speed 37–49
• Qwen3.5-9B BF16 Image input
  Context length 131,072 Estimated 23.8 GB 1.97 GB left Speed 33–44
• Ornith-1.5-9B-uncensored F16
  Context length 131,072 Estimated 23.3 GB 2.46 GB left Speed 34–45
• Qwen3.6-35B-A3B Q4_1 Image input
  Context length 65,536 Estimated 25.3 GB 0.45 GB left (tight) Speed not estimated (MoE)
• Ternary-Bonsai-2-27B Q2_0
  Context length 131,072 Estimated 17.4 GB 8.35 GB left Speed 79–106
• Swift-1.5-Qwen3.8-27B-GSQ-RCO IQ3_S
  Context length 131,072 Estimated 21.9 GB 3.86 GB left Speed 50–67
• Qwen3.5-4B BF16 Image input
  Context length 262,144 Estimated 18.4 GB 7.42 GB left Speed 70–93
• Ornith-1.0-35B Q4_K_M
  Context length 131,072 Estimated 25.0 GB 0.78 GB left (tight) Speed not estimated (MoE)
• Qwen3.8-27B-TURBO-Fable-Cold-Fusion-735-882-Heretic-Uncensored-NEO-CODER-MAX-MTP Q6_K Image input
  Context length 8,192 Estimated 25.3 GB 0.45 GB left (tight) Speed 26–34
• Dirk-Qwen3.8-27B UD-Q6_K Image input
  Context length 32,768 Estimated 25.3 GB 0.44 GB left (tight) Speed 28–37
• Wan2.1 Q4_K_M
  Context length 65,536 Estimated 22.3 GB 3.43 GB left Speed 36–48
• Qwen3.8-27B-GSQ-RCO IQ3_S Image input
  Context length 131,072 Estimated 21.9 GB 3.86 GB left Speed 50–67
• Qwen3-ASR-1.7B F16 Speech / audio
  Context length 65,536 Estimated 12.5 GB 13.3 GB left Speed 148–197