Skip to content
localllm
日本語

Discrete GPU

Arc A380

Data as of

Memory bandwidth
186 GB/s
Source
Checked:

With 6 GB

One entry per model, up to 30, sorted by popularity. For each model we show the largest (most precise) quantization that fits in this memory with a context of at least 8K tokens. The context length is the longest one that quantization fits (for models whose maximum is under 8K, such as embedding models, that maximum). Models that are not for chat or text generation are labelled with their use. "Left" is the memory minus the estimate; under 5% of the memory is marked "tight" (not a guarantee that it fits).

  • gemma-4-12B-it IQ2_S Image input
    Context length 8,192 Estimated 6.42 GB 0.02 GB left (tight) Speed 24–32
  • mxbai-embed-large-v1 F32 Embedding (not for chat)
    Context length 512 Estimated 2.41 GB 4.03 GB left Speed 83–111
  • Qwen3.5-9B-The-Defiant-Fable-Uncensored-Heretic-NEO-IMATRIX-MAX-MTP IQ2_M Image input
    Context length 8,192 Estimated 6.31 GB 0.13 GB left (tight) Speed 23–30
  • MOSS-Music-8B-Instruct F16 Speech / audio
    Context length 16,384 Estimated 4.99 GB 1.45 GB left Speed 65–86
  • Qwen3.5-9B UD-Q3_K_XL Image input
    Context length 8,192 Estimated 6.42 GB 0.02 GB left (tight) Speed 22–29
  • Ornith-1.5-9B-uncensored Q3_K_L
    Context length 8,192 Estimated 6.29 GB 0.15 GB left (tight) Speed 23–30
  • Qwen3.5-4B Q8_0 Image input
    Context length 16,384 Estimated 6.26 GB 0.19 GB left (tight) Speed 24–32
  • Wan2.1 Q4_K_M
    Context length 32,768 Estimated 6.10 GB 0.34 GB left Speed 41–55
  • Qwen3-ASR-1.7B F16 Speech / audio
    Context length 8,192 Estimated 5.88 GB 0.56 GB left Speed 27–36
  • gemma-4-E4B-it-qat-q4_0 Q4_0 Image input
    Context length 8,192 Estimated 6.40 GB 0.04 GB left (tight) Speed 22–29
  • Ternary-Bonsai-2-9B Q8_0 Image input
    Context length 65,536 Estimated 5.92 GB 0.52 GB left Speed 42–56
  • HyperCLOVAX-SEED-Text-Instruct-1.5B-Q4_K_M Q4_K_M
    Context length 32,768 Estimated 5.12 GB 1.32 GB left Speed 98–131
  • Hy-MT2-7B Q4_K_M
    Context length 4,096 Estimated 5.96 GB 0.48 GB left Speed 24–32
  • Ternary-Bonsai-2-2B Q2_0 Image input
    Context length 262,144 Estimated 4.83 GB 1.61 GB left Speed 204–272
  • MiMo-V2.6-Distill-Qwen-9B IQ3_M Image input
    Context length 8,192 Estimated 6.21 GB 0.23 GB left (tight) Speed 23–31
  • Cyber-Ornith-1.5-9B-OBLITERATED Q3_K_L
    Context length 8,192 Estimated 6.29 GB 0.15 GB left (tight) Speed 23–30
  • Sharp-Spark-X2.5-4B Q6_K_XL
    Context length 32,768 Estimated 5.74 GB 0.70 GB left Speed 31–41
  • AdaVaR-3B Q8_0
    Context length 32,768 Estimated 5.34 GB 1.10 GB left Speed 34–45
  • Qwen3.5-9B-Kimi-k3-Distilled Q3_K_L
    Context length 8,192 Estimated 6.42 GB 0.03 GB left (tight) Speed 22–29
  • Qwen3-4B-Instruct-2507 Q8_0
    Context length 8,192 Estimated 6.34 GB 0.11 GB left (tight) Speed 26–35
  • B1-9B Q3_K_L
    Context length 8,192 Estimated 6.42 GB 0.03 GB left (tight) Speed 22–29
  • Qwen3-VL-4B-Instruct Q8_0 Image input
    Context length 8,192 Estimated 6.34 GB 0.11 GB left (tight) Speed 26–35
  • Qwen2.5-7B-Instruct Q4_K_L
    Context length 8,192 Estimated 6.41 GB 0.04 GB left (tight) Speed 22–29
  • Qwen3-4B Q8_0
    Context length 8,192 Estimated 6.34 GB 0.11 GB left (tight) Speed 26–35
  • Qwen-Image-2.1-Turbo-Uncensored Q3_K Image / video generation
    Context length 8,192 Estimated 6.24 GB 0.20 GB left (tight) Speed 27–36
  • gemma-4-E4B-it UD-Q4_K_XL Image input
    Context length 8,192 Estimated 6.38 GB 0.07 GB left (tight) Speed 22–29
  • gemma-4-E2B-it UD-Q8_K_XL Image input
    Context length 8,192 Estimated 6.42 GB 0.02 GB left (tight) Speed 21–28
  • Spark-X2.6-4B Q8_0
    Context length 16,384 Estimated 5.90 GB 0.54 GB left Speed 26–34
  • Gemma-4-E4B-Uncensored-HauhauCS-Aggressive IQ4_XS Image input
    Context length 8,192 Estimated 6.32 GB 0.12 GB left (tight) Speed 22–29
  • Qwen3-VL-8B-Instruct-abliterated Q3_K_M
    Context length 8,192 Estimated 6.41 GB 0.04 GB left (tight) Speed 27–36

Frequently asked questions

Which local LLMs run on Arc A380?
On this site, 473 models have a quantization that fits in 6 GB at 8,192 tokens (estimated). Popular ones include Ornith-1.5-9B, gemma-4-12B-it, Qwen3.5-9B, MOSS-Music-8B-Instruct, Qwen3-VL-8B-Instruct.
What is the largest model that runs on Arc A380?
Among the models listed here, Qwen3.8-Flash-Next-FP4-Local-Strata (about 51.2B); BF16 fits in 6 GB (estimated at 8,192 tokens).
How fast does Arc A380 generate text?
With 186 GB/s of memory bandwidth, Ornith-1.5-9B at Q2_K_L (5.06 GB) runs at an estimated 22–29 tokens per second (60–80% of bandwidth ÷ file size; prompt processing not included).