Skip to content
localllm
日本語

Discrete GPU

Arc A370M

Data as of

Memory bandwidth
112 GB/s
Source
Checked:

With 4 GB

One entry per model, up to 30, sorted by popularity. For each model we show the largest (most precise) quantization that fits in this memory with a context of at least 8K tokens. The context length is the longest one that quantization fits (for models whose maximum is under 8K, such as embedding models, that maximum). Models that are not for chat or text generation are labelled with their use. "Left" is the memory minus the estimate; under 5% of the memory is marked "tight" (not a guarantee that it fits).

  • mxbai-embed-large-v1 F32 Embedding (not for chat)
    Context length 512 Estimated 2.41 GB 1.88 GB left Speed 50–67
  • MOSS-Music-8B-Instruct F16 Speech / audio
    Context length 8,192 Estimated 3.78 GB 0.51 GB left Speed 39–52
  • Qwen3.5-4B UD-Q4_K_XL Image input
    Context length 8,192 Estimated 4.28 GB 0.02 GB left (tight) Speed 23–31
  • Wan2.1 Q4_K_M
    Context length 4,096 Estimated 4.22 GB 0.07 GB left (tight) Speed 25–33
  • Qwen3-ASR-1.7B Q8_0 Speech / audio
    Context length 8,192 Estimated 3.97 GB 0.32 GB left Speed 31–41
  • Ternary-Bonsai-2-9B Q8_0 Image input
    Context length 8,192 Estimated 4.04 GB 0.25 GB left Speed 25–33
  • HyperCLOVAX-SEED-Text-Instruct-1.5B-Q4_K_M Q4_K_M
    Context length 16,384 Estimated 3.51 GB 0.79 GB left Speed 59–79
  • Ternary-Bonsai-2-2B Q2_0 Image input
    Context length 131,072 Estimated 3.22 GB 1.07 GB left Speed 123–164
  • Cyber-Ornith-1.5-9B-OBLITERATED IQ1_M
    Context length 8,192 Estimated 4.24 GB 0.05 GB left (tight) Speed 23–31
  • Sharp-Spark-X2.5-4B Q4_K_XL
    Context length 16,384 Estimated 4.20 GB 0.09 GB left (tight) Speed 25–33
  • AdaVaR-3B Q6_K
    Context length 16,384 Estimated 3.99 GB 0.30 GB left Speed 26–35
  • Qwen3-4B-Instruct-2507 UD-Q3_K_XL
    Context length 8,192 Estimated 4.18 GB 0.11 GB left (tight) Speed 32–42
  • Qwen3-VL-4B-Instruct UD-Q3_K_XL Image input
    Context length 8,192 Estimated 4.18 GB 0.11 GB left (tight) Speed 32–42
  • Qwen2.5-7B-Instruct IQ2_M
    Context length 8,192 Estimated 4.10 GB 0.20 GB left (tight) Speed 24–32
  • Qwen3-4B UD-Q3_K_XL
    Context length 8,192 Estimated 4.19 GB 0.11 GB left (tight) Speed 32–42
  • gemma-4-E2B-it Q4_1 Image input
    Context length 8,192 Estimated 4.29 GB 0.00 GB left (tight) Speed 21–28
  • Qwen3.8-4B-Distill Q4_K_M
    Context length 8,192 Estimated 4.15 GB 0.14 GB left (tight) Speed 24–32
  • Spark-X2.5-4B-Heretic-jp Q5_K_M
    Context length 8,192 Estimated 4.20 GB 0.10 GB left (tight) Speed 23–30
  • nomic-embed-text-v1.5 F32 Embedding (not for chat)
    Context length 2,048 Estimated 1.62 GB 2.67 GB left Speed 123–164
  • jinen-v1-small F16
    Context length 1,024 Estimated 1.55 GB 2.74 GB left Speed 152–202
  • Qwen3.8-2B-Distill Q8_0
    Context length 65,536 Estimated 3.95 GB 0.35 GB left Speed 32–43
  • Qwen2.5-1.5B-Instruct F16
    Context length 8,192 Estimated 4.18 GB 0.12 GB left (tight) Speed 22–29
  • Qwen3.5-2B UD-Q8_K_XL Image input
    Context length 16,384 Estimated 4.10 GB 0.19 GB left (tight) Speed 24–32
  • bge-large-en-v1.5 F32 Embedding (not for chat)
    Context length 512 Estimated 2.41 GB 1.88 GB left Speed 50–67
  • Qwen3.5-0.8B-Artemis Q5_K_M Image input
    Context length 131,072 Estimated 3.25 GB 1.05 GB left Speed 118–157
  • bge-small-en-v1.5 F32 Embedding (not for chat)
    Context length 512 Estimated 1.21 GB 3.09 GB left Speed 503–671
  • MiniCPM5-2B Q8_0
    Context length 16,384 Estimated 4.19 GB 0.11 GB left (tight) Speed 25–33
  • Qwen2.5-0.5B-Instruct F16
    Context length 8,192 Estimated 2.22 GB 2.08 GB left Speed 53–71
  • bge-base-en-v1.5 F32 Embedding (not for chat)
    Context length 512 Estimated 1.51 GB 2.78 GB left Speed 154–205
  • gemma-4-E2B-it-Artemis Q3_K_L Image input
    Context length 131,072 Estimated 4.20 GB 0.09 GB left (tight) Speed 29–39

Frequently asked questions

Which local LLMs run on Arc A370M?
On this site, 277 models have a quantization that fits in 4 GB at 8,192 tokens (estimated). Popular ones include Ornith-1.5-9B, MOSS-Music-8B-Instruct, Qwen3.5-4B, Qwen3-4B, Ternary-Bonsai-2-9B.
What is the largest model that runs on Arc A370M?
Among the models listed here, Qwen3.8-Flash-Next-FP4-Local-Strata (about 51.2B); BF16 fits in 4 GB (estimated at 8,192 tokens).
How fast does Arc A370M generate text?
With 112 GB/s of memory bandwidth, Ornith-1.5-9B at IQ1_M (2.81 GB) runs at an estimated 24–32 tokens per second (60–80% of bandwidth ÷ file size; prompt processing not included).