Out-of-memory errors, or very slow

It does not fit in VRAM. Use a smaller quantization or a shorter context. In llama.cpp, a q8_0 KV cache reduces memory for long contexts. Check required memory on the model page.

"unknown model architecture" when loading

The llama.cpp inside the app is too old for the new model format. Update the app. Each model page shows which llama.cpp version first supports it.

Strange output or repeated words

Very small quantizations (2–3 bits) can lose a lot of quality. Try one size larger. The quantization guide has a quality guide.

Can I run a model with no GPU or not enough VRAM?

Yes. llama.cpp, Ollama and LM Studio can keep the layers that do not fit on the GPU in system memory (RAM) and compute them on the CPU (or run everything on the CPU if there is no GPU). Generation becomes much slower, and speed is set mostly by RAM bandwidth. You need RAM for the file size plus the KV cache and so on. The memory figures on this site assume the whole model is on the GPU, and we do not estimate CPU speed.

A large model does not run on a Mac

macOS limits how much memory the GPU can use, so not all unified memory is available. Leave headroom over the required memory.

Guide

How to run local LLMs (Ollama, LM Studio, llama.cpp)

GGUF files published on Hugging Face run in Ollama, LM Studio and llama.cpp. This page walks through choosing a model, running it in each of the three apps, and common problems. Command examples use Qwen3-8B at Q4_K_M.

Last updated Oct 19, 2026

Before you start: choose a model and quantization

Check how much VRAM your GPU has (unified memory on a Mac).
On the model's page on this site, find a quantization that fits. Required memory grows with conversation length (context length), so look at the column for the length you will use.
Note the source (Hugging Face repository name) and the quantization name (such as Q4_K_M). The commands below use these two.

Which app to use

All of them load GGUF files with the same llama.cpp engine inside.

App	Good for
Ollama	Starting with a single command, and using models from other apps or programs through an API.
LM Studio	Searching, downloading and chatting entirely through a graphical interface.
llama.cpp	Fine-grained settings, and the mainline project that supports new model formats first.

Run with Ollama

Install the app from ollama.com. GGUF files on Hugging Face run directly by giving the repository and quantization names. The first run downloads the file.

ollama run hf.co/Qwen/Qwen3-8B-GGUF:Q4_K_M

To change the context length (conversation length), type the following in the chat. Longer contexts need more memory.

/set parameter num_ctx 8192

While it runs, other apps can use it through the API at http://localhost:11434. Ollama documentation · Hugging Face × Ollama

Run with LM Studio

Install the app from lmstudio.ai.
In the model search screen (Discover), paste the Hugging Face repository name (for example unsloth/Qwen3-8B-GGUF).
From the list of quantized files, choose the quantization (for example Q4_K_M) and download it.
In the settings when loading the model, set the context length and how much to put on the GPU (GPU Offload).

LM Studio documentation

Run with llama.cpp

Install a prebuilt release from GitHub, or use a package manager (brew on Mac, winget on Windows, etc.). Give llama-server the repository and quantization with -hf; it downloads the file and starts. Then open http://localhost:8080 in a browser for a chat screen.

llama-server -hf Qwen/Qwen3-8B-GGUF:Q4_K_M -c 8192

Common options

Option	Meaning
-c 8192	Context length. Longer contexts need more memory for the KV cache.
-ngl 99	Number of layers on the GPU. All layers is fastest; if it does not fit in VRAM, lower it to run part on the CPU (slower).
-fa on	Use Flash Attention. Helps memory use and speed.
-ctk q8_0 -ctv q8_0	Store the KV cache in 8 bits to reduce memory for long contexts.

llama.cpp documentation

Troubleshooting

Out-of-memory errors, or very slow: It does not fit in VRAM. Use a smaller quantization or a shorter context. In llama.cpp, a q8_0 KV cache reduces memory for long contexts. Check required memory on the model page.
"unknown model architecture" when loading: The llama.cpp inside the app is too old for the new model format. Update the app. Each model page shows which llama.cpp version first supports it.
Strange output or repeated words: Very small quantizations (2–3 bits) can lose a lot of quality. Try one size larger. The quantization guide has a quality guide.
Can I run a model with no GPU or not enough VRAM?: Yes. llama.cpp, Ollama and LM Studio can keep the layers that do not fit on the GPU in system memory (RAM) and compute them on the CPU (or run everything on the CPU if there is no GPU). Generation becomes much slower, and speed is set mostly by RAM bandwidth. You need RAM for the file size plus the KV cache and so on. The memory figures on this site assume the whole model is on the GPU, and we do not estimate CPU speed.
A large model does not run on a Mac: macOS limits how much memory the GPU can use, so not all unified memory is available. Leave headroom over the required memory.