moonshotai
Kimi-K2-Instruct
- Parameters
- 1,026.4B
- Max context length
- 131,072
- Architecture
- deepseek2
- License
- Conditional
- Base model
- moonshotai/Kimi-K2-Instruct
- License
- modified-mit
About Kimi-K2-Instruct
Kimi-K2-Instruct is a GGUF model based on moonshotai/Kimi-K2-Instruct on Hugging Face. GGUF files are published by unsloth/Kimi-K2-Instruct-GGUF. It has about 1,026.4B parameters, the deepseek2 architecture, a maximum context length of 131,072 tokens (values written in the GGUF).
There are 27 quantizations, with file sizes from 244 GB (UD-TQ1_0) to 2,053 GB (BF16) (actual sizes on Hugging Face).
Running Q4_K_M (621 GB) with an 8,192-token context needs an estimated 622 GB of memory, and an estimated 624 GB at 32,768 tokens. The smallest, UD-TQ1_0 (244 GB), needs an estimated 245 GB at 8,192 tokens.
The original model is licensed under Modified MIT License (Moonshot AI's Kimi), which allows commercial use with conditions such as scale or purpose. Check the original license for the terms of use.
File size and estimated memory per quantization
- Up to 8 GB
- Up to 16 GB
- Up to 24 GB
- Up to 48 GB
- Up to 128 GB
- More
File sizes are the actual values on Hugging Face. Memory is estimated for each context length (KV cache f16). The formula is in How memory is calculated.
All memory figures are Estimated (KV cache f16). It should run on a setup with at least this much VRAM (or Mac unified memory). Click a value to see the devices and models that fit that amount (Find by GPU / Mac).
About the "quality guide" (how far a quantization drifts from the original)
For each quantization type, how far the output drifts from the original model, grouped into six levels from values llama.cpp measured on Llama-3-8B.
The value is mean KLD (the difference between the original and quantized model in next-token probabilities); smaller means closer to the original. Quantizations made with an imatrix (importance data) come out slightly smaller. Levels use the value without an imatrix (or with one, for types that only have that).
These values were not measured on this model. Larger models and MoE models tend to drift less with the same type. Distributor-specific mixed types (such as UD-Q4_K_XL) cannot be matched by type name alone, so they show "-".
| Quantization | Quality guide | KLD (no imatrix) | KLD (imatrix) | Size (Llama-3-8B) |
|---|---|---|---|---|
| Q8_0 | Nearly identical | 0.0014 | - | 7.96 GiB |
| Q6_K | Very small | 0.0055 | - | 6.14 GiB |
| Q5_K_M | Very small | 0.0108 | - | 5.33 GiB |
| Q5_K_S | Small | 0.0166 | - | 5.21 GiB |
| Q5_1 | Small | 0.0180 | - | 5.65 GiB |
| Q5_0 | Small | 0.0222 | - | 5.21 GiB |
| Q4_K_M | Small | 0.0313 | 0.0282 | 4.58 GiB |
| Q4_K_S | Medium | 0.0431 | 0.0320 | 4.37 GiB |
| IQ4_NL | Small | - | 0.0357 | 4.35 GiB |
| IQ4_XS | Small | - | 0.0363 | 4.14 GiB |
| Q4_1 | Medium | 0.0717 | - | 4.78 GiB |
| Q4_0 | Medium | 0.0719 | - | 4.34 GiB |
| Q3_K_L | Medium | 0.0872 | 0.0731 | 4.03 GiB |
| Q3_K_M | Medium | 0.1019 | 0.0844 | 3.74 GiB |
| IQ3_M | Medium | - | 0.1025 | 3.53 GiB |
| IQ3_S | Medium | - | 0.1113 | 3.42 GiB |
| IQ3_XS | Large | - | 0.1387 | 3.28 GiB |
| IQ3_XXS | Large | - | 0.1836 | 3.05 GiB |
| Q3_K_S | Large | 0.2282 | 0.1998 | 3.41 GiB |
| IQ2_M | Large | - | 0.3260 | 2.74 GiB |
| Q2_K | Very large | 0.4451 | 0.3322 | 2.96 GiB |
| Q2_K_S | Very large | - | 0.4034 | 2.96 GiB |
| IQ2_S | Very large | - | 0.4392 | 2.56 GiB |
| IQ2_XS | Very large | - | 0.5463 | 2.43 GiB |
| IQ2_XXS | Very large | - | 0.8120 | 2.24 GiB |
| IQ1_M | Very large | - | 1.3931 | 2.01 GiB |
| IQ1_S | Very large | - | 2.2546 | 1.88 GiB |
Source: llama.cpp (MIT), measured on Llama-3-8B against FP16, Wikitext-2, revision f364eb6f. Original table · Reading and choosing quantizations
Last checked: Oct 5, 2026
- Architecture
- deepseek2
- Supported before llama.cpp b4409
- Parameters
- 1,026.4B
- Max context length
- 131,072
- License
- modified-mit Conditional
| Quantization | File size Measured | Quality guide | 4,096 | 8,192 | 16,384 | 32,768 | 65,536 | 131,072 max |
|---|---|---|---|---|---|---|---|---|
| UD-TQ1_0 (5 parts) | 244 GB | - | 245 GB | 245 GB | 246 GB | 247 GB | 249 GB | 254 GB |
| UD-IQ1_S (6 parts) | 280 GB | - | 281 GB | 282 GB | 282 GB | 283 GB | 286 GB | 290 GB |
| UD-IQ1_M (7 parts) | 304 GB | - | 305 GB | 306 GB | 306 GB | 307 GB | 310 GB | 314 GB |
| UD-IQ2_XXS (7 parts) | 329 GB | - | 330 GB | 330 GB | 331 GB | 332 GB | 334 GB | 339 GB |
| UD-IQ2_M (8 parts) | 347 GB | - | 348 GB | 349 GB | 349 GB | 350 GB | 353 GB | 357 GB |
| Q2_K (8 parts) | 373 GB | Very large | 374 GB | 375 GB | 375 GB | 376 GB | 379 GB | 383 GB |
| Q2_K_L (8 parts) | 373 GB | - | 375 GB | 375 GB | 375 GB | 377 GB | 379 GB | 384 GB |
| UD-Q2_K_XL (8 parts) | 382 GB | - | 383 GB | 383 GB | 384 GB | 385 GB | 387 GB | 392 GB |
| UD-IQ3_XXS (9 parts) | 417 GB | - | 418 GB | 418 GB | 419 GB | 420 GB | 422 GB | 427 GB |
| Q3_K_S (10 parts) | 442 GB | Large | 444 GB | 444 GB | 444 GB | 446 GB | 448 GB | 453 GB |
| UD-Q3_K_XL (10 parts) | 452 GB | - | 453 GB | 454 GB | 454 GB | 455 GB | 458 GB | 462 GB |
| Q3_K_M (11 parts) | 489 GB | Medium | 491 GB | 491 GB | 491 GB | 493 GB | 495 GB | 499 GB |
| IQ4_XS (12 parts) | 547 GB | Small | 548 GB | 548 GB | 549 GB | 550 GB | 552 GB | 557 GB |
| IQ4_NL (12 parts) | 578 GB | Small | 580 GB | 580 GB | 581 GB | 582 GB | 584 GB | 589 GB |
| Q4_0 (13 parts) | 581 GB | Medium | 582 GB | 582 GB | 583 GB | 584 GB | 586 GB | 591 GB |
| Q4_K_S (13 parts) | 583 GB | Medium | 584 GB | 584 GB | 585 GB | 586 GB | 588 GB | 593 GB |
| UD-Q4_K_XL (13 parts) | 587 GB | - | 588 GB | 589 GB | 589 GB | 590 GB | 593 GB | 597 GB |
| Q4_K_M (13 parts) | 621 GB | Small | 622 GB | 622 GB | 623 GB | 624 GB | 626 GB | 631 GB |
| Q4_1 (13 parts) | 643 GB | Medium | 644 GB | 644 GB | 645 GB | 646 GB | 648 GB | 653 GB |
| Q5_K_S (15 parts) | 707 GB | Small | 708 GB | 708 GB | 709 GB | 710 GB | 712 GB | 717 GB |
| Q5_K_M (16 parts) | 728 GB | Very small | 730 GB | 730 GB | 730 GB | 732 GB | 734 GB | 738 GB |
| UD-Q5_K_XL (16 parts) | 731 GB | - | 732 GB | 732 GB | 733 GB | 734 GB | 736 GB | 741 GB |
| Q6_K (18 parts) | 843 GB | Very small | 844 GB | 844 GB | 845 GB | 846 GB | 848 GB | 853 GB |
| UD-Q6_K_XL (19 parts) | 879 GB | - | 880 GB | 881 GB | 881 GB | 882 GB | 885 GB | 889 GB |
| Q8_0 (23 parts) | 1,091 GB | Nearly identical | 1,092 GB | 1,093 GB | 1,093 GB | 1,094 GB | 1,097 GB | 1,101 GB |
| UD-Q8_K_XL (25 parts) | 1,190 GB | - | 1,191 GB | 1,192 GB | 1,192 GB | 1,193 GB | 1,196 GB | 1,200 GB |
| BF16 (45 parts) | 2,053 GB | Original precision | 2,054 GB | 2,055 GB | 2,055 GB | 2,056 GB | 2,059 GB | 2,063 GB |
Ollama ollama run hf.co/unsloth/Kimi-K2-Instruct-GGUF:Q4_K_M
llama.cpp llama-server -hf unsloth/Kimi-K2-Instruct-GGUF:Q4_K_M
License and commercial use
Modified MIT License (Moonshot AI's Kimi) Conditional
MIT License plus one condition. If a commercial product or service has more than 100 million monthly active users or more than US$20 million in monthly revenue, the model name (Kimi K2, Kimi K2.6, Kimi K2.7 Code, etc.) must be prominently displayed in its interface. Otherwise it is like MIT: keep the copyright notice and license text. We checked the original text of Moonshot AI's Kimi K2, K2.6 and K2.7 Code; other authors using the same name may have different terms.
Based on the license on the original model's Hugging Face page. Original license text (checked Nov 1, 2026)
The summary states facts this site read from the original text and is not legal advice. Always check the original license and its latest terms before use. All licenses
How to run
Example commands using Q4_K_M. To use another quantization, replace the name with one from the table above. How to run local LLMs
Ollama
Run it by pointing directly at the GGUF on Hugging Face.
ollama run hf.co/unsloth/Kimi-K2-Instruct-GGUF:Q4_K_M
llama.cpp
Pass the repository and quantization name to llama-server (or llama-cli) with -hf.
llama-server -hf unsloth/Kimi-K2-Instruct-GGUF:Q4_K_M
LM Studio
Paste unsloth/Kimi-K2-Instruct-GGUF into the search box, download it, and choose the quantization from the file list.
Other models from moonshotai
-
2,779.5B
moonshotai
Kimi-K3
- UD-Q1_0 size
- 466 GB
- Memory at 8K
- 468 GB
Measured Estimated
-
1,026.4B
moonshotai
Kimi-K2.7-Code
- UD-IQ1_M size
- 304 GB
- Memory at 8K
- 305 GB
Measured Estimated
-
1,026.4B
moonshotai
Kimi-K2.6
- UD-Q2_K_XL size
- 340 GB
- Memory at 8K
- 342 GB
Measured Estimated
-
49.1B
moonshotai
Kimi-Linear-48B-A3B-Instruct
- Q4_K_M size
- 30.1 GB
- Memory at 8K
- 31.0 GB
Measured Estimated VRAM 32 GB+
-
72.7B
moonshotai
Kimi-Dev-72B
- UD-IQ1_S size
- 23.0 GB
- Memory at 8K
- 26.6 GB
Measured Estimated VRAM 32 GB+
Models of a similar size (other developers)
Frequently asked questions
- How much memory do I need to run Kimi-K2-Instruct (Q4_K_M)?
- The file size is 621 GB (measured). With a 8,192-token context, the estimated memory including the KV cache is 622 GB. The formula is published at https://localllm.az-arte.com/en/guide/memory-calculation.
- Can Kimi-K2-Instruct be used commercially?
- The original model's license is Modified MIT License (Moonshot AI's Kimi) (Commercial use with conditions).MIT License plus one condition. If a commercial product or service has more than 100 million monthly active users or more than US$20 million in monthly revenue, the model name (Kimi K2, Kimi K2.6, Kimi K2.7 Code, etc.) must be prominently displayed in its interface. Otherwise it is like MIT: keep the copyright notice and license text. We checked the original text of Moonshot AI's Kimi K2, K2.6 and K2.7 Code; other authors using the same name may have different terms. Check the original license text before deciding.
- Does Kimi-K2-Instruct run in 8 GB, 12 GB, 16 GB or 24 GB of VRAM?
- 8 GB: no quantization fits; 12 GB: no quantization fits; 16 GB: no quantization fits; 24 GB: no quantization fits (estimated at 8,192 tokens with an f16 KV cache, files from the most-downloaded source).
- What is the command to run Kimi-K2-Instruct with Ollama?
- Run "ollama run hf.co/unsloth/Kimi-K2-Instruct-GGUF:Q4_K_M" to use the GGUF on Hugging Face directly. To use another quantization, replace the name at the end.