Kimi K3 hardware requirements: what it takes to run the open weights

Kimi K3 needs about 1,454 GiB of GPU memory for its weights alone, which is one 8-GPU B300 server or sixteen H200 or B200 GPUs. No single GPU and no VM you can rent by the hour holds the released weights. They are public: Moonshot put them on Hugging Face on 2026-07-27 as 1,560.94 GB of files with the experts in 4 bits, under its own Kimi K3 License. The Moonshot model that does fit hardware you can launch today is Kimi Linear 48B.

What Moonshot released#

Moonshot's Hugging Face page holds five models that matter here. The figures are from its model cards and the files in each repository:

Kimi K3 is a mixture-of-experts model with 896 experts, of which each token uses 16 plus 2 shared ones. It has 93 layers: 69 use Kimi Delta Attention, a linear attention with a fixed-size state, and 24 use a gated form of multi-head latent attention. It takes text and images, and its model card says it understands video as well. Thinking is always on. Moonshot trained it with quantization in the loop, so the release format is the 4-bit one: MXFP4 weights with MXFP8 activations. Moonshot publishes no full-precision original.

Kimi K3.1 Pro is the same size. Moonshot describes it as post-trained from K3, with the 24 full-attention layers replaced by sparse attention, so the weight sizing on this page applies to it too.

The Kimi K3 licence#

The Kimi K3 License reads like MIT with two commercial conditions added, and both are tied to size. The first covers anyone who "operates a Model as a Service business": when "the aggregate revenue of the Licensee and its affiliates exceeds 20 million US dollars" over any consecutive 12 months, the licensee "must enter into a separate agreement with Moonshot AI" before commercial use. The licence defines Model as a Service as giving a third party access to inference or fine-tuning, for example through an API, in a way that lets it "exercise meaningful control over the inputs, parameters, or training data", and excludes end-user products that only embed the model in their features.

The second is attribution. A commercial product with more than 100 million monthly active users, or more than 20 million US dollars in monthly revenue, must display "Kimi K3" prominently in its interface. Neither condition applies to internal use that exposes nothing to third parties. Below those thresholds the grant is broad: use, modify, fine-tune, deploy and sell. Kimi K2.6 and K2.7 Code are under a Modified MIT licence that carries only the attribution condition, and Kimi Linear is plain MIT.

Can you run Kimi K3 locally#

The honest answer is no, not on GPUs you can put under a desk or rent by the hour, and the arithmetic is short. 2.8 trillion parameters at half a byte would be 1,400 GB. The files are 1,560.94 GB, because the attention layers, the shared experts and the vision encoder stay in 16 bits and the 4-bit blocks carry scales. That is 1,453.74 GiB, and every expert has to be in GPU memory, so the 104B active parameters do not make it smaller.

The largest GPU QuantaCloud rents by the hour, the H200 NVL, reports 140.4 GiB. The K3 weights are 10.4 times that. This is how they compare with whole servers, at vLLM's default of 92% of the reported memory (our calculation):

So the model needs about 1,454 GiB, 1.42 TiB, of aggregate GPU memory before any cache. It does not fit one 8-GPU H200 or B200 server, and it does fit one 8-GPU B300 server. vLLM's team, which shipped support for K3 on the day the weights came out, says the same in its launch post: "The entire model can barely fit in a single NVIDIA DGX B300 and requires a minimum of 16 NVIDIA B200/GB200 GPUs to serve on that hardware generation." Take the word barely seriously. Our 523.1 GiB is arithmetic on file sizes, before activations, the cache and the engine's own overhead.

The cache is modest next to the weights. Only the 24 latent-attention layers grow with the context, at 24 x (512 + 64) x 2 = 27,648 bytes per token by the formula in the KV cache guide, which is 27 GiB for one conversation at the full 1,048,576 tokens (our calculation, 16-bit cache).

H200, B300 and NVL72 terms, and the network between servers#

On H200s, Kimi K3 is a two-server deployment: sixteen GPUs. vLLM's recipe lists the H200 as verified, with a profile that caps the context at 32,768 tokens and five concurrent sequences, so it works but gives up most of the model's long context. On H100s the recipe starts at 32 GPUs.

On B300s it is one 8-GPU server, which is why vLLM calls eight B300 GPUs the easiest way to run it. Inside that server the GPUs talk over NVLink through NVSwitch, and nothing has to cross a network.

In NVL72 terms it is a small part of a rack. A GB300 NVL72 has 72 GPUs in 18 trays of four, all in one NVLink domain, and the K3 weights use 7.9% of the rack's memory budget (our calculation). vLLM's recipe sets the minimum at eight GB300 GPUs, two trays. On a GB200 NVL72 the minimum is sixteen GPUs.

Once the model spans two servers, the network between them sits in the path of every token. vLLM's post says most production deployments of K3 "run multi-node with expert and data parallelism, connected over RDMA or NVLink", which in practice means InfiniBand or RoCE between 8-GPU servers, or the NVLink backplane of an NVL72. InfiniBand vs NVLink and InfiniBand vs Ethernet explain the choice. On-demand instances are single VMs, so a multi-server K3 deployment is a cluster we build to order: send a capacity brief with the model and the traffic you expect, and the configuration, lead time and terms come back in writing. B300 servers, GB300 racks and GPU clusters describe the hardware.

What serves Kimi K3#

Moonshot's model card recommends three engines: vLLM, SGLang and TokenSpeed. vLLM's recipe gives 0.29.0 as the minimum version and tells you to use its vllm/vllm-openai:kimi-k3 image, a CUDA 13 build that needs an R580 or newer NVIDIA driver. The K3 architecture is also registered in the stable 0.30.0 and 0.33.0 releases. This is the command from vLLM's launch post, for eight B300 GPUs:

vllm serve moonshotai/Kimi-K3 \
  --tensor-parallel-size 8 \
  --trust-remote-code \
  --load-format fastsafetensors \
  --enable-prefix-caching \
  --enable-auto-tool-choice \
  --tool-call-parser kimi_k3 \
  --reasoning-parser kimi_k3

Kimi K3 is not an Ollama model you can download. The Ollama library has a kimi-k3 page with a single tag, kimi-k3:cloud, which runs on Ollama's servers.

GGUF files do exist, from the community and not from Moonshot. Unsloth publishes K3 quantizations for llama.cpp, which added the K3 text model in August 2026. They do not change the answer. K3 already ships at 4 bits, so the 4-bit GGUF is 1,508.67 GB, barely smaller than the original. The smaller files get there by going to 2 bits, 861.28 GB, or lower, down to 466.37 GB, and that costs quality. The smallest 1-bit files, 466.37 GB to 594.04 GB, are small enough on size for the largest on-demand VM, 8x A100 SXM4 with a budget of 588.8 GiB, when the catalog lists it, and llama.cpp can also leave experts in system memory with --cpu-moe, which trades the memory problem for a speed problem. I would still not judge a 2.8T model by a 1-bit copy of it.

Kimi K3.1 Pro is newer than its tooling. On 2026-11-12, vLLM's recipe for it named vLLM 0.34.0 as the minimum, while 0.33.0 was the latest release, so it runs from the dedicated kimi-k31 images for now. The recipe's baseline is the same eight GPUs: one B300 server or two GB300 trays.

Smaller Kimi models that fit less hardware#

Two older Moonshot models need far less, and one of them runs on a VM you can launch.

Kimi K2.6 and K2.7 Code are 1T-parameter models that ship in 4-bit INT4, 595.18 GB, which is 554.30 GiB. Moonshot's deployment guide serves K2.6 "on a H200 single node with TP8", and the arithmetic agrees: an 8-GPU H200 server has a budget of 1,033.4 GiB, which leaves 479.1 GiB for the cache (our calculation). That is still a built-to-order server, but one server instead of two.

Kimi Linear 48B is the model where Moonshot introduced Kimi Delta Attention, the attention K3 is built on. It has 48B parameters with 3B active, a 1M-token context and an MIT licence, and its BF16 weights are 98.25 GB, 91.50 GiB. That fits three on-demand configurations on size (our calculation):

Its cache is small, 8,064 bytes per token, or 7.9 GiB for a full 1M-token conversation (our calculation). Moonshot's own command uses four GPUs:

vllm serve moonshotai/Kimi-Linear-48B-A3B-Instruct \
  --port 8000 \
  --tensor-parallel-size 4 \
  --max-model-len 1048576 \
  --trust-remote-code

On two A100s, set --tensor-parallel-size 2. vLLM's recipe for this model dates from April 2026, pins an old vLLM release and lists the H200 as verified, not the RTX A6000 or the A100, so confirm that the version you run loads it on your GPUs before you build on it. The vLLM Docker guide covers the setup, and the vLLM multi-GPU guide covers splitting a model across cards. Launch the four-GPU VM when the catalog lists it. These are the RTX A6000 configurations the catalog lists right now:

Kimi K3 FAQ#

How much VRAM does Kimi K3 need?

About 1,454 GiB for the weights, plus up to 27 GiB of cache for each full-length conversation. In hardware that is one 8-GPU B300 server, sixteen H200 or B200 GPUs, or eight GPUs of a GB300 NVL72.

Is Kimi K3 open source?

The weights are open: anyone can download them from Hugging Face without a gate. The licence is Moonshot's own Kimi K3 License, not MIT or Apache, and it adds the two commercial conditions described above.

Where are the Kimi K3 weights on Hugging Face?

In the moonshotai/Kimi-K3 repository, as 96 safetensors files that total 1,560.94 GB. Kimi K3.1 Pro is in moonshotai/Kimi-K3.1-Pro.

Is Kimi K3 on Ollama?

Only as kimi-k3:cloud, which Ollama hosts. There is no tag that downloads the model to your GPU.

What do Kimi K2 and K2.6 need?

Kimi K2.6 and K2.7 Code are 595.18 GB and fit one 8-GPU H200 server. The earlier Kimi K2 Thinking is 594.23 GB in the same INT4 format, and Kimi K2 Instruct 0905 is 1,029.19 GB in FP8, which an 8-GPU H200 server holds with about 75 GiB to spare (our calculation).

The rule I follow for Kimi K3 is to decide the server before anything else: one 8-GPU B300 server if you can get it, two 8-GPU H200 or B200 servers on an RDMA network if you cannot. Neither is something to rent by the hour, so the first step is a brief. If what you need is a Moonshot model on a GPU today, start with Kimi Linear, and see the best open-source LLMs for models of other sizes.