Qwen 3.8 and 3.6 GPU requirements: what each model needs to run

Qwen3.8-27B needs one GPU with 80 GB or more at full precision, and a 48 GB card holds its FP8 and 4-bit builds. The BF16 weights are 55.56 GB, Qwen's own FP8 checkpoint is 30.87 GB and the Ollama tag is 18 GB. Qwen3.6-27B has the same size, and Qwen3.6-35B-A3B is a little larger. The other two Qwen 3.8 models are a different class: Qwen3.8-Flash-Next is 185.52 GB in FP8, a four-GPU model, and Qwen3.8-2.4T-A95B, the open model behind Qwen3.8-Max, is 2,496 GB in FP8, which takes two 8-GPU B300 servers.

Which Qwen models have open weights#

These are the eight open-weight Qwen language models people search for now. Every figure below is from Qwen's own model cards and the files in each repository:

Qwen3.8-Max itself is not a download. Qwen's model card calls it "the official version based on Qwen3.8-2.4T-A95B", served through Qwen's hosted API with vision input and a non-thinking mode. The open 2.4T weights are text-only, and thinking cannot be switched off. In the same way, the hosted Qwen3.8-Flash is based on Qwen3.8-Flash-Next, which Qwen describes as an experimental preview of the architecture that will underpin Qwen4.

Every model here except the two 27B models is a mixture-of-experts model, and all of its experts sit in GPU memory even though each token uses a few of them. Size these models on total parameters and file size, never on the active count.

The licence differs by model#

The 27B models, the Qwen 3.6 family and the coder models are Apache-2.0, which allows commercial use. For Qwen3-Coder-Next that comes from its model card, since its repository holds no licence file. The two largest Qwen 3.8 models carry custom licences, and both add two conditions to an otherwise permissive grant.

The first condition is the same in both: above 100,000,000 monthly active users or US$20,000,000 of monthly revenue, the model name must be displayed in the product. The second is where they differ. Under the Qwen Community License 1.0 of Flash-Next, a licensee that "conducts a Model as a Service or AI Work Assistant business" shall "obtain a separate license from Qwen" before using the model for any commercial purpose, at any size. Under the Qwen3.8-Max License of the 2.4T model, that applies only when "the aggregate revenue of the licensee and its affiliates exceeds US$50,000,000" during any consecutive twelve months. The licences define an AI Work Assistant as an independent product built mainly for AI-assisted coding or office work, so a coding assistant counts. Both exempt internal use from the second condition, as long as nothing is made available to third parties. If you plan to sell API access to either model, read the licence file before you size hardware.

File sizes at BF16, FP8 and 4-bit#

The weights are the fixed cost, and Qwen publishes FP8 builds of every 3.6 and 3.8 model next to the BF16 originals. These are the safetensors sizes on Hugging Face, with the 4-bit column from the Ollama library because Qwen ships no 4-bit build of the 3.6 and 3.8 models:

The arithmetic checks out for the 27B: 27.78B parameters at 2 bytes are 55.6 GB (our calculation). The FP8 file is larger than 27.78 GB because 3.08B of the parameters, the vision encoder among them, stay in BF16. Qwen's FP8 builds are block-scaled, with one scale per 128 x 128 block, and vLLM runs that format as real FP8 math only on Hopper and Blackwell cards. On an RTX A6000, an A100 or an L40S it still loads and still saves the memory, but computes in 16 bits, as the vLLM quantization guide explains.

KV cache per token#

Qwen's hybrid attention keeps the cache small for a model of this size. Only one layer in four is a full-attention layer whose cache grows with the context, and the other three are Gated DeltaNet layers with a fixed-size state. The formula from the KV cache guide is 2 x full-attention layers x KV heads x head dimension x 2 bytes, and for Qwen3.8-27B that is 2 x 16 x 4 x 256 x 2 = 65,536 bytes per token (our calculation, 16-bit cache):

An FP8 cache, --kv-cache-dtype fp8 in vLLM, halves every figure. Flash-Next adds a small index for its sparse attention on top. The native context of all of them is 262,144 tokens. Qwen documents a YaRN setting that stretches it to about a million, and warns that the static YaRN the open-source engines implement can hurt shorter prompts, so I would leave it off unless you need it. Qwen's 3.6 cards also advise keeping at least 128K tokens of context, because the models use it for their thinking.

Which QuantaCloud GPU holds which model#

The table shows what is left for the cache after the weights, and how many tokens of 16-bit cache that is. The budget is 92% of the memory each card reports in nvidia-smi, vLLM's default, in GiB (our calculation):

Read each cell as the pool that all running requests share. These are upper bounds, before activations and CUDA graphs, and vLLM prints the real figure in its GPU KV cache size log line. The 48 GB column covers the RTX A6000, RTX 6000 Ada, L40 and L40S and assumes ECC is on. With ECC off an RTX A6000 reports 49,140 MiB, which adds 2.8 GiB.

Three things follow. A 48 GB card holds the 27B models in FP8 with one 131,072-token conversation, which takes 8 GiB of the 12.6 GiB left. One 80 GB A100 holds them at full precision with one full 262,144-token conversation, which takes 16 GiB of the 21.9 GiB left. And the 96 GB RTX PRO 6000 is the card I would pick for a small team: it leaves 36.2 GiB of cache at full precision, and it is on the list of GPUs vLLM's recipe has verified for Qwen3.8-27B, which the A100 and the 48 GB cards are not. Two 48 GB cards also hold the BF16 weights on size, with 31.0 GiB left across the pair, and the vLLM multi-GPU guide covers that setup.

Prices checked 13 Nov 2026, 23:26 UTC

Qwen3.8-Flash-Next and the 2.4T model need more than one GPU#

Flash-Next in FP8 is 185.52 GB, 172.78 GiB, more than any single GPU QuantaCloud rents. vLLM's recipe lists 4x RTX PRO 6000 as a verified configuration, and that VM has a budget of 351.8 GiB, which leaves 179.0 GiB after the weights (our calculation). Two H200 NVL cards hold the weights with 85.6 GiB to spare on size alone, though the recipe's H200 setup uses all eight GPUs of a server. Launch either only when the catalog lists it: multi-GPU RTX PRO 6000 and H200 NVL configurations are not always available. The BF16 files, 335.28 GiB, belong on an 8-GPU server.

The 2.4T model is built-to-order hardware and nothing less. Its FP8 files are 2,324.68 GiB. One 8-GPU B300 server has a budget of 1,976.9 GiB, so it takes sixteen B300 GPUs, two servers, with about 1,629 GiB left for the cache (our calculation). On H200 the arithmetic says 18 GPUs, but vLLM's recipe puts it at 32, four 8-GPU servers, because it sizes with headroom and the tensor-parallel size has to divide the model's 64 attention heads. For the BF16 files, 4,556.37 GiB, the recipe asks for 24 B300 GPUs. The recipe also lists a third-party NVFP4 build of 1.32 TiB that fits one 8-GPU B300 server. Multi-server serving needs a fast network between the nodes, which InfiniBand vs NVLink explains.

We build servers and clusters like these to order. Send a capacity brief with the model, the precision and the traffic you expect, and the configuration, lead time and terms come back in writing. B300 servers and GPU clusters describe the hardware.

Run Qwen3.8-27B with vLLM#

vLLM is one of the engines Qwen's model card points to. Its recipe for Qwen3.8-27B gives 0.17.0 as the minimum version, and the setup is the one in the vLLM Docker guide: the Bare Metal template, the driver check that picks the image tag, and an API key in VLLM_API_KEY. On an A100 80 GB or an RTX PRO 6000:

docker run -d --name vllm --gpus all --ipc=host \
  -p 127.0.0.1:8000:8000 \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  "vllm/vllm-openai:$VLLM_TAG" \
  Qwen/Qwen3.8-27B \
  --max-model-len 131072 \
  --reasoning-parser qwen3 \
  --enable-auto-tool-choice --tool-call-parser qwen3_coder \
  --api-key "$VLLM_API_KEY"

On a 48 GB card, change the model to Qwen/Qwen3.8-27B-FP8, and if startup runs out of memory, lower --max-model-len to 65536. Of these cards, only the RTX PRO 6000 is on the recipe's verified list. The A100 and the 48 GB cards are a fit on size only, and the recipe's FP8 build names Hopper, data center Blackwell and RTX 5090 cards, none of them Ampere or Ada, so read the first startup log closely. The same flags serve Qwen3.6-27B and Qwen3.6-35B-A3B. Qwen 3.8 thinks by default, and --reasoning-parser qwen3 returns that thinking separately from the answer. Add --language-model-only to skip the vision encoder when you only send text, which frees memory for the cache. The repositories are not gated, so you need no Hugging Face token. Stopping the instance deletes its disk, so each new launch downloads the weights again.

Run Qwen with Ollama#

The Ollama library has a tag for every model here except the 2.4T. The sizes are the library's own:

The 27B and 35B tags fit one 48 GB card with room for context: at 64 KiB per token, a 32,768-token conversation adds 2 GiB to the 18 GB of qwen3.8:27b (our calculation). qwen3-coder-next at 52 GB needs an 80 GB card. Ollama chooses its default context from the GPU's memory, and Ollama context length shows how to set it yourself. The quickest way to try a tag is the Open WebUI + Ollama template, where you type the tag into the model selector and pull it. Open WebUI with Ollama walks through the template.

Qwen hardware FAQ#

How much VRAM does Qwen3.8-27B need?

About 52 GiB for the BF16 weights, 29 GiB for Qwen's FP8 checkpoint and 18 GB for the Ollama tag, plus 64 KiB of cache per token of context. That means an 80 GB or 96 GB GPU at full precision and a 48 GB GPU at FP8 or 4-bit.

Is Qwen3.8-Max open source?

The hosted Qwen3.8-Max is not downloadable. Qwen published the model it is based on, Qwen3.8-2.4T-A95B, with open weights under the Qwen3.8-Max License. Those weights are 2,496 GB in FP8 and need sixteen B300 GPUs.

Do Qwen 3.8 and Qwen 3.6 27B need different hardware?

No. Both have 27,781,427,952 parameters by Hugging Face's count and the same layer layout, and their BF16 and FP8 files are the same size to within a few hundred bytes. A GPU that runs one runs the other.

Which Qwen coder model fits one GPU?

Qwen3-Coder-30B-A3B-Instruct in FP8, 31.18 GB, fits a 48 GB card. Qwen3-Coder-Next is 80.38 GB in FP8, which needs a 96 GB RTX PRO 6000 or a 141 GB H200 NVL, and its 48.41 GB GGUF needs an 80 GB card. Local models for coding agents covers how to connect one to an agent.

Is Qwen free for commercial use?

The 27B models, Qwen 3.6 and the coder models are Apache-2.0. Flash-Next and the 2.4T model add the conditions above for model-as-a-service and work-assistant businesses.

The rule I follow for Qwen is to start with Qwen3.8-27B and pick the precision by the card: FP8 on 48 GB, BF16 on 80 GB or 96 GB. Move to Flash-Next only when you have four large GPUs and have read its licence, and treat the 2.4T model as a cluster project. The LLM VRAM calculator checks a context length and a number of users against every GPU configuration, and the best open-source LLMs compares Qwen with the other families.