Open-weight coding models: which to self-host, and on what GPU

The open-weight coding model I would self-host first is Qwen3.8-27B: it is Apache-2.0, it has a 262,144-token context, its tool calls work in vLLM with the qwen3_coder parser, and at 79.8% it has the highest Terminal-Bench 2.1 score, in Artificial Analysis's own runs, of the models here that fit one GPU. Its FP8 build fits a 48 GB card and its full-precision weights fit an 80 or 96 GB card. The model I would test against it is Nanbeige4.3-33B, which is the only one of them that gets anywhere on the newer Terminal-Bench 4.0. The dedicated coder models have fallen behind the general ones: Qwen3-Coder-Next scores 38.2% on the same run.

The models, with their licences and vLLM flags#

A coding agent needs three things from a model: tool calls the server can parse, a long context, and a licence you can ship under. The table is sorted by the size of the maker's own checkpoint, read from Hugging Face on 2026-11-12.

The flags come from vLLM's parser registry at v0.33.0 and from vLLM's recipe for each model. Where there is no recipe, they come from the maker's own serve command (Devstral, GLM-4.7-Flash, Nanbeige and Qwen3-Coder-Next), or for Qwen3-Coder-30B-A3B-Instruct, whose card has none, from vLLM's tool calling docs. Those docs list the Qwen3-Coder parser as qwen3_xml, and qwen3_coder is a second name for the same parser. The Fits column is our calculation from the file size against 92% of the memory each GPU reports, and open-weight LLMs by GPU memory shows that arithmetic and the licences in full. Five licences in this table carry conditions. Devstral 2's Modified MIT gives no rights to a company with more than $20 million in monthly revenue. Qwen3.8-Flash-Next's licence requires a separate licence from Qwen for any business that sells model access or an AI work assistant, which it defines as a product for AI-assisted coding or office work. Kimi K2.7 Code's Modified MIT asks only that a product with more than 100 million monthly users or $20 million in monthly revenue show the model's name, and the roundup's licence table summarises the Kimi K3.1-Pro and GLM-5.3 licences.

What the public benchmarks say#

The one thing I always check is who ran the benchmark. Artificial Analysis runs Terminal-Bench itself, with the same harness for every model, so its two columns compare like with like. The third column is what each maker published about its own model. All figures were read on 2026-11-12.

Artificial Analysis runs all 89 tasks of Terminal-Bench 2.1 with the Terminus 2 harness and all 66 tasks of Terminal-Bench 4.0 with mini-swe-agent, and reports pass@1 averaged over three runs of each task. The official Terminal-Bench 4.1 leaderboard, updated on 2026-11-11, lists no model that fits one GPU: its open-weight entries are GLM-5.3 at 41.2% and GLM-5.3-Flash at 35.5%, against 65.8% for the top closed model.

What the scores do and do not tell you#

The core problem with coding scores is that the same model gets different numbers from different people. Qwen reports 73.0 for Qwen3.8-27B on Terminal-Bench 2.1, Nanbeige's card lists the same 73.0 for it, and Artificial Analysis gets 79.8%, all three with the Terminus harness. A gap of a few points between two models is inside that noise.

A maker's figure also depends on the harness it chose. DeepSeek's 82.7 for V4 Flash on Terminal-Bench 2.1 was run with its own harness, which it had not released when the card was written. Qwen's SWE-bench Pro figures use the Claude Code harness with a 256K context. None of that is hidden, but it means you cannot line the third column up across rows.

Old benchmarks stop separating models, and new ones are harsh. Six of the eight single-GPU models that Artificial Analysis ran on Terminal-Bench 4.0 score under 1%, including three that pass 40% on version 2.1. SWE-bench Verified has the opposite problem: its own leaderboard has not added an entry since 2026-02-26, so every later figure for it is self-reported. Qwen reports 73.4 on it for Qwen3.6-35B-A3B. SWE-rebench, which draws its problems from a dated window of repositories, has the same model at 24.7% for 2026-05-15 to 2026-07-01, and Qwen3.6-27B at 31.2%.

Three things no leaderboard measures for you. The scores are for the full-precision model or the maker's own format, so a 4-bit build may behave differently. They say nothing about tokens per second on your GPU. And they are not your repository: an hour with your own tasks tells you more than a five-point lead.

Context is the memory cost that decides the GPU#

An agent session fills its context with files, tool output and the model's own reasoning, so the cache matters as much as the weights. Ollama's docs ask for a context of 64k or more for Claude Code and OpenCode. This table is our calculation: the memory left after the weights inside vLLM's default 92%, divided by each model's cache per token.

The totals are shared by every request the server is handling, and they are upper bounds: activations, the vision encoder and vLLM's own workspace come out of the same budget. The Qwen models keep a small cache because only one layer in four uses full attention. Dense models such as Devstral and Nanbeige4.3-33B pay for every layer, which is why Devstral 2 fits a 141 GB H200 NVL but leaves it no room to work. --kv-cache-dtype fp8 halves the cache per token, and the KV cache guide covers what that costs. The 48 GB rows assume ECC is on. vLLM's recipe lists Qwen's FP8 build for Hopper and Blackwell GPUs, so on an Ada or Ampere 48 GB card treat it as untested.

What I would run on 48 GB and on 96 GB#

On a 48 GB card I would run Qwen3.8-27B in FP8 with --max-model-len 131072. One full context of that length takes 8 GiB of the 12.6 GiB that is left, so it is a one-developer server. If the FP8 build gives trouble in vLLM on that card, Ollama's qwen3.8:27b tag is an 18 GB quantized build of the same model.

On a 96 GB RTX PRO 6000 I would run Qwen3.8-27B in BF16, with room for two full 262,144-token sessions. vLLM's recipe marks the RTX PRO 6000 as verified hardware for this model and carries a BF16 profile for it. Then I would test Nanbeige4.3-33B in FP8 against it on my own repository, because it scores 29.8% where Qwen3.8-27B scores 5.6% on Terminal-Bench 4.0. It is a new model from a smaller lab, its repository is under three weeks old, and it has no vLLM recipe yet, so I would not start there.

Prices checked 14 Nov 2026, 09:38 UTC

Running a coding agent against your own GPU has the serve command and the settings for Claude Code, OpenCode and Qwen Code. Past one GPU, the models at the bottom of the first table need four or eight GPUs: two H200 NVLs or four RTX PRO 6000s are on-demand configurations when the catalog lists them, and anything larger is reserved capacity that we order and build to your spec. Send a capacity brief with the model and the number of developers.

Questions about open-weight coding models#

What is the best local LLM for coding?

On one GPU, by independent scores, it is Qwen3.8-27B: 79.8% on Terminal-Bench 2.1 in Artificial Analysis's run, Apache-2.0, and small enough in FP8 for a 48 GB card. Nanbeige4.3-33B is the one to watch, with 29.8% on Terminal-Bench 4.0.

What is the best Ollama model for coding?

Ollama's library has qwen3.8:27b at 18 GB, qwen3-coder:30b at 19 GB, devstral-small-2:24b at 15 GB and qwen3.6:35b at 23 GB, all of which fit a 48 GB card. I would start with qwen3.8:27b for the reasons above, and set the context to 64k or more: Ollama context length shows how.

Is Qwen3-Coder still worth running?

Less than it was. Qwen3-Coder-Next scores 38.2% on Terminal-Bench 2.1 and 0.0% on 4.0 in Artificial Analysis's runs, against 79.8% and 5.6% for Qwen3.8-27B, and it needs a 96 GB card where Qwen3.8-27B fits 48 GB. Qwen3-Coder-30B-A3B-Instruct is the smaller one people search for, at 31.18 GB in FP8, and I found no independent score for it on the leaderboards above.

Can I use Devstral 2 commercially?

Only if your company's monthly revenue is $20 million or less. Above that, Mistral's Modified MIT licence gives you no rights and you need a commercial licence from Mistral. Devstral Small 2 is Apache-2.0 with no such limit.

How much context does a coding agent need?

Ollama's integration docs ask for 64k or more for Claude Code and OpenCode. I would plan for 131,072 tokens per developer, which costs 8 GiB of cache on Qwen3.8-27B and about 26 GiB on Nanbeige4.3-33B.

My decision rule: choose by an independent score on a current benchmark, then check that the context you need fits beside the weights, then read the licence. Today that gives Qwen3.8-27B on either card, with Nanbeige4.3-33B as the model to test against it on 96 GB. The LLM VRAM calculator checks the fit for your own context and number of developers.