H100 PCIe vs H200 NVL

The H200 NVL is worth its premium when the model and its context need more than 80 GB, or when someone waits on every token, and the H100 PCIe is the better buy for batch work that fits. Both are Hopper GPUs with compute capability 9.0 and FP8 Tensor Cores, and their dense FP8 rates are within 10% of each other: 1,670.5 TFLOPS on the H200 NVL against 1,513 on the H100 PCIe. The difference is memory. The H200 NVL has 141 GB of HBM3e at 4.8 TB/s against 80 GB of HBM2e at 2.0 TB/s, which is 1.76 times the capacity and 2.4 times the bandwidth. On 2026-09-27 a 1x H200 NVL cost $3.43 an hour and a 1x H100 PCIe $2.59, 1.32 times as much (today the console price and the console price).

H100 PCIe and H200 NVL specs side by side#

The two columns differ on three lines that matter: memory, bandwidth and power. QuantaCloud rents the H100 as the PCIe card and the H200 as the NVL card, so those are the versions in the table.

The H100 PCIe figures come from NVIDIA's H100 architecture whitepaper and H100 PCIe product brief, and the H200 NVL figures from NVIDIA's H200 product page and datasheet. NVIDIA quotes the H200's Tensor Core throughput with sparsity, so the table halves those figures to compare dense with dense.

The compute gap is small and the memory gap is large (our calculation). The H200 NVL has 1.10 times the dense FP8 rate (1,670.5 / 1,513), 1.11 times the BF16 rate (835.5 / 756) and 1.17 times the FP64 rate (30 / 25.6). It has 1.76 times the memory (141 / 80) and 2.4 times the bandwidth (4.8 / 2.0), for a power limit that is 1.7 times as high.

NVIDIA's own H100 comparison is narrower than this one. Its H200 page says the H200 has nearly double the capacity of the H100 with 1.4 times the memory bandwidth, which matches the H100 SXM module at 3.35 TB/s, not the PCIe card (4.8 / 3.35 = 1.43). The same page reports Llama 2 70B inference at 1.9 times the H100's throughput, measured on SXM modules with the H100 at a batch size of 8 and the H200 at 32. Against the PCIe card that QuantaCloud rents by the hour, the bandwidth step is the larger 2.4 times.

Prices per GPU-hour#

The H200 NVL cost 32% more per GPU-hour on 2026-09-27, and the live table shows today's prices.

Prices checked 14 Nov 2026, 00:42 UTC

That day a 1x H100 PCIe listed at $2.59 an hour and a 1x H200 NVL at $3.43 (today the console price and the console price). Divide the premium by what it buys and the H200 comes out cheaper on memory and bandwidth and dearer on math (our calculation: the price ratio of 1.32 divided by each spec ratio):

Neither card is always in the catalog. I use the 2026-09-27 prices because that is the most recent snapshot I have with both listed, and on 2026-11-12 the catalog listed neither. So read every recommendation here as "when the catalog lists it", and check the table above before you plan a run. The H100 page and the H200 page show the live configurations, and the pricing page covers billing. When a job cannot wait for on-demand capacity, an H100 or H200 server built to order is the other path: send a capacity brief.

What fits on one card: 80 GB against 141 GB#

The working budgets are 73.3 GiB on the H100 PCIe and 129.2 GiB on the H200 NVL. vLLM reserves 92% of GPU memory by default, the cards report 81,559 and 143,771 MiB in nvidia-smi, and weights, KV cache and activations share that budget (our calculation: MiB / 1,024 x 0.92).

The KV cache is what fills the difference. Each token of context costs 2 x layers x KV heads x head size x 2 bytes at BF16, the formula from the KV cache guide. Take Llama-3.3-70B: 80 layers and 8 KV heads of 128 dimensions make 2 x 80 x 8 x 128 x 2 = 327,680 bytes, 320 KiB, for every token. Its per-channel FP8 checkpoint is 72.67 GB, which is 67.68 GiB. On the H100 PCIe that leaves 73.28 - 67.68 = 5.6 GiB, about 18,000 tokens of context across every request. On the H200 NVL it leaves 129.17 - 67.68 = 61.5 GiB, about 201,000 tokens: six conversations of 32,768 tokens at once, or one full 131,072-token context, which takes 40 GiB, with 21.5 GiB to spare.

The same arithmetic for five checkpoints, all sizes read from Hugging Face on 2026-11-12:

Three patterns come out of that table. A model near 70 GiB loads on both cards and is only useful on the H200: the FP8 build of Llama-3.3-70B has 11 times the context there (61.5 / 5.6). A model near 60 GiB serves on both, and the H200 holds about 5.5 times the requests (68.4 / 12.5 GiB for gpt-oss-120b). A model past 129 GiB needs two cards either way. Every figure assumes a BF16 cache and leaves out activations, so treat it as a ceiling, and --kv-cache-dtype fp8 doubles the token counts on both cards.

Fine-tuning splits the same way. QLoRA on a 70B model needs 41 GB in Unsloth's table and QLoRA on gpt-oss-120b needs 65 GB, so both fit either card. 16-bit LoRA on a 32B model needs 76 GB, which is tight on the H100's 80 GB and comfortable on the H200. 16-bit LoRA on a 70B model needs 164 GB, which is two H200 NVL cards. The fine-tuning VRAM guide covers other sizes.

When the H200 NVL pays for itself#

The premium pays in three cases, and in each one the reason is memory.

The first is a job over 73 GiB. Without the H200 it needs two H100s, and on 2026-09-27 a 2x H100 PCIe VM cost $5.18 an hour against $3.43 for one H200 NVL, so the single card held the model for 34% less (our calculation: 1 - 3.43 / 5.18). Today the two list at the console price and the console price. One card also means no model split across GPUs.

The second is one fast stream. NVIDIA describes token-by-token generation as memory-bound, so for a person or an agent loop reading tokens as they arrive, bandwidth sets the ceiling. The H200 NVL has 2.4 times the bandwidth for 1.32 times the price on 2026-09-27, so on paper each token costs 55% as much and arrives sooner. This is the case people miss: for interactive use, the dearer card is the cheaper one per token.

The third is many long contexts at once. vLLM 0.33.0 gives both cards the same batch defaults, 1,024 concurrent sequences and 8,192 batched tokens per step, so the KV cache is what caps concurrency. With gpt-oss-120b that cap is 2.8 full-length conversations on the H100 PCIe and 15.2 on the H200 NVL.

When the H100 PCIe is the better buy#

The H100 wins when the job fits in 73 GiB and keeps the Tensor Cores busy. Offline batch inference, embedding jobs and QLoRA runs are mostly bound by Tensor Core math, where the H200 NVL is 10% faster and was 32% dearer on 2026-09-27, so the same batch costs about 20% more on the H200 (our calculation: 1.32 / 1.10 = 1.20). A 32B model in a per-channel FP8 checkpoint, 34.3 GB, is the typical case: it leaves 41.3 GiB of the H100's budget for KV cache.

The H100 also wins the idle hour. A notebook left open or a server waiting for traffic costs the full hourly price on either card, and on 2026-09-27 that was $2.59 against $3.43 (today the console price and the console price).

FAQ#

Is the H200 faster than the H100?

For generating tokens, yes on paper, by up to 2.4 times against the H100 PCIe, because that work is bound by memory bandwidth. For batched Tensor Core math the two are close: the H200 NVL's dense FP8 rate is 1.10 times the H100 PCIe's.

Are these the SXM versions?

No. QuantaCloud rents the H100 as the PCIe card, with 80 GB of HBM2e, and the H200 as the NVL card, a 600 W PCIe card with the same 141 GB as the SXM module. The SXM versions come on 4 and 8-GPU HGX boards, and we build those servers to order as reserved capacity.

Can two H100s replace one H200?

For capacity, yes: a 2x H100 PCIe VM has a 146.6 GiB budget against 129.2 GiB on one H200 NVL, and it holds Llama-3.3-70B in BF16, which one H200 does not. It cost $5.18 an hour on 2026-09-27 against $3.43 (today the console price and the console price), and the model has to be split across the two cards. Tensor parallelism in vLLM covers how, and the link between the cards decides how well it runs.

How much of the 141 GB can I plan with?

About 129 GiB. vLLM takes 92% of the memory the card reports by default, and weights, KV cache and activations all come out of that. The same rule gives about 73 GiB on the H100 PCIe.

Should I buy one instead of renting?

That turns on how many hours a year the GPU would be busy. The H100 price guide and the H200 price guide work through card prices against hourly rates.

What happens to my data when I stop the VM?

It is deleted. Stopping an instance terminates it and deletes its disk, and there are no volumes or snapshots, so copy checkpoints and outputs off first. The unused seconds of the current hour go back to your balance, as the pricing page explains.

The rule I follow: add up the weights, the KV cache for your context and the requests you expect in flight. Under about 73 GiB with the GPU busy on batches, rent the H100 PCIe. Over it, or when a person waits on every token, rent the H200 NVL, when the catalog lists it. If neither is listed, the 96 GB RTX PRO 6000 Blackwell sits between them, and RTX PRO 6000 vs H100 and RTX PRO 6000 vs H200 cover it. A100 vs H100 covers the cheaper 80 GB card, and H200 vs B200 covers the step after the H200.