A100 vs H100 PCIe

Rent the A100 for 16-bit work that fits in 80 GB and spends its hours waiting on people, and the H100 PCIe for FP8 serving, long prompts and training. Both cards carry 80 GB of HBM2e and read it at about 2 TB/s, so nothing changes about what fits or how fast one stream of tokens comes out. What Hopper adds is math: FP8 Tensor Cores, which the A100 does not have, and 756 dense BF16 TFLOPS against 312. On 2026-09-27 a 1x H100 PCIe cost $2.59 an hour and a 1x A100 SXM4 $1.50, 1.73 times as much (today the console price and $1.50/GPU-hr). The H100 clears that bar on Tensor Core work and misses it on bandwidth, and that split decides which one to rent.

A100 and H100 PCIe specs side by side#

The two cards match on memory and differ on compute. The table has both 80 GB A100s that QuantaCloud rents, the SXM4 module and the PCIe card, next to the H100 PCIe, which is the H100 in the on-demand catalog.

The A100 figures come from NVIDIA's A100 datasheet, the H100 PCIe figures from NVIDIA's H100 architecture whitepaper and its H100 PCIe product brief, and compute capability from NVIDIA's CUDA GPU list. NVIDIA also prints Tensor Core figures with sparsity, at twice these values, and the table keeps to the dense half in every column.

Three ratios carry the comparison (our calculation). The H100 PCIe has 2.42 times the A100's dense 16-bit Tensor Core rate (756 / 312) and about 2.6 times its FP32 and FP64 rates (51.2 / 19.5 and 25.6 / 9.7). With an FP8 checkpoint the gap is 4.85 times, because the A100 has no FP8 and multiplies at its 16-bit rate (1,513 / 312). NVIDIA's whitepaper shows where the 4.85 comes from: 114 streaming multiprocessors against 108, twice the Tensor Core rate in each one, twice again for FP8, and a 1,620 MHz clock against 1,410 (114 / 108 x 2 x 2 x 1,620 / 1,410 = 4.85). Memory bandwidth is the ratio that does not move: 2,000 GB/s against 2,039 on the A100 SXM4 and 1,935 on the A100 PCIe.

What Hopper adds: FP8 and the Transformer Engine#

The H100's main addition is FP8, an 8-bit floating-point format that halves the bytes per value and doubles Tensor Core throughput against 16-bit. FP8 Tensor Core math needs compute capability 8.9 or newer. The H100 is 9.0 and the A100 is 8.0, so no driver or library update brings it to the A100. The FP8, FP16 and BF16 guide covers the formats themselves.

The Transformer Engine is how NVIDIA puts FP8 to work. Its H100 whitepaper describes it as software plus Hopper Tensor Core technology that chooses between FP8 and 16-bit for each layer of a transformer and handles the scaling between the two. The same whitepaper claims up to 9 times faster AI training and up to 30 times faster AI inference on large language models than the A100. Those are NVIDIA's best cases for the H100 generation. The whitepaper's own summary of peak compute is more modest, about 6 times an A100 for the 132-multiprocessor SXM module, and for the PCIe card the paper figure is the 4.85 above.

Today the Transformer Engine is also an open-source NVIDIA library for training. Version 2.21.0, released on 2026-11-10, lists FP8 training on Hopper, Ada, Blackwell and Rubin GPUs, and FP16 and BF16 optimizations on Ampere and later. So the same training script runs on both cards, and only the H100 can move its linear layers to FP8.

Serving follows the same line. In vLLM 0.33.0, released on 2026-11-02, a per-channel FP8 checkpoint runs with FP8 weights and FP8 activations from compute capability 8.9. On the A100 the same file loads weight-only: the weights take half the memory of BF16, and the multiplications still run at the 16-bit rate. vLLM also sizes its batches by GPU. On a GPU with at least 70 GiB and under 160 GiB whose name does not contain "a100", such as the H100 PCIe, its API server defaults to 1,024 concurrent sequences and 8,192 batched tokens per step. On an A100 it uses 256 and 2,048, and a comment in the code says a large batched-token limit reduces A100 throughput.

Two smaller Hopper items matter to some jobs. FlashAttention-3 lists an H100 or H800 as a requirement and FlashAttention-4 targets Hopper and Blackwell, while FlashAttention-2 runs on both generations. PCIe Gen5 also doubles the host link, to 128 GB/s.

What Hopper does not add on this card is memory or bandwidth. The H100 PCIe uses the same HBM2e as the A100 80GB. The H100 with HBM3 at 3.35 TB/s is the SXM module, which ships on 4 and 8-GPU HGX boards and is built to order here.

Prices per GPU-hour#

The A100 hour cost 42% less on 2026-09-27, and the live table shows today's prices.

Prices checked 14 Nov 2026, 10:39 UTC

That day a 1x A100 SXM4 listed at $1.50 an hour, the A100 PCIe at $1.475 per GPU-hour in a 2-GPU VM and a 1x H100 PCIe at $2.59 (today $1.50/GPU-hr, the console price and the console price). The H100 costs less per job whenever it finishes at least 1.73 times as fast as the A100 SXM4 (our calculation: 2.59 / 1.50 = 1.73). I use the 2026-09-27 prices because that is the most recent snapshot I have with both cards listed: on 2026-11-12 the A100 prices were unchanged and the H100 PCIe was not in the catalog. The catalog moves with capacity, so when the table shows no H100, the A100 is the 80 GB card you can launch, and an H100 build goes through a capacity brief. The A100 page and the H100 page list the live configurations, and the pricing page explains how a partly used hour is refunded.

Price per unit of work#

The H100 is the cheaper card for Tensor Core work and the dearer one for memory-bound work. NVIDIA describes token-by-token generation as memory-bound and the prompt pass as parallel compute, so each kind of work follows a different row of the spec table. At the 2026-09-27 prices, the cost of the same work is the price ratio of 1.73 divided by the speed ratio (our calculation):

Read the last column as dollars. A batch job that costs $1.00 on the A100 costs about $0.71 on the H100 in BF16 and $0.36 with an FP8 checkpoint, and a single stream that costs $1.00 on the A100 costs $1.76 on the H100. These are paper ratios from peak rates. Real jobs mix the rows, since every request has a prompt pass and a generation pass, and an idle GPU costs the full hour on either card.

Which to rent for inference, fine-tuning and training#

For inference, the number of requests in flight decides it. One person or one agent loop reading tokens as they arrive is bound by bandwidth, which is the same on both cards, so the A100 gives about the same speed for 42% less at the 2026-09-27 prices. A server that keeps many requests in flight, or reads long prompts for retrieval, spends its time in batched Tensor Core math, where the H100 does 2.42 times the work in BF16 and 4.85 times with a per-channel FP8 checkpoint. My rule: with an FP8 checkpoint and a busy server, rent the H100 when the catalog lists it or have an H100 server built, and for everything else that fits, start on the A100. The LLM inference page covers the serving setup on either card.

For fine-tuning, memory is the same, so the choice is time against idle hours. QLoRA on a 70B model needs 41 GB in Unsloth's table, and 16-bit LoRA on a 32B model needs 76 GB, which is tight on either card. Training steps are Tensor Core math, so on paper the H100 finishes a BF16 run in 41% of the time for 71% of the cost at the 2026-09-27 prices (our calculation: 312 / 756 = 0.41, and 0.41 x 1.73 = 0.71). The A100 wins the hours a fine-tune spends not training: debugging a data loader, waiting on an evaluation, or sitting idle overnight. I would debug on the A100 and move the long run to the H100 when the catalog lists it. The fine-tuning VRAM guide has the memory figures for other models.

For training across several GPUs, the shape of the catalog decides before the chip does. On 2026-09-27 the A100 SXM4 came as an 8-GPU VM with 640 GB of GPU memory for $11.92 an hour, and the H100 PCIe stopped at two per VM (today the A100 SXM4 starts at $1.50/GPU-hr). Eight A100s in one VM is the on-demand route for a job that shards across eight GPUs, when the catalog lists it. Eight H100s in one server is the HGX H100 platform with SXM modules, which we build to order as reserved capacity: send a capacity brief. FP8 training with the Transformer Engine is a reason to ask for Hopper or newer in that brief.

What fits in 80 GB on each card#

Nothing fits on one of these cards that does not fit on the other. vLLM reserves 92% of GPU memory by default, which gives a budget of 73.6 GiB on the A100 and 73.3 GiB on the H100 PCIe (our calculation from the 81,920 and 81,559 MiB that nvidia-smi reports).

The KV figures assume a BF16 cache, 320 KiB per token for Llama-3.3-70B, and leave out activations, so treat them as ceilings. The KV cache guide has the formula. When a model needs more than 80 GB on one card, H100 vs H200 and A100 vs H200 cover the 141 GB step.

FAQ#

Does the A100 support FP8?

Not in hardware. FP8 Tensor Core math needs compute capability 8.9 and the A100 is 8.0. vLLM still loads a per-channel FP8 checkpoint on it, weight-only, which halves the memory of the weights and keeps the math at the 16-bit rate.

Is an H100 faster than an A100 for running a chat model?

For one conversation, barely, on paper. Generating tokens one at a time is bound by memory bandwidth, and the H100 PCIe reads memory at 2,000 GB/s against 2,039 on the A100 SXM4. The H100 pulls ahead when the server handles many requests at once or reads long prompts, where its Tensor Cores do 2.42 times the work in BF16.

Is the H100 on QuantaCloud the SXM or the PCIe version?

The PCIe version: 80 GB of HBM2e at 2.0 TB/s with a 350 W limit. The SXM version, with HBM3 at 3.35 TB/s, comes on HGX boards and is a reserved build. The H100 page compares the two.

Can I get eight of either in one VM?

Eight A100 SXM4 GPUs in one VM has been an on-demand configuration, at $11.92 an hour on 2026-09-27 (today the A100 SXM4 starts at $1.50/GPU-hr), so rent it when the catalog lists it: the live table on the A100 page shows today's configurations. On 2026-09-27 the H100 PCIe came one or two per VM. Eight H100s in one server is an HGX H100 build, which QuantaCloud builds to order as reserved capacity, with the configuration, lead time and terms confirmed in writing.

Should I buy one of these instead of renting?

That depends on how many hours a year the card would be busy. The A100 price guide and the H100 price guide work through purchase prices against hourly rates, and the A100 is past end of sale, which changes the answer for that card.

What happens to my data when I stop the VM?

It is deleted. Stopping an instance terminates it and deletes its disk, and there are no volumes or snapshots, so copy checkpoints and outputs off first. The unused seconds of the current hour go back to your balance, as the pricing page explains.

The rule I follow for this pair: the memory is the same, so pay the H100's premium only for math. If the job is one stream of tokens, a notebook or a fine-tune you are still debugging, rent the A100. If it is an FP8 model under load, long prompts or a training run that keeps the GPU busy, rent the H100 PCIe when the catalog lists it, and measure tokens per second on your own prompts before you commit to either. The deploy guide covers launching and connecting.