GPU guides
Build something with your GPU.
Step-by-step guides for running ComfyUI, vLLM, Ollama and fine-tuning on cloud GPUs, plus sizing guides for GPU memory and prices.
AI server cost in 2026: what the hardware costs to buy, build or rent
What an AI server costs in late 2026, from a $4,999 desk machine to $550,000 HGX servers and racks in the millions, with power, cooling and break-even.
Batch inference with vLLM on a cloud GPU
Run offline batch inference with vLLM's LLM class: a runnable Python script, how continuous batching and prefix caching work, and the knobs for throughput.
Chat with your documents in Open WebUI (RAG) on your own GPU
Chat with your documents in Open WebUI on a GPU VM: knowledge bases, the embedding model and its settings, GPU memory, and saving the knowledge base.
Check your GPU, NVIDIA driver and CUDA version
nvidia-smi shows the newest CUDA your driver supports, not what is installed. Check driver, toolkit and PyTorch versions, then fix the usual mismatches.
ComfyUI and Stable Diffusion GPU requirements: VRAM by model
The GPU for Stable Diffusion and ComfyUI depends on the model: the VRAM SD 1.5, SDXL, FLUX, Qwen-Image and Wan 2.2 need, and what offloading changes.
ComfyUI with more than one GPU: what works and what does not
ComfyUI on two or more GPUs: one instance per GPU, the native MultiGPU nodes, ComfyUI-Distributed and Raylight, and two 48 GB cards or one 96 GB.
Connect to a cloud GPU with SSH, VS Code and port forwarding
Connect to a cloud GPU as ubuntu with one ~/.ssh/config entry, open it in VS Code Remote-SSH and tunnel Jupyter, ComfyUI, vLLM and Ollama.
CUDA_VISIBLE_DEVICES: choose which GPUs a process can use
CUDA_VISIBLE_DEVICES hides GPUs and renumbers the rest from 0. Why the numbers differ from nvidia-smi, UUIDs, one job per GPU, Docker, vLLM and Ollama.
DeepSeek GPU requirements: what runs on 48 to 141 GB GPUs
What DeepSeek needs to run: real file sizes for the R1 distills and the 671B models, what fits one 48 to 141 GB GPU, what needs more, and the licences.
DeepSeek V4 GPU requirements: Flash, Pro and what it takes to self-host
What DeepSeek V4-Flash, V4-Pro and V4.1-Flash need to self-host: file sizes, GPU memory, the multi-GPU setups that hold them, vLLM, GGUF and Ollama.
DeepSpeed ZeRO on one multi-GPU VM: stages, memory and configs
DeepSpeed ZeRO stages and offload, memory per GPU from the ZeRO paper and DeepSpeed's estimator, configs, launching on 2 to 8 GPUs, and when to use FSDP.
Deploy SGLang with Docker on a GPU server
Run SGLang 0.5.23 from its Docker image: driver check, pinned tag, sglang serve, API key, the memory and context flags, quantized models and metrics.
Deploy vLLM with Docker on a GPU server
Serve an open model behind an OpenAI-compatible API with vLLM's Docker image: driver check, pinned tag, Compose file, API key and which GPU fits.
DGX Spark price, and what the same money rents in cloud GPUs
A DGX Spark costs $6,950 with 128 GB since October 2026, up from $3,999 at launch. What it runs, where it is slow, and the cloud GPU hours it equals.
Download Hugging Face models fast on a GPU server
Use the Hugging Face CLI's hf download with Xet, a token for gated models and file filters, then script it for every fresh GPU instance.
Edit images with Qwen-Image-Edit in ComfyUI on a cloud GPU
Qwen-Image-Edit 2509, 2511 and Qwen-Image-2.1 in ComfyUI: what changed, licences, file sizes, FP8 or INT8, editing with several images, LoRAs and GPUs.
Fine-tune an LLM with Unsloth on a cloud GPU (QLoRA, step by step)
Unsloth QLoRA fine-tuning on one RTX A6000: install, train Gemma 4 E4B, push the LoRA adapter to Hugging Face before the VM stops, and size bigger runs.
Fine-tune gpt-oss-20b on a single cloud GPU
LoRA fine-tuning of gpt-oss-20b with TRL and PEFT on one 80 GB GPU: memory by recipe, the harmony format, the script, and serving the result with vLLM.
Fix "CUDA out of memory" (and know when you need a bigger GPU)
Read the CUDA out of memory error, find what holds GPU memory, apply fixes in order of cost from batch size to a bigger GPU, and fix vLLM's startup errors.
Fix vLLM out-of-memory errors
Fix vLLM out-of-memory errors: the five startup errors in vLLM 0.30.0, what each memory flag trades, and worked maths for 48 and 80 GB GPUs.
FP8 vs FP16 vs BF16: choosing precision for training and inference
Bits, range and precision of FP32, TF32, FP16, BF16, FP8 and FP4, which NVIDIA GPUs compute each in hardware, and which to use for training and inference.
Gemma 4 GPU requirements: sizes, VRAM and what to run it on
Gemma 4 in five sizes, E2B to 31B: parameters, file sizes, context window, Google's own memory table, KV cache and which GPU holds each on vLLM and Ollama.
Generate video with Wan 2.2 in ComfyUI on a cloud GPU
Run Wan 2.2 video in ComfyUI: the 5B and A14B files and sizes, VRAM with and without offloading, resolution and length settings, and which GPU to rent.
Get models into ComfyUI on a fresh cloud instance
Where each file goes in ComfyUI's models folder, and a script and model list that fetch your set from Hugging Face and Civitai on every new instance.
GGUF explained: the format, the quantization types and which file to download
What a GGUF file holds, what reads it, llama.cpp's quantization types with bits per weight, which file fits a 48, 80 or 96 GB GPU, and how to convert one.
GGUF models in ComfyUI: when to use them, and when not to
ComfyUI-GGUF explained: install the custom node, its loaders and folders, GGUF against FP8 and 16-bit files in memory and speed, and LoRAs.
Google Colab pricing vs renting a GPU by the hour
Colab Pro is $9.99 a month for 100 compute units and Pro+ $49.99 for 600. When renting a 48 GB GPU by the hour costs less, with the arithmetic.
gpt-oss-120b and gpt-oss-20b: GPU requirements
How much GPU memory gpt-oss-120b and gpt-oss-20b need, which GPUs fit them, and how to serve them on H100, H200 NVL, RTX PRO 6000 and RTX A6000.
How much does it cost to fine-tune an LLM on rented GPUs? Worked examples
Estimate a fine-tune's cost: count the tokens, get tokens per second from a published or measured run, turn that into hours, and price them at live rates.
How much VRAM do you need for AI? LLMs, images, video and fine-tuning
How much VRAM you need for LLM inference, image and video generation, and fine-tuning: the formulas, and which workloads fit 48, 80, 96 and 141 GB GPUs.
How much VRAM do you need to fine-tune an LLM?
How much GPU memory QLoRA, LoRA and full fine-tuning need for 7B to 70B models, why published tables disagree, and which GPU setups fit.
How to rent a GPU for AI
Rent a cloud GPU for AI step by step: choose GPU memory, pick a template, add credit, launch, connect over SSH, save your results and stop.
How to run ComfyUI on a cloud GPU
Run ComfyUI on a rented NVIDIA GPU: pick a GPU for your models, launch the template or install it yourself, load models, and save outputs before you stop.
InfiniBand vs Ethernet for AI clusters
InfiniBand and Ethernet both ship at 400 and 800 Gb/s. How each handles packet loss, what RoCE, Spectrum-X and Ultra Ethernet 1.1 are, and how to choose.
InfiniBand vs NVLink: what connects what in a GPU cluster
NVLink links GPUs inside a server at 900 GB/s to 1.8 TB/s per GPU. InfiniBand links servers at 400 or 800 Gb/s per port. What that means for your cluster.
Install ComfyUI-Manager and custom nodes on a remote GPU
Install ComfyUI-Manager on a remote GPU, what its security levels allow, how to pin custom node versions, and how to reinstall your nodes on each launch.
Install flash-attn on a GPU server, and when you do not need it
Whether you still need flash-attn, the prebuilt wheel that matches your torch, CUDA and Python, the source build, the common errors, and FA3 and FA4.
Install llama-cpp-python with CUDA on a GPU server
Fix a CPU-only llama-cpp-python: install 0.3.36 with CUDA on Ubuntu 22.04 from source or a prebuilt wheel, prove the GPU offload and run its server.
Install or update NVIDIA drivers on an Ubuntu GPU server
Install NVIDIA drivers on Ubuntu 22.04 or 24.04 servers from Ubuntu's -server packages or NVIDIA's repo, fix the version mismatch, remove a driver cleanly.
Install vLLM and serve a model with vllm serve
Install vLLM 0.30.0 with uv or pip, match the wheel to your NVIDIA driver (CUDA 13 or cu129), then serve an OpenAI-compatible API with vllm serve.
Kimi K3 hardware requirements: what it takes to run the open weights
Kimi K3 is 1,561 GB of open weights: the licence, why it needs an 8-GPU B300 server or 16 H200s, what serves it, and the smaller Kimi models that fit less.
KV cache explained: how much GPU memory LLM serving really needs
What the KV cache is, the formula to size it, worked numbers for Llama, Qwen, gpt-oss and Kimi, and how it decides between 48, 80, 96 and 141 GB GPUs.
LiteLLM proxy in front of vLLM and Ollama on your own GPU
Run the LiteLLM proxy in Docker as one OpenAI-compatible endpoint for vLLM and Ollama: the version to pin, keys, budgets, fallbacks and request logs.
llama-swap: serve several models from one GPU behind one endpoint
Install llama-swap v272 on an Ubuntu GPU VM, configure two models on a 48 GB card with idle unload and groups, and compare llama-server's router mode.
llama.cpp vs Ollama on a GPU server
Ollama 0.40 runs llama.cpp's llama-server underneath. What still differs: model sources, default context, concurrency, GPU offload control and the API.
LLM quantization formats: GGUF, AWQ, GPTQ, FP8 and bitsandbytes
What quantization does to LLM memory and quality, then each format by the engine that runs it: GGUF, AWQ, GPTQ, FP8, NVFP4, bitsandbytes and EXL3.
MiniMax M3 GPU requirements: memory, configurations and serving
MiniMax M3 has about 428B parameters: 443.75 GB at MXFP8 and 854.18 GB at BF16. The licence, the 8-GPU servers that hold it, vLLM, GGUF and M3.1-Flash.
Move datasets and checkpoints to and from a cloud GPU
Copy data to and from a GPU VM with rsync, scp, sftp, rclone and the Hugging Face Hub, resume broken transfers and save results before the disk is deleted.
Multi-GPU fine-tuning with Axolotl on one 8-GPU VM
Install Axolotl, write a LoRA config, spread it over 8 GPUs with FSDP2 or DeepSpeed, size jobs with Axolotl's table, and push the adapter off the VM.
Multi-GPU training on one node with PyTorch DDP and FSDP
DDP or FSDP2 on one 2 to 8 GPU VM: memory per GPU, a runnable torchrun script, communication without NVLink, and checkpoints that survive the VM.
Neocloud vs hyperscaler: GPU prices and what the difference buys
Single-GPU list prices at AWS, Azure and Google Cloud on 2026-11-12 against a neocloud, plus quotas, egress and what the hyperscaler premium buys.
NVIDIA A100 price: buying vs renting per hour
What an NVIDIA A100 80GB costs to buy now that it is past end of sale, against renting one by the hour, with the break-even arithmetic shown.
NVIDIA DGX B300: specs, price and the HGX alternative
What the NVIDIA DGX B300 is: 8 Blackwell Ultra GPUs, 2.1 TB, 14.5 kW, air-cooled. How it is sold, what it costs, and DGX vs HGX vs MGX explained.
NVIDIA H100 price: buying vs renting per hour
What an NVIDIA H100 costs to buy, from dated sources, against renting an H100 PCIe by the hour, with the break-even arithmetic shown.
NVIDIA H200 price: buying vs renting per hour
What an NVIDIA H200 costs to buy, from dated sources, against renting an H200 NVL by the hour, with the break-even arithmetic shown.
nvidia-smi commands for a GPU server
The nvidia-smi commands worth knowing on a GPU server: read the default table, loop it, query CSV fields, log a run, and check ECC, power and topology.
Ollama context length: the default, how to change it, and what it costs in VRAM
Ollama's default context is 4,096, 32,768 or 262,144 tokens, chosen by GPU memory. How to check it, three ways to set it, and what it costs in VRAM.
Ollama environment variables for a GPU server
The Ollama server settings that matter on a GPU server: defaults in 0.40.16, how to set them under systemd and Docker, and the few worth changing.
Ollama GPU requirements: which models fit 48, 80, 96 and 141 GB
How much GPU memory Ollama models need: library sizes by quantization, the KV cache of Ollama's default context, CPU offload and which GPU fits.
Ollama not using the GPU: how to find out why
Why Ollama runs on the CPU instead of the GPU: read ollama ps and the server log, then check memory, the driver, Docker's GPU flag, variables and num_gpu.
On-premise LLM: what it takes, what it costs, and the dedicated alternative
What running an LLM on premises takes: hardware by model size, power, people and software. Plus the three alternatives, and who sees your data in each.
Open-source image generation models compared: licence, VRAM and which GPU
FLUX.2, FLUX 3 Klein, Qwen-Image, Z-Image, HiDream, Hunyuan Image, Krea 2, SD 3.5 and SDXL compared: licences, file sizes, VRAM and which GPU.
Open-source text-to-speech models compared: licence, VRAM and what each is for
Kokoro, Chatterbox, VibeVoice, Qwen3-TTS, F5-TTS, XTTS, Fish Audio and Orpheus compared: licences, voice cloning, streaming and GPU needs.
Open-source video generation models and the GPUs they need
Wan 2.2, HunyuanVideo 1.5, LTX-2.3, LTX-2.5, Kandinsky 5 and MiniMax H3 compared: parameters, licences, file sizes, VRAM and ComfyUI support.
Open-weight coding models: which to self-host, and on what GPU
Open-weight coding models by GPU memory: licence, context, vLLM tool-calling flags, dated benchmark scores, and what to run on 48 GB and 96 GB cards.
Open-weight LLMs in late 2026, sorted by the GPU memory they need
Open-weight LLMs in November 2026 sorted by the GPU memory they need: parameters, licences, context, file sizes and the GPU that holds each one.
Qwen 3.8 and 3.6 GPU requirements: what each model needs to run
What Qwen3.8-27B, Flash-Next, the 2.4T model behind Qwen3.8-Max, Qwen 3.6 and Qwen3-Coder need: file sizes, KV cache, licences and which GPU holds each.
Reserved vs on-demand GPUs: where the break-even sits
The utilization at which a committed monthly GPU price beats paying by the hour, with a reusable formula, two worked examples and what else changes.
RTX PRO 6000 Blackwell: Workstation, Max-Q and Server Edition compared
The three RTX PRO 6000 Blackwell editions from NVIDIA's datasheets: 600 W or 300 W, 1,792 or 1,597 GB/s, cooling and size, dated prices, and buy or rent.
Run Claude Code, OpenCode or Qwen Code against a model on your own GPU
Serve a tool-calling model with vLLM on a GPU VM and point Claude Code, OpenCode or Qwen Code at it: flags, context cost, key, tunnel and cost per day.
Run ComfyUI in Docker on an Ubuntu GPU server
Build your own ComfyUI Docker image from the official install steps, give it the GPU, keep models on the VM's disk and reach it through an SSH tunnel.
Run ComfyUI workflows from Python with the ComfyUI API
Export an API-format workflow, queue it with POST /prompt from Python, follow progress over the websocket and download the images through an SSH tunnel.
Run Docker containers with NVIDIA GPUs
Install the NVIDIA Container Toolkit, pick GPUs with --gpus, choose CUDA images your driver runs, use GPUs in Compose, keep ports on 127.0.0.1, fix errors.
Run FLUX in ComfyUI on a cloud GPU
Which FLUX model fits a 48, 80 or 96 GB GPU, where each file goes in ComfyUI, the settings that work, and what the FLUX dev and schnell licenses allow.
Run FLUX.2 in ComfyUI on a cloud GPU
FLUX.2 dev, klein 4B and klein 9B in ComfyUI: which have open weights, what each licence allows, file sizes, VRAM on 48 and 96 GB GPUs, and setup steps.
Run llama.cpp with full GPU offload on a cloud GPU
Run llama.cpp on an NVIDIA cloud GPU: the CUDA server image or a source build, full offload with -ngl, GGUF quant choice and an OpenAI-compatible API.
Run LTX-2 and LTX-2.5 in ComfyUI on a cloud GPU
LTX-2, LTX-2.3 and LTX-2.5 in ComfyUI: checkpoints and file sizes, video with audio, the $10M licence threshold, VRAM on each GPU and the setup steps.
Run Ollama in Docker with an NVIDIA GPU
Run the official ollama/ollama image on an NVIDIA GPU: a pinned run command, a Compose file with Open WebUI, GPU checks, updates and two common mistakes.
Run Qwen-Image in ComfyUI on a cloud GPU
Qwen-Image, Qwen-Image-2512 and Qwen-Image-2.1 in ComfyUI: file sizes, fp8 or bf16, the text encoder, VRAM on each GPU, workflow settings and the licences.
Run Stable Diffusion WebUI Forge on a cloud GPU with Docker
AUTOMATIC1111 and the original Forge have stalled, and Forge Neo carries on. Run it in Docker on an Ubuntu GPU VM, on localhost, through an SSH tunnel.
Run Z-Image Turbo in ComfyUI on a cloud GPU
Z-Image Turbo and Z-Image Base in ComfyUI: the Apache-2.0 licence, file sizes, VRAM, the 8-step settings, LoRAs, ControlNet and which GPU to rent.
Self-hosting an LLM: the model, the GPU, the server and the real cost
How to self-host an LLM on a rented GPU: pick the model, GPU and server, lock it down, and compare the monthly cost with a dated API price.
Serve embedding models for RAG with Text Embeddings Inference
Run Hugging Face Text Embeddings Inference 1.9 on a GPU VM: the image tag per GPU, embedding and reranker models, the API, an API key and batching flags.
Set up Open WebUI with Ollama on a cloud GPU
Set up Open WebUI with Ollama on an NVIDIA cloud GPU two ways: the ready template, or a pinned Docker Compose file on Ubuntu. Then save your chats.
Slurm vs Kubernetes for a GPU cluster
Slurm fits training jobs that end and Kubernetes fits services that stay up. GPU support on each, the batch add-ons, Slinky, and what each costs to run.
Speculative decoding in vLLM: when it speeds up generation
What speculative decoding is, the vLLM 0.33 methods with their exact --speculative-config, what each costs in GPU memory, and when it slows a busy server.
Stop paying for idle GPUs: auto-stop a QuantaCloud instance
A watchdog script that reads nvidia-smi and stops an idle QuantaCloud GPU instance through the REST API, run by systemd or cron. Copy results off first.
Stress-test and health-check a GPU server on Linux
Health-check a Linux GPU server before a long run: gpu-burn, NVIDIA DCGM diagnostics, Xid errors, ECC and row remapping, throttling and a PyTorch test.
SXM vs PCIe GPUs and NVLink: what it means for multi-GPU AI
PCIe cards, SXM modules, NVLink bridges and NVSwitch with NVIDIA's bandwidth figures, when the link between GPUs matters, and how to check it on a GPU VM.
The GPUs with the most VRAM, from 24 GB to 288 GB
NVIDIA GPUs ranked by memory, from the 32 GB RTX 5090 to the 288 GB Rubin: what each size really holds, what pools across cards and what you can rent.
Train a FLUX LoRA on a cloud GPU (AI Toolkit or FluxGym)
Train a FLUX.1 [dev] LoRA on a 48 GB cloud GPU with AI Toolkit, and why not FluxGym: dataset, config, memory, copying the LoRA off, and the licence.
Transcribe audio with Whisper on a cloud GPU
Batch-transcribe audio with faster-whisper and Whisper large-v3 on a GPU VM, add speaker labels with WhisperX, and copy transcripts off before you stop.
Upscale video with SeedVR2 in ComfyUI on a cloud GPU
SeedVR2 3B and 7B in ComfyUI: native nodes or the numz custom node, files, the settings that decide VRAM, why 4K wants 48 GB or more, and FlashVSR.
Use Ollama inside ComfyUI for prompt generation on one GPU
Run Ollama next to ComfyUI on one cloud GPU, add the comfyui-ollama node, split GPU memory between them and build a workflow that writes its own prompts.
Use the Ollama API on a remote GPU server
Call the Ollama API on a remote NVIDIA GPU over an SSH tunnel: curl, Python and the OpenAI-compatible endpoint, with port 11434 never exposed.
vLLM on multiple GPUs: tensor and pipeline parallelism
Split a model across 2, 4 or 8 GPUs with vLLM: tensor or pipeline parallelism, the flags, how to check the GPU links, and which VMs to rent.
vLLM quantization: which format to run on which GPU
The quantization formats vLLM 0.30 loads, what AWQ, GPTQ, FP8, NVFP4 and MXFP4 need from the GPU, real checkpoint sizes, and what fits on 48 to 141 GB.
vLLM vs llama.cpp: which server for which job
vLLM vs llama.cpp by design: continuous batching vs slots, safetensors vs GGUF, GPU memory, several GPUs, RAM offload, tool calling and structured output.
vLLM vs Ollama: which one to run on a GPU server
vLLM vs Ollama on one GPU: design, GGUF vs safetensors, batching and concurrency, memory use and OpenAI compatibility, and when to use each.
vLLM vs SGLang: how the two serving engines differ
vLLM vs SGLang from both projects' docs and source: prefix caching, scheduling, structured output, memory flags, security defaults and which to run.
What is a neocloud? How GPU clouds differ from hyperscalers
A neocloud is a cloud provider built around renting GPU compute for AI. How neoclouds differ from hyperscalers, and what to check before you choose one.
What is GPU as a service (GPUaaS)?
GPU as a service means renting GPU compute by the hour or for a term instead of buying hardware. The four models, how pricing works and what to compare.
What is NVFP4? 4-bit inference on NVIDIA Blackwell
NVFP4 is NVIDIA's 4-bit float for Blackwell, with an FP8 scale per 16 values. How it compares with MXFP4 and FP8, real checkpoint sizes and where it runs.
Why GPU utilization is low during training, and how to fix it
Why GPU utilization is low in training: measure it with nvidia-smi, dmon and the PyTorch profiler, then fix data loading, small batches and sync points.