Transcribe audio with Whisper on a cloud GPU

Whisper large-v3, batched through faster-whisper on one GPU, is the quickest way to transcribe a folder of audio, with WhisperX on top when you also need to know who spoke. On QuantaCloud both run on the Bare Metal template, an Ubuntu 22.04 VM with the NVIDIA driver and Docker that you reach over SSH as ubuntu. Pick the smallest GPU on offer: Whisper fills a small part of a 48 GB card, so a bigger one buys speed you pay for by the hour, not memory you need.

The audio and the transcripts stay on your VM. Nothing listens on a network port: transcription is a batch job you start over SSH, so there is no service to lock down.

Pick the smallest GPU on offer#

The smallest GPU on offer is the right one, because Whisper is small and every GPU QuantaCloud rents has at least 48 GB. OpenAI publishes the memory each model needs in its own PyTorch implementation, and faster-whisper, which runs the same weights on the CTranslate2 inference engine, needs less.

faster-whisper's README puts numbers on its own use. Its benchmark transcribed 13 minutes of audio with large-v2, the same size as large-v3, in 63 seconds using 4,525 MB at FP16, and in 17 seconds using 6,090 MB with a batch size of 8, on an 8 GB RTX 3070 Ti. The batched run is about 46 times real time (our calculation: 780 seconds of audio in 17).

Against the 46,068 MiB an RTX A6000 reports with ECC on, that batched run would fill about 13% of the card (our calculation: 6,090 / 46,068). The rest does nothing for a transcription job, which is why I choose the GPU on price. The RTX A6000 had the lowest price in the catalog on 2026-09-27, $0.48 an hour (today: $0.48/GPU-hr). A bigger card only pays off if it is faster by more than it costs: the H100 PCIe was $2.59 an hour that day (today: the console price), 5.4 times the A6000 (our calculation: 2.59 / 0.48), so it would have to transcribe more than 5.4 times as fast to cost less per hour of audio.

One more thing to watch: faster-whisper decodes the audio, detects speech with Silero VAD through onnxruntime and computes the spectrogram on the CPU, not the GPU. The RTX A6000 VMs listed on 2026-09-27 had 6 to 12 vCPUs. On a long queue, keep nvidia-smi and top open side by side, because an idle GPU next to busy CPUs means the CPU is the limit.

What runs the transcription#

Four open-source projects do the work, and all four are maintained. The last two rows are the models this page uses.

faster-whisper has had no release since 1.2.1, and its main branch went quiet for ten months, but it is not abandoned: a fix landed on 2026-09-30, and its maintainer added a second maintainer that day. WhisperX had commits as recently as 2026-09-26. I pin exact versions of both, and the install section shows why.

Check the driver first#

The one thing to check before installing is the NVIDIA driver, because it decides which CTranslate2 you can run. Launch the Bare Metal template (deploying a GPU), connect over SSH (the SSH and VS Code guide sets up the quanta-gpu alias used below) and read it:

nvidia-smi --query-gpu=name,driver_version,memory.total --format=csv

CTranslate2 4.6.3 and newer are built with CUDA 12.8 Update 1, which NVIDIA lists as needing driver 570.124.06 or newer. On an older driver they fail with cudaErrorUnsupportedPtxVersion: the provided PTX was compiled with an unsupported toolchain, as reported in CTranslate2's issue tracker. The fix is ctranslate2==4.6.2, the last release built with CUDA 12.4: add it to the install lines below. Checking the driver and CUDA version explains the numbers nvidia-smi prints.

Install faster-whisper with uv#

A uv virtual environment keeps everything out of the system Python, and the CUDA libraries come as pip packages, so the VM needs no CUDA toolkit:

curl -LsSf https://astral.sh/uv/install.sh | sh
source $HOME/.local/bin/env

mkdir -p ~/whisper && cd ~/whisper
uv venv --python 3.12 --seed --managed-python
source .venv/bin/activate
uv pip install "faster-whisper==1.2.1" "av<19" nvidia-cublas-cu12 "nvidia-cudnn-cu12==9.*"

cat >> .venv/bin/activate <<'EOF'
export LD_LIBRARY_PATH=$(python -c 'import nvidia.cublas.lib as b, nvidia.cudnn.lib as d; print(b.__path__[0] + ":" + d.__path__[0])')
EOF
source .venv/bin/activate
python -c "import ctranslate2; print(ctranslate2.__version__, ctranslate2.get_cuda_device_count())"

Two lines differ from faster-whisper's README, and each works around a known break.

The av<19 pin keeps PyAV, the library faster-whisper decodes audio with, below 19.0.0. PyAV 19.0.0, released on 2026-09-29 for Python 3.12 and newer, removed the metadata_errors argument that faster-whisper 1.2.1 passes when it opens a file, so in an environment like this one transcribing any file path fails. The fix was merged into faster-whisper's main branch on 2026-09-30 and is not in a release yet. PyAV bundles its own FFmpeg, so this route needs no system FFmpeg.

The LD_LIBRARY_PATH line tells CTranslate2 where pip put cuBLAS and cuDNN, and appending it to the environment's activate script sets it every time you activate the environment. The README's version reads each package's __file__, which the current NVIDIA wheels no longer set, so it fails with TypeError: expected str, bytes or os.PathLike object, not NoneType. Reading __path__ works with old and new wheels. CTranslate2 4.6.3 and newer no longer need cuDNN at all, but 4.6.2 does.

The last command prints the CTranslate2 version and the number of GPUs it sees, which should be 1.

Or build a Docker image#

Docker is the other route, and it suits a batch job you rebuild on every new instance. The container gets the GPU through the NVIDIA Container Toolkit: if sudo docker run --rm --gpus all ubuntu nvidia-smi does not print the GPU table, running Docker with a GPU installs it. faster-whisper's own Dockerfile starts from nvidia/cuda:12.3.2-cudnn9-runtime-ubuntu22.04. I use the CUDA 12.8.1 image instead, because current CTranslate2 releases are built with CUDA 12.8. Create the folder with mkdir -p ~/whisper-docker, save this in it as Dockerfile, and save transcribe_folder.py from the next section next to it:

FROM nvidia/cuda:12.8.1-cudnn-runtime-ubuntu22.04
RUN apt-get update && apt-get install -y python3-pip && rm -rf /var/lib/apt/lists/*
RUN pip3 install "faster-whisper==1.2.1" "av<19"
WORKDIR /app
COPY transcribe_folder.py .
ENTRYPOINT ["python3", "transcribe_folder.py"]

mkdir -p ~/transcripts ~/.cache/huggingface
sudo docker build -t whisper-batch ~/whisper-docker
sudo docker run --rm --gpus all --user "$(id -u):$(id -g)" \
  -e HF_HOME=/cache -v ~/.cache/huggingface:/cache \
  -v ~/audio:/audio:ro -v ~/transcripts:/out \
  whisper-batch /audio /out

The base image is a 2.86 GB download before the Python packages. The cache mount keeps the model on the VM's disk between runs, and --user writes the transcripts as ubuntu rather than root. Drop the sudo once ubuntu is in the docker group. The image needs a driver that supports CUDA 12.8, which CTranslate2 needs anyway.

Transcribe a folder#

The script below transcribes every audio file in a folder with the batched pipeline and writes a plain-text transcript and an SRT subtitle file for each one. Save it as ~/whisper/transcribe_folder.py:

# transcribe_folder.py: transcribe every audio file in a folder with faster-whisper
import os
import sys
import time
from pathlib import Path

from faster_whisper import BatchedInferencePipeline, WhisperModel

MODEL = os.environ.get("WHISPER_MODEL", "large-v3")
BATCH = 16
AUDIO = {".wav", ".mp3", ".m4a", ".flac", ".ogg", ".opus", ".webm", ".mp4"}


def stamp(t):
    h, rest = divmod(t, 3600)
    m, s = divmod(rest, 60)
    return f"{int(h):02d}:{int(m):02d}:{s:06.3f}".replace(".", ",")


src, dst = Path(sys.argv[1]), Path(sys.argv[2])
dst.mkdir(parents=True, exist_ok=True)
pipe = BatchedInferencePipeline(model=WhisperModel(MODEL, device="cuda", compute_type="float16"))

audio_s = busy_s = 0.0
for f in sorted(p for p in src.iterdir() if p.suffix.lower() in AUDIO):
    if (dst / f"{f.stem}.srt").exists():
        continue
    start = time.perf_counter()
    segments, info = pipe.transcribe(str(f), batch_size=BATCH)
    segments = list(segments)
    took = time.perf_counter() - start
    (dst / f"{f.stem}.txt").write_text(" ".join(s.text.strip() for s in segments) + "\n")
    (dst / f"{f.stem}.srt").write_text("".join(
        f"{i}\n{stamp(s.start)} --> {stamp(s.end)}\n{s.text.strip()}\n\n" for i, s in enumerate(segments, 1)))
    audio_s += info.duration
    busy_s += took
    print(f"{f.name}: {info.duration / 60:.1f} min, {info.language}, {info.duration / took:.1f}x real time", flush=True)

if busy_s:
    print(f"total: {audio_s / 3600:.2f} h of audio at {audio_s / busy_s:.1f}x real time (RTFx)")

Upload the audio from your laptop with rsync -avP ./audio/ quanta-gpu:~/audio/. Then start the job inside tmux on the VM, so it keeps running if your SSH connection drops. If tmux is missing, sudo apt-get update && sudo apt-get install -y tmux adds it.

tmux new -s whisper
cd ~/whisper && source .venv/bin/activate
python transcribe_folder.py ~/audio ~/transcripts

Detach with Ctrl+B then D, and come back with tmux attach -t whisper. The first run downloads the model, 3.09 GB for large-v3. The script skips every file that already has an .srt, so you can stop it and start it again.

Three settings in the script matter. batch_size=16 sends 16 chunks of speech through the model at once, and batching is where the speed comes from: the README's batched run was 3.7 times faster than the unbatched one (our calculation: 63 / 17 seconds). The batched pipeline first cuts the audio at pauses with its voice detection, which is on by default. WHISPER_MODEL=turbo switches to the faster model, but keep large-v3 for translation, because OpenAI notes that turbo is not trained for it and returns the original language. When every file is in one language, add language="en" (or your language's code) to the transcribe call to skip detection.

Add speaker labels with WhisperX#

My route to speaker labels is WhisperX: it transcribes with faster-whisper, aligns each word to the audio with a wav2vec2 model, and assigns speakers with pyannote's community-1 pipeline. That pipeline is gated on Hugging Face, so it takes accepted terms and a token.

1. Sign in to Hugging Face, open the pyannote/speaker-diarization-community-1 model page and accept its user conditions. It asks for your company or university and a use case, and grants access automatically.

2. Create a read token at huggingface.co/settings/tokens.

3. Install FFmpeg, which WhisperX calls to decode audio and pyannote needs too, and WhisperX in an environment of its own:

   TerminalCopy

   sudo apt-get update && sudo apt-get install -y ffmpeg
   mkdir -p ~/whisperx && cd ~/whisperx
   uv venv --python 3.12 --seed --managed-python
   source .venv/bin/activate
   uv pip install "whisperx==3.8.6"
   cat >> .venv/bin/activate <<'EOF'
   export LD_LIBRARY_PATH=$(python -c 'import nvidia.cublas.lib as b, nvidia.cudnn.lib as d; print(b.__path__[0] + ":" + d.__path__[0])')
   EOF
   source .venv/bin/activate
   

4. Load the token without leaving it in your shell history, switch off pyannote's usage metrics, and run WhisperX on the folder:

   TerminalCopy

   read -rs -p "Hugging Face token: " HF_TOKEN; echo; export HF_TOKEN
   export PYANNOTE_METRICS_ENABLED=0
   whisperx ~/audio/* --model large-v3 --batch_size 16 --diarize --output_dir ~/transcripts-speakers
   

WhisperX writes a .txt, .srt, .vtt, .tsv and .json file for each input, and with --diarize the lines of the text and subtitle files start with a label such as [SPEAKER_00]:. Its default model is small, so always pass --model. Add --min_speakers 2 --max_speakers 2 when you know the number of speakers, and --language en to skip detection.

WhisperX's README asks for the CUDA 12.8 toolkit first. This route skips it: WhisperX 3.8.6 pins PyTorch 2.8.0, whose CUDA 12.8 build installs cuBLAS and cuDNN as pip packages, and the library line points CTranslate2 at them. On a driver older than 570.124.06, add "ctranslate2==4.6.2" to the install line here too.

pyannote.audio 4 sends usage metrics by default: the pipeline's name, the length of each file and the speaker-count settings, to otel.pyannote.ai. Its README says it tracks nothing that identifies you, and PYANNOTE_METRICS_ENABLED=0 keeps even that on the VM.

Diarization needs little GPU memory: the pipeline's two models are 32.6 MB together. For speed, pyannote publishes 31 seconds per hour of audio for community-1 on an H100, measured on hour-long meeting recordings. The pipeline is licensed CC BY 4.0, which asks for attribution when you share it.

Copy the transcripts off before you stop#

Stopping an instance terminates it and deletes its disk, transcripts included, and there are no volumes or snapshots to fall back on. From your laptop, copy the results down and check the copy before you press Stop:

rsync -avP quanta-gpu:~/transcripts/ ./transcripts/
rsync -avP quanta-gpu:~/transcripts-speakers/ ./transcripts-speakers/
rsync -avnc quanta-gpu:~/transcripts/ ./transcripts/

The last command is a dry run with checksums, and it lists no files when both sides match. Moving files to and from a GPU server covers object storage for large batches. For a queue that runs overnight, turn on auto top-up: if your balance cannot cover the next hour, QuantaCloud terminates the instance and deletes its disk, finished transcripts and all. The idle GPU guide stops the instance once the queue is done. You pay from launch to stop: the first hour is charged at launch and the unused seconds are refunded when you stop (pricing).

Speed and cost per hour of audio#

The number that decides the cost is speed, written as RTFx: seconds of audio transcribed per second of processing, the measure Hugging Face's Open ASR Leaderboard reports. The cost of an hour of audio is the hourly price divided by RTFx, so a GPU that is twice as fast and costs twice as much per hour gives the same cost per hour of audio.

How to measure the speed

The test set is four recorded meetings from the AMI Meeting Corpus, ES2004a to ES2004d, published under CC BY 4.0. They add up to 2.2 hours of 16 kHz mono audio and a 254 MB download (our calculation from the file sizes). These steps measure any GPU the same way:

1. Download the meetings:

   TerminalCopy

   mkdir -p ~/audio-ami && cd ~/audio-ami
   for m in ES2004a ES2004b ES2004c ES2004d; do
     curl -fLO "https://groups.inf.ed.ac.uk/ami/AMICorpusMirror/amicorpus/$m/audio/$m.Mix-Headset.wav"
   done
   

2. In ~/whisper with its environment active, run python transcribe_folder.py ~/audio-ami ~/out-large once to download the model, delete ~/out-large, run it again and keep the total line. Log GPU memory in a second terminal with nvidia-smi --query-gpu=memory.used --format=csv -lms 500.

3. Repeat with WHISPER_MODEL=turbo python transcribe_folder.py ~/audio-ami ~/out-turbo, again keeping the second run.

4. Run the WhisperX command from the WhisperX section on ~/audio-ami/* twice, and time the second run with time. Its RTFx is 7,951.5 seconds of audio divided by the wall time.

5. Divide the hourly price on the day of the test by each RTFx.

Whisper on a GPU: questions#

Does Whisper large-v3 need a 48 GB GPU?

No. OpenAI estimates about 10 GB for its large model in its own implementation, and faster-whisper's batched benchmark used 6,090 MB. You rent 48 GB on QuantaCloud because that is the smallest card on offer, and the RTX A6000 was the lowest-priced one on 2026-09-27. Speech in the other direction is no bigger: open-source text-to-speech models compares the open voice models, and none of them needs a 48 GB card either.

Is faster-whisper as accurate as OpenAI's Whisper?

Its README says it is up to 4 times faster than openai/whisper for the same accuracy, with less memory, and it runs the same weights converted to CTranslate2's format. The batched pipeline splits the audio into chunks at pauses and decodes them side by side, while the unbatched WhisperModel.transcribe works through the audio in order, so compare a few transcripts from both before you switch a production pipeline.

Does my audio leave the VM?

No. Transcription and diarization run on the GPU in your VM, and the software downloads only models and data files on first use. With PYANNOTE_METRICS_ENABLED=0 set, pyannote sends no usage metrics either. Keep the original audio on your side, because it is deleted with the disk when you stop.

Can I run several transcription jobs on one GPU?

Yes. Batching keeps the GPU busy within one file, and with this much spare memory a second copy of the script on another folder fits as well. Watch nvidia-smi and top to see whether the GPU or the CPU runs out first.

My rule for transcription is one RTX A6000, faster-whisper with large-v3 at batch size 16, and WhisperX only for the files that need speaker labels. Pin the versions on this page, copy the transcripts off with a checksum check before you stop, and move to a bigger GPU only when your own RTFx beats its price ratio. For models you serve rather than run in batches, start at LLM inference on QuantaCloud.