Transcribe 13-Minute Audio in 59 Seconds, Fully Local

Start with the bill you are used to paying: cloud transcription charges by the audio minute -- an hour of meetings can cost tens of dollars -- and the audio uploads to someone else's server. The alternative: on an RTX 3070 Ti 8GB, faster-whisper's official benchmark transcribes 13 minutes of audio in 59 seconds at int8 precision, and batched inference squeezes that to 16 seconds. No upload, no per-minute billing.

The tool is open source at https://github.com/SYSTRAN/faster-whisper . Under the hood it reimplements OpenAI's Whisper on CTranslate2: up to 4 times faster than openai/whisper at the same precision, leaner on memory, quantizable to 8-bit on CPU and GPU alike. Five steps follow: model selection, installation, the minimal script, speed and VRAM tuning, and advanced features.

Two framing notes. First, the numbers: unless stated otherwise, every speed and VRAM figure here comes from the official faster-whisper README benchmark table, measured with large-v2, beam_size=5, 13 minutes of audio, an NVIDIA RTX 3070 Ti 8GB, and CUDA 12.4; the CPU benchmarks use the small model on an i7-12700K. Second, the scope: this is a transcription (dictation) guide -- speech in, verbatim text out. Simultaneous interpretation is a different track with its own article.

1. Pick a Tier: Model and Precision by VRAM

Answer three questions before touching the terminal: do you have an NVIDIA GPU, how much VRAM, and do you need real-time output? GPU owners pick model and precision by VRAM; CPU-only readers go to the downgraded tier; real-time needs shape the VAD and small-model combination in step five.

The model tier list, per OpenAI's README: large has 1,550M parameters and OpenAI calls for roughly 10GB of VRAM; large-v3's fp16 weights are about 3.1GB, about 1.55GB in int8; turbo is 809M at roughly 6GB, around 8 times faster but it drops the translation task; medium is 769M at about 5GB; small is 244M at about 2GB. Any model name passed to WhisperModel auto-downloads ready-made CTranslate2 weights from Hugging Face -- no manual conversion.

Match the tier to your card:

• 24GB VRAM: large-v3 + float16 + beam_size=5. Quality first; this is the ceiling config.
• 8GB VRAM: large-v3 + int8_float16; if that still does not fit, drop to medium.
• CPU only: small or base + int8.

Two cautions. First, never request plain int8 on a GPU -- that is the CPU tier. The CTranslate2 docs require compute capability 7.0 or higher, or exactly 6.1, for int8 on NVIDIA GPUs. Cards at 6.0 or below (Maxwell, Pascal) silently fall back to float32 when you request int8 or float16: no error, just slower and hungrier -- a CPU int8 route beats a pretend-quantized GPU. Capability 6.1 (GTX 10-series) can use int8_float32. Second, verify quantization took effect: check the loading logs, set CT2_VERBOSE=1, or call ctranslate2.get_supported_compute_types("cuda"). Old-card downgrades are silent.

One new option, in one paragraph: in October 2026, NVIDIA opened the weights of Nemotron 3.5 Transcribe, a MoE model with roughly 4.3B activated parameters covering 40 transcription languages; GGUF weights are published and RTX Spark-class devices can run it locally. The open-source column piece in this batch covers its scores and deployment details.

The compute_type parameter will surface during setup, and the CTranslate2 docs list ten values: default, auto, int8, int8_float32, int8_float16, int8_bfloat16, int16, float16, float32, bfloat16. auto picks the fastest type your system supports; int16 works only on Intel CPUs with MKL, bfloat16 needs compute capability 8.0 or higher. Quantization also shrinks disk footprint: CTranslate2's own table shows a base Transformer at 364MB in float32 versus 95MB in int8_float16 -- 4 times smaller with minimal accuracy loss.

2. Install: One pip Line Plus the GPU Duo

CPU users are done in one line:

pip install faster-whisper

GPU users need two more packages: CUDA 12 builds of cuBLAS and cuDNN 9. The latest ctranslate2 only accepts the CUDA 12 + cuDNN 9 combination; cuDNN 8 fails GPU initialization. On Linux you can pip-install the NVIDIA libraries directly:

pip install nvidia-cublas-cu12 nvidia-cudnn-cu12==9.*
export LD_LIBRARY_PATH=$(python3 -c 'import os, nvidia.cublas, nvidia.cudnn; print(os.path.dirname(nvidia.cublas.__path__[0]) + "/cublas/lib:" + os.path.dirname(nvidia.cudnn.__path__[0]) + "/cudnn/lib")')

Set LD_LIBRARY_PATH before starting Python. To skip the local setup entirely, use the official NVIDIA image nvidia/cuda:12.3.2-cudnn9-runtime-ubuntu22.04, which ships both cuBLAS and cuDNN. Stuck on an old stack? CUDA 11 + cuDNN 8 takes ctranslate2==3.24.0; CUDA 12 + cuDNN 8 takes ctranslate2==4.4.0.

Verify the GPU is actually visible:

python -c "import ctranslate2; print(ctranslate2.get_cuda_device_count())"

A return of 0 means you are on CPU. Do not skip this -- every "out of VRAM" fix later assumes the GPU is doing the work.

A note for tinkerers: PyPI also allows force-installing the master branch from the GitHub archive, but the stable release is safer. CPU works out of the box because CTranslate2 underneath is a multi-backend engine for x86-64 and ARM64 that dispatches AVX/AVX2 automatically.

3. The Minimal Script and the Generator Trap

The README's minimal example is a working starting point (large-v3, GPU fp16, beam_size=5):

from faster_whisper import WhisperModel

model_size = "large-v3"
model = WhisperModel(model_size, device="cuda", compute_type="float16")
# Not enough VRAM? Switch to compute_type="int8_float16".
# No GPU? Use device="cpu" with compute_type="int8".

segments, info = model.transcribe("audio.mp3", beam_size=5)
print("Detected language '%s' with probability %f" % (info.language, info.language_probability))

for segment in segments:
    print("[%.2fs -> %.2fs] %s" % (segment.start, segment.end, segment.text))

Device and precision must be paired: cuda takes float16 or int8_float16; cpu takes int8 -- "On CPU int8 will give the best performance," per the whisper-ctranslate2 docs.

Here hides the biggest trap in this article: segments returned by transcribe is a generator. Until you iterate over it, nothing has been transcribed. The README warns explicitly that segments = list(segments) is where transcription actually executes; every "13 minutes in one second" fake benchmark traces back to this. Time yourself with the clock stopping after the loop finishes, or you will measure zero seconds.

4. Three Levers, In Order: Precision, Batching, Beam

The official benchmark table, with the caliber repeated (large-v2, beam_size=5, 13 minutes of audio, RTX 3070 Ti 8GB):

The transformers row is omitted: same caliber, fp16 in 1m52s with 4960MB of VRAM, behind on both counts, and a README footnote notes it OOMs at batches above 1.

Three levers, from biggest payoff down.

Lever one is precision. fp16 to int8 takes VRAM from 4525MB to 2926MB, about 1.6GB saved, while time only nudges from 1m03s to 59s. On an 8GB card the real value is freeing headroom for lever two.

Lever two is batching. faster-whisper ships BatchedInferencePipeline as a drop-in replacement for WhisperModel.transcribe:

from faster_whisper import WhisperModel, BatchedInferencePipeline

model = WhisperModel("turbo", device="cuda", compute_type="float16")
batched_model = BatchedInferencePipeline(model=model)
segments, info = batched_model.transcribe("audio.mp3", batch_size=16)

for segment in segments:
    print("[%.2fs -> %.2fs] %s" % (segment.start, segment.end, segment.text))

The README example uses batch_size=16; the official benchmark table uses 8. On an 8GB card, start at batch_size=8: int8 plus batch=8 costs about 1.5GB of extra VRAM (2926MB to 4500MB) and cuts 59 seconds to 16, roughly 3.7x. Tight? Drop to 4. Do not mix up the two batch rows: fp16 with batch=8 is 17 seconds at 6090MB, which an 8GB card cannot hold; the config that fits is int8's 16 seconds at 4500MB. One caveat: batched mode ignores several sequential parameters -- compression_ratio_threshold, logprob_threshold, no_speech_threshold, and condition_on_previous_text.

Lever three is beam, and it comes last: lowering beam_size yields the least. One discipline rule: speed comparisons must align beam_size -- the official benchmarks all lock beam_size=5; lock yours too.

Quality evidence exists too: in the official distil-whisper-large-v3 benchmark (batch_size=16, beam_size=5, YT Commons), faster-whisper hit a word error rate of 13.527 in 25m50s while transformers took 46m12s for 14.801.

CPU users deserve an honest split: at fp32, whisper.cpp (2m05s / 1049MB) is actually faster and leaner than faster-whisper fp32 (2m37s / 2257MB). faster-whisper's CPU advantage cashes in only through int8 -- 1m42s at 1477MB, 51s at 3608MB with batch=8. The pure-CPU menu: faster-whisper with small or base in int8, or whisper.cpp outright (its OpenVINO backend is even faster).

There is a third shelf for readers without Python: the whisper-standalone-win project ships standalone CLI executables for Windows and macOS; the speaches project provides an OpenAI-compatible API server with Docker deployment, streaming, and real-time transcription.

5. Beyond Basics: VAD, Word Timestamps, SRT, Diarization

Long audio starts with VAD. faster-whisper bundles a Silero VAD model that strips non-speech segments; the conservative default only removes silence longer than 2 seconds:

segments, _ = model.transcribe(
    "audio.mp3",
    vad_filter=True,
    vad_parameters=dict(min_silence_duration_ms=500),
)

Remember one behavioral asymmetry: vad_filter defaults to off on the sequential path and on for the batched path. Pass it explicitly in comparison experiments.

Word-level timestamps come from word_timestamps=True:

segments, _ = model.transcribe("audio.mp3", word_timestamps=True)
for segment in segments:
    for word in segment.words:
        print("[%.2fs -> %.2fs] %s" % (word.start, word.end, word.word))

SRT subtitles require manual work: the library ships no SRT writer. The common recipe is iterating segments and assembling the timeline yourself -- HH:MM:SS,mmm joined to HH:MM:SS,mmm, one numbered block per segment. Prefer zero code? The whisper-ctranslate2 CLI accepts openai/whisper-style arguments, and --output_format takes txt, vtt, srt and more.

Speaker diarization is not faster-whisper's job -- use WhisperX, repo at https://github.com/m-bain/whisperX . It claims 70 times realtime since v3 (large-v2, batched), runs a faster-whisper backend with large-v2 + beam_size=5 under 8GB of VRAM, uses wav2vec2 alignment for word timestamps and pyannote-audio for multi-speaker separation. Install with pip install whisperx, then one command:

whisperx path/to/audio.wav --model large-v2 --diarize --highlight_words True

If the speaker count is known, add --min_speakers 2 --max_speakers 2. Prerequisites: a Hugging Face read token (--hf_token) is required, and you must first accept the user agreement for the speaker-diarization-community-1 model on HF, or --diarize errors out. Tight on VRAM? --batch_size 4, or split into two passes: transcribe and align first, then diarize. Alignment has an accelerator too: pip install "whisperx[fast]" pulls in Numba, speeding long-audio alignment by 30-100 times. Behavioral differences from openai/whisper: batched single-pass inference wants --without_timestamps True; VAD-segmented transcription lowers WER per the paper; condition_on_prev_text defaults to False to reduce hallucination. Alignment models are picked per language -- English, French, German, Spanish, and Italian are built in, others auto-download from torchaudio or Hugging Face; the full list lives in DEFAULT_ALIGN_MODELS_HF in the repo's alignment.py.

Real-time transcription is the third track -- the mainline here is offline files, so only a sketch. whisper.cpp's stream example emits text in 3-second chunks, small-model latency around 0.5-1.5s; on the faster-whisper side, vad_filter=True segments by silence, giving small/medium latency of roughly 0.5-2s, and tiny runs under 0.5 seconds with a markedly higher word error rate. Zero-code option: whisper-ctranslate2's --live_transcribe True transcribes from your microphone in real time.

6. Ten Pitfalls, One List

The traps the steps above could not fully absorb, collected for a pre-flight scan:

• Generators do not compute until iterated. transcribe returns a generator; segments = list(segments) is where the work happens.
• Plain int8 on GPU. The GPU tier is int8_float16; plain int8 is the CPU tier and will not saturate the card.
• Silent downgrades on old cards. Compute capability 6.0 or below silently reverts int8/float16 to float32. Verify with CT2_VERBOSE=1 or get_supported_compute_types.
• Mismatched device and precision. cpu pairs with int8; cuda pairs with float16 or int8_float16.
• cuDNN version mismatch. Latest ctranslate2 wants CUDA 12 + cuDNN 9 only; downgrade ctranslate2 (3.24.0 or 4.4.0) to match older stacks.
• Benchmarks without aligned beam_size. The official tables all lock beam_size=5; lock yours before comparing.
• Batch mode ignores sequential parameters. Thresholds like compression_ratio_threshold do nothing on the batched path.
• Diarization is not plug-and-play. Accept the model agreement on Hugging Face and supply a read token first.
• The three-step VRAM descent. Lower batch_size, then the model (large to base), then precision (int8); the first two can cost quality.
• turbo skips translation. About 8 times faster, transcription only -- task=translate users must look elsewhere.

FAQ

Q1: Can I run this without an NVIDIA GPU?

Yes, with the right tier. On CPU, use small or base with int8: on an i7-12700K with 8 threads, faster-whisper int8 transcribes 13 minutes of audio in 1m42s at 1477MB of memory, 51s with batch=8. whisper.cpp is the leaner CPU alternative. Running large on CPU alone is painfully slow -- do not.

Q2: My script "finishes instantly" with no output. Why?

transcribe returns a generator, and nothing executes until you iterate. The seconds you measured were just generator creation; the real transcription happens inside the for loop. To materialize everything at once, write segments = list(segments).

Q3: large-v3 OOMs on my 8GB card. What now?

Descend in three steps: batch_size from 8 down to 4, then the model down to medium, then precision down to int8. On an 8GB card the correct starting point was already large-v3 + int8_float16.

Q4: Can it export SRT subtitle files directly?

The library has no SRT writer. Either iterate segments and assemble the timeline yourself (HH:MM:SS,mmm -- comma before milliseconds, not a period), or use the whisper-ctranslate2 CLI, where --output_format srt writes the file directly.

Q5: Can it translate Chinese to English while transcribing?

Not as an interpreter. Whisper's translate task outputs English only and runs as offline whole-passage translation; turbo drops the translation task entirely. Transcription outputs the source text, simultaneous interpretation streams the translation while you speak -- different tracks, different articles on this site.

Related reading

• AI Speech-to-Text Showdown: How to Pick Among Five Transcription Tools
• Which Open-Source Streaming ASR Never Rewrites Your Words?
• Nemotron 3.5 Transcribe Open Weights: 40-Language Transcription