One Hour of Audio, Eight Speakers: The Weights Are Out
Turning a one-hour meeting recording into a clean transcript — who said what, terms spelled right — used to mean manual work or an error-stacking cascade. On October 28, NVIDIA open-weighted its answer: Nemotron 3.5 Transcribe, live on Hugging Face and ModelScope. Here is the repository: NVIDIA-Nemotron-3.5-Transcribe-30B-A4B. Repository: huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Transcribe-30B-A4B.
Three qualifiers first. It is a transcription model: the model card and NVIDIA research page define it as a promptable offline speech-audio model on the SALM (Speech-Augmented Language Model) architecture, for turning existing recordings into text. Second, it is a MoE: per the card, 34B total parameters including the audio encoder, only about 4.3B active per token — why one GPU can carry it. Third, it does not stream: NVIDIA states internal audio chunking is not a streaming service; it processes recordings that already exist, not live captions.
A scope note: our previous piece covered Tencent's open simultaneous interpretation model — listening and translating live. This piece covers the other job: one hour of eight-speaker audio, turned into a document that keeps every speaker straight. Transcription is not interpretation.
The License: OpenMDW-1.1, Commercial Use in Writing
Commercial safety is a question for the license field, not the headlines. Both the model card's License field and the NVIDIA research page agree: OpenMDW-1.1, officially "ready for commercial or non-commercial use" — introduced in 2025 by the Linux Foundation and the PyTorch Foundation for ML model distribution, and permissive.
Per the standard OpenMDW-1.1 text (per NVIDIA-family repositories): a free, unrestricted grant over the Model Materials covering copyright, patents, database rights, and trade secrets; redistribution requires keeping the license copy and attribution notices; initiating patent or copyright litigation terminates the grant, defensive counterclaims excepted; no restrictions attach to model outputs; and the model ships "AS IS," without warranty. That clause lands squarely on transcription: whether a recording was made legally, and whether participants consented, is the user's responsibility.
Bottom line: commercial use, redistribution, and fine-tuning are permitted without extra authorization — confirmed by the model card, the research page, and independent outlets.
30B in the Name, 34B on the Card: One Sentence Reconciles Both
The repository is named NVIDIA-Nemotron-3.5-Transcribe-30B-A4B, yet the model card reports 34B total parameters. No contradiction: the language backbone is Nemotron 3.5 Lightning 30B-A3B — a hybrid Mamba-2 and Transformer MoE, 30B total and 3B activated; with the speech-side components attached, the card's figure rises to 34B total, about 4.3B active per token. The repo name inherited the backbone's naming; the card reports the full assembly.
Per the research page, the speech side is a Parallel Expert Encoder design: a 600M-parameter Transformer ASR encoder, a frozen ~99M-parameter speaker encoder called Nemotron-3-Diarization, and a CTC word-timestamp head, all feeding the backbone in parallel. Notably, speaker labels are produced during decoding — Sortformer cues enter the language model with the acoustic features — not patched on afterward. NVIDIA claims this is the first SALM to beat dedicated cascade systems on diarized ASR with same-speaker overlap.
Boundaries: 65,536-token context, one hour of audio per recording, 16 kHz mono input, WAV/FLAC/MP3/OGG supported. Word-level timestamps come from a lightweight CTC adapter applied after decoding, aligning any generated transcript back to the audio — long recordings and heavily overlapped speech included.
Promptable Transcription: Same Weights, Different Instructions
The name undersells the model card's capability matrix: transcription in 40 languages, translation to and from English in 31, speaker attribution in 13, up to 8 speakers per recording. The same checkpoint switches tasks on natural-language instructions — "transcribe and label speakers," "translate into English," "what decisions were made in this meeting," "output a PII-redacted transcript." Style is promptable too: verbatim or cleaned-up register, filler words kept or dropped, number formatting.
For terminology-sensitive work, the context word list matters: hand the model an expected-entity list and named-entity errors drop to roughly one-sixth (the official "six-fold" figure, from ContextASR-Bench). The official blog's examples are HbA1c and metformin — in medicine, misspelling a drug name is expensive.
One qualifier is easy to miss: the capability matrix is tiered, from Leading down to Supported, and the blog notes audio question answering was trained in English only. Forty languages does not mean forty at the same level; validate against the tier table before production.
The Scoreboard: 4.71% and 5.33% Both Stand
Two English WER figures stand simultaneously. Model-card basis: on 9 public test sets (collected October 12, 2026), English mean WER is 4.71%, next-best open model Qwen3-ASR-1.7B at 4.94%; the 7-language multilingual average is 4.62%, also lowest among open models, versus 6.49%. Leaderboard basis: on the Open ASR Leaderboard site, first among open weights at 5.33% (API models excluded). The gap is documented: as of October 9 the leaderboard began averaging in the private held-out sets of Appen and DataoceanAI plus URGENT24, while the card's 4.71% reflects the older 9-public-set basis. Cite either, label the basis, never mix them.
Per the leaderboard site as of October 9, 2026 (Nemotron at its release figure):
| Model | License | Avg WER | Note |
|---|---|---|---|
| Zoom Scribe v2 Pro | Proprietary | 4.70 | Closed reference |
| Nemotron 3.5 Transcribe 30B-A4B | OpenMDW-1.1 | 5.33 | First among open weights; 4.71 on the card's 9-set basis |
| Qwen3-ASR-1.7B | Apache-2.0 | 5.40 | 4.94 on the card basis |
| Cohere Transcribe 2B | Apache-2.0 | 5.77 | First among open on long audio (12.23%) |
| Canary-Qwen-2.5B | CC-BY-4.0 | 5.94 | English only |
| Parakeet TDT 0.6B v3 | CC-BY-4.0 | 6.18 | RTFx 6,132 |
| Voxtral Small 24B | Apache-2.0 | 6.38 | AA-WER 2.77%, 24B active |
| Whisper large-v3 | MIT | 7.16 | 99 languages |
| Whisper large-v3-turbo | MIT | 7.64 | 809M, no translation |
Third-party numbers agree: on Artificial Analysis's AA-WER, Nemotron scores 2.81% (relayed by NVIDIA as 2.8%), tied with Voxtral Small's 2.77% atop the open field — while activating roughly 4B parameters against Voxtral's 24B, about a sixth per step. On the voice-agent slice AA-AgentTalk it posts 1.8% against 2.3% for the next-best open model.
Hard cases hold: long-audio mean WER of 10.98% versus 13.37% for the runner-up Parakeet-TDT-0.6B-v3; mean cpWER of 15.80 across the full-recording sets AMI, CALLHOME, Mixer 6, and NOTSOFAR-1, versus 16.83 for DiCoW v3.3 plus DiariZen. On style controllability, Preference-ASR shows 7.56 default and 6.16 instruction-following WER, against Canary-Qwen's 8.16 and 8.36.
Speaker attribution is the centerpiece, backed by two independent benchmarks. On David AI's DAI-ASR-I18N, spanning 21 languages and 14 systems, Nemotron Transcribe posts the lowest diarization error rate, 18.26%, narrowly beating NVIDIA's own dedicated Nemotron 3 Diarization at 18.38%; the combined transcription-plus-diarization track ranks first among open models, second overall. On VoiceArena (English): lowest cpWER among 14 systems, 32.4%, versus 34.2% for Multitalker Parakeet plus Nemotron 3 Diarization. More striking are deletions: 8.32% of reference words against a median near 31% — others drop whole sentences; it drops far less.
Throughput and Deployment: Spectacular Numbers, One Caveat
Official throughput first, qualifier attached: on the optimized vLLM inside the Nemotron Transcribe NIM, a single RTX PRO 6000 Blackwell Server Edition with NVFP4 and DFlash2 speculative decoding reaches up to 2,238 audio-seconds per second, returning a 30-second recording in 0.13 seconds; an alternative framing has 100 hours of audio processed in under 3 minutes (high concurrency on 30-second chunks). NVIDIA itself states the caveat: these service optimizations have not landed in the open-source release — self-hosted vLLM will not hit NIM numbers.
The deployment anchor is a single H100 80GB; the single-GPU range runs from Jetson AGX Thor, DGX Spark, and RTX Spark up to Blackwell data-center cards. Serving offers vLLM (NeMo Speech plugin) or SGLang-Omni, both with OpenAI-compatible chat endpoints. Inference optimization is native Multi-Token Prediction speculative decoding — the model's own MTP layers are the draft model — with a separately trained DFlash2 as an option.
Weights ship in four flavors: BF16 (full-precision reference), FP8, NVFP4 (for Blackwell), plus GGUF. GGUF needs its own paragraph: the research page already lists a GGUF companion repository aimed at NeMo-Speech.cpp for local inference — not llama.cpp, and not the community GGUFs around the Lightning line — while the official Hugging Face blog at launch still said quantized versions were "coming soon." Published GGUF weights do not mean a ready local toolchain; NeMo-Speech.cpp support is whatever the official repositories say, so do not write "one-click local inference" before it exists.
Hosted endpoints are available from Baseten, DeepInfra, and Simplismart, with demos on HF Spaces (nvidia/nemotron-transcribe) and ModelScope Studios. Artificial Analysis measured the DeepInfra endpoint at roughly 75 times realtime — one minute of audio in under a second — priced at $2.00 per thousand minutes.
One confusion to head off: NVIDIA also ships a sister model, nemotron-3.5-asr-streaming-0.6b — a 600M native streaming ASR with five latency tiers (80/160/320/560/1120 ms), 40 locales (32 out of the box), realtime on pure CPU. That is the streaming line; the 34B offline transcriber here is a different product.
Granary v2 and Fine-Tuning: Two Hour-Counts and a Medical Case
The training data hub is Granary v2, with two hour-counts that must be kept apart. The public manifest: 463,062 hours, 48 languages, 16 source corpora (Granary v1 covered only 25 languages). The full processing set: roughly 843,600 unique audio hours. Translation coverage adds about 142,800 hours translated into English and about 55,000 hours of English translated into 31 target languages. NVIDIA states these figures describe the dataset release, not Transcribe's complete training mix.
The dataset offers three transcription targets matching the model's promptable styles: tn_raw (text-normalized verbatim), itn_raw (inverse-normalized, fillers kept), itn_clean (filler-free). Its four-stage pipeline — Metadata, ASR, Text, HiFi — is built with NeMo Curator and supports Ray and NVIDIA Dynamo backends, with numbers behind it: the HiFi stage filtered VoxPopuli from 354,000 hours down to 111,000, and label-quality filtering moved the 9-benchmark average WER from 4.91% to 4.67%.
The model's own training scale: about 1.72 million unique indexed audio hours, 372 million samples, and 5.6 billion unique text tokens, mixing Granary v2, multi-speaker meeting corpora, audio-understanding data, and synthetic PII-aware speech — trained via supervised balanced mixing (sampling by answer-token quality) with one SFT stage plus a short RL phase. Worth crediting: some data was licensed from David AI, whose DAI-ASR-I18N evaluation set is explicitly excluded from training — the leaderboard win's independence is accounted for.
The fine-tuning bar is friendly: a single GPU adapts new languages, domains, or tasks, 17 tasks typically costing 0.5 to 2.3 GPU hours each; the NeMo Speech recipe trains the speech encoder and projection layers while the language model gets LoRA. Tooling lives on GitHub: NVIDIA-NeMo/Speech, repository: github.com/NVIDIA-NeMo/Speech — the SALM toolkit includes LoRA merging, a vLLM evaluator, and 16 tutorial notebooks, with the recipe hub in NVIDIA-NeMo/Nemotron (Apache-2.0) containing the Transcribe recipe map.
Healthcare example: on Eka Care's Indian medical voice benchmark (3,619 clips), prompt tuning alone — specifying English output in Latin script — cut the WER from 30.8% to 23.8%; after fine-tuning, the speaker-held-out 2,764 clips went from 22.44% to 12.17%, a 45.8% relative drop, with the whole process costing about 14 GPU hours (the official case ran on a single GB300); adaptation data was about 153 hours of synthetic speech, 29 hours of heavily accented Common Voice English, and 63 hours of MultiMed medical recordings. Prompt tuning costs zero training compute; fine-tuning stays far short of a major project.
FAQ
Q1: Can I use it commercially?
A1: Yes. The model card and research page both tag OpenMDW-1.1, explicitly covering commercial and non-commercial use; redistribution and fine-tuning are permitted, with no restrictions on model outputs. One obligation sits with the user: confirm the rights to a recording before transcribing it.
Q2: Can a consumer GPU run it?
A2: The reference deployment is a single H100 80GB, and the official single-GPU range spans Jetson AGX Thor, DGX Spark, and RTX Spark up to Blackwell data-center cards — DGX Spark and RTX Spark are desk-reachable. GGUF weights are listed, aimed at NeMo-Speech.cpp, but the local toolchain's maturity is whatever the official repos show. NIM throughput figures (2,238 audio-seconds per second) come from the optimized serving stack; self-built vLLM will not match them.
Q3: Can it do live captions?
A3: No. NVIDIA states this is a batch offline model without streaming support, and internal audio chunking is not a streaming service. For live captions, the sister model nemotron-3.5-asr-streaming-0.6b is a 600M native streaming ASR with latency tiers from 80 ms up, under the same OpenMDW-1.1 license.
Q4: Is the WER 4.71% or 5.33%?
A4: Both, on different bases. The 4.71% is the model card's English mean WER over 9 public test sets (collected October 12, 2026); the 5.33% is the Open ASR Leaderboard site's figure, which since October 9 has averaged in private held-out sets plus URGENT24. Cite either with its basis labeled — never mix them in a comparison.
Q5: How good is the speaker attribution, really?
A5: Independent benchmarks say strong: lowest diarization error rate among 14 systems on David AI's DAI-ASR-I18N across 21 languages, at 18.26%; on the English VoiceArena, a deletion rate of just 8.32% against a median near 31% — far fewer dropped words. But NVIDIA advises spot-checking names, speaker transitions, and audio QA results — keep human review in the loop.
Related reading
- Tencent Open-Sourced Simultaneous Interpretation: 3s Lag: the interpretation lane, live translation
- Too Heavy or Too Old: Six Full-Duplex Voice Models Reviewed: open voice models compared for realtime conversation
- Five Open Speech Models in One Drop: Ant's MIT-Licensed Ming-3.0: the full-stack setup guide for Ant Group's five speech models