Tencent Open-Sourced Simultaneous Interpretation: 3s Lag

The 3-Second Interpreter Is Now Open Weights

On October 26, Tencent Hunyuan released Hy-SimulMT, a realtime simultaneous interpretation model, on GitHub with public weights. The README news entry reads: "Hy-SimulMT released, supporting bidirectional Chinese-English realtime simultaneous interpretation and realtime voice cloning." The official model page pins the same date and frames the project as "a low-latency, high-quality model for real-time simultaneous translation." Here is the repository: Hy-SimulMT. Repository: github.com/Tencent-Hunyuan/Hy-SimulMT.

The traffic picture first, per a GitHub API snapshot taken October 28, 2026: created on October 22, last pushed October 27, standing at 149 stars and 14 forks with 2 open issues — early-stage numbers for a week-old project. Weights live in three places: the tencent/hy-simulmt collection on Hugging Face, a mirrored collection on ModelScope, and the GitHub repository itself. The Hugging Face card carries the translation pipeline tag, with 131 downloads and 38 likes as of collection time. Non-developers get an entry point: the official Hy Translation App is free, with Android and iOS QR codes in the README.

A scope note: we previously reviewed Hy-Nomas, another Tencent Hunyuan open release — interactive world generation, unrelated to interpretation. Our full-duplex voice comparison covers realtime conversation. This piece covers translation: you speak Chinese, the other side hears English, while you keep talking. Three different jobs, three articles.

The License: Three Layers, One Sentence

Licensing is where open-source projects collect the most confusion, and Hy-SimulMT is a textbook case: its status looks different in three places.

The first layer is the only one that legally matters: the LICENSE-Hy-SimulMT.txt file states "Hy-SimulMT is licensed under Apache-2.0, except for the third-party components listed below," and adds a key guarantee — it "does not impose any additional restrictions" beyond the original licenses of those components. The third-party list contains exactly one item, Qwen3-ASR-0.6B, itself Apache-2.0. The copyright line reads Copyright (C) 2026 Tencent, and the grant is scoped to the publicly released inference code, parameters, and weights.

The second layer: the GitHub repository page shows NOASSERTION in its license field. The reason is mundane — Tencent used the custom filename LICENSE-Hy-SimulMT.txt, and GitHub's detector cannot classify the merged text. That is a display artifact, not a legal change. The third layer: the Hugging Face card is tagged license:apache-2.0, consistent with the file.

One sentence to remember: Apache-2.0, with a third-party exception list containing a single entry, and no added commercial restrictions. If you plan commercial use, read the license file and ignore the gray text on the repository page.

Architecture: What the Name Says and What It Does Not

The model is named Hy-SimulMT-30B-A3B. Following MoE naming conventions, 30B denotes total parameters and A3B roughly 3B activated per token — but a discipline note is needed: neither the official README body nor the model page ever states "about 30 billion total, about 3 billion active." The activation scale is implied by the name alone. The verifiable figures come from the Hugging Face weight metadata: roughly 30.39B total parameters in BF16, spread across 17 safetensors shards plus one audio-tower file, about 60.85 GB of storage; the MoE structure uses 128 experts with 8 activated per token; the architecture class is HYV3SpeechForCausalLM. Anything beyond that would be putting words in the vendor's mouth.

Six weight variants ship, in two output families:

The Interleave variants also emit the source-language transcript, which matters for bilingual subtitles and verification workflows. One detail deserves a flag: the Hugging Face card carries 36 language tags (zh, en, fr, pt, es, ja, ko, ru, ar, and others), yet the README body promises only bidirectional Chinese-English interpretation. The tag list overstates the actual commitment. Treat the README as the contract before planning any minor-language pilot.

The Correction: It Is Not End-to-End Speech-to-Speech

This is the most important correction in this piece, because much of the coverage has already got it wrong.

Hy-SimulMT is widely described as end-to-end speech-to-speech interpretation, but the official README draws the capability boundary of the base checkpoint plainly: it returns translated text only; speech output and voice cloning require a separate TTS service. The "3-second" translated voice in demos is a two-stage pipeline: Hy-SimulMT produces the translated text, then a TTS system synthesizes and plays it. In the official deployment recipe, that slot is filled by Qwen3-TTS — the SimulInfer documentation specifies Qwen3-TTS-12Hz-0.6B-Base deployed via vLLM-Omni — and once the tts_clone_source flag is set to true, the system automatically clones the voice from the uploaded audio or microphone input, with no reference sample required.

The corollary is direct: the roughly 3-second figure in the README measures the translation stage only, excluding synthesis and playback time. Build the full "speaker talks, listener hears the translated voice" system, and the end-to-end perceived latency adds a TTS slice on top of those 3 seconds. That is not a knock on the model — text-level simultaneous interpretation is the hardest stage of the chain — but procurement math needs the whole bill. The official demo, in which an English-to-Chinese input contains three speakers rendered in three distinct voices so listeners can tell who is talking, is delivered by exactly this combination.

The Scoreboard: 3 Seconds, 85 to 86, and a Small Human Study

Every performance number below is official-sourced, with the qualifier attached.

Latency: on the RealSI long-audio benchmark, StreamLAAL comes in at about 3 seconds, the lowest among the systems shown in the official comparison chart. Two qualifiers are mandatory. First, the README itself notes that StreamLAAL is not the time to first spoken output; it is an academic metric trading off quality against lag, not the perceived wait for the next sentence. Second, the body text says only "about 3 seconds"; finer values such as 3.1 or 3.3 seconds appear in the accompanying figures, not the text, and are not quoted here as body-text numbers.

Quality: on RealSI long audio, the GEMBA score is roughly 85 to 86 per the official comparison chart, with GPT6.1-Sol as the judging model — again, that precise value and the judge's name come from the chart, while the README body offers only the conclusion: GEMBA above the compared open models, and above four of five commercial APIs. The short-audio table across four test sets is more concrete: nine systems, higher is better, and Hy-SimulMT posts FLEURS 90.3, CoVoST2 75.5, ACL6060 81.2, and RealSI-Sentence 84.7. The official reading: three of the four beat every other open model in the table, with RealSI-Sentence 84.7 the highest score overall. The weak spot is stated just as plainly: CoVoST2 is not first; Qwen3.8-LiveTranslate leads that column at 82.1.

The human comparison is the most eye-catching claim, and it carries the most qualifiers. The test used 18 Chinese and English audio clips newly published to the internet after September 2026, each about 6 minutes, spanning speeches, interviews, and meetings across ten domains from software and AI to medicine, law, education, and sports. The six interpreters were all certified — three CATTI Level 1, three CATTI Level 2 — each level represented by its best of three. The official wording: "in this small-scale capability assessment, Hy-SimulMT reached the translation quality of human simultaneous interpreters, though often faster (lower latency)." The accompanying chart puts expert latency at 4.5 seconds for Level 1 and 4.7 seconds for Level 2, again chart-sourced. The official model page adds a stricter disclaimer than the GitHub version: the conclusions apply only to this sample, these interpreters, and this evaluation setup, and cannot be generalized to mean the model comprehensively surpasses professional interpreters in all scenarios. Eighteen clips, six interpreters — that is a small-sample study. Whenever a secondary report says "beats human interpreters," remember those words.

The competitive landscape in one paragraph: ByteDance's Seed-LiveInterpret 2.0 is a commercial API, not open-sourced, and scores around 77 GEMBA in Tencent's table; Qwen3.8-LiveTranslate has no visible open weights; Gemini3.5-LiveTranslate takes the single best FLEURS score at 92.4. The caveat: this is Tencent's self-reported table; the vendors' own metrics use incompatible systems, and the cross-model conclusions hold only inside this table.

Speak First, Repair Later

The textbook dilemma of simultaneous interpretation is whether to wait: waiting too long explodes latency, not waiting breeds errors. Hy-SimulMT's route is "speak first, repair later" — deliver the translation under current confidence, then revise as more context arrives. Wang Weiwei, vice dean of the Graduate School of Translation and Interpretation at Guangdong University of Foreign Studies, put it this way: to control latency, the system speaks first and repairs afterward, and the repair sounds close to an interpreter's self-correction; she frames AI interpretation as moving from comprehension aid toward usable communication in low-risk scenarios.

The second endorsement comes from Zhong Weihe, former president of Guangdong University of Foreign Studies and former executive vice president of the Translators Association of China: the model shows solid capability in recognition stability, response speed, and delivery quality. The official page closes the loop itself: for scenarios demanding strict accountability, professional review, or high-stakes decisions, professional interpreters remain essential. Two translation-education figures endorsing the model alongside one self-imposed limit — the subtext is clear. Business meetings, classrooms, and tourism briefings are the battleground; courtrooms and clinical settings still belong to humans.

Deploying It: SimulInfer, SimulTrain, and an App Shortcut

The official deployment paths come in three tiers, by appetite for engineering.

Tier one is the SimulInfer serving stack, for developers who want interpretation running: it requires vLLM 0.31 or later with HYV3ForCausalLM support, and provides a web interface, audio file upload, realtime WebSocket streaming, and optional TTS voice cloning. The service layer is light — Python 3.11 or later, deployable on a CPU-only machine — while the one-click route is:

# One-click deploy (web and API default to port 8080, local model port 8800)
bash deploy.sh --install-server --model tencent/Hy-SimulMT-30B-A3B-Interleave

Chunking comes in two flavors: standard mode at 1600 ms per chunk (the recommended default), and short-chunk mode with a mandatory 1600 ms first chunk followed by 800 ms chunks, trading stability for responsiveness. A noise-resistant mode is switchable, and remote microphones require HTTPS.

Tier two is the runtime. The README documents SGLang-Omni support: BF16 weights occupy about 57 GiB excluding KV cache and runtime overhead, deployable on a single H200, with measured speedups up to 6.5x over Transformers at batch size 1. Limits are stated too: only the BF16 translation-only checkpoint is supported, returning text only. One trap: a single request caps audio at 30 seconds, and overlong files are rejected outright rather than truncated; one code path rebuilds the translation context every 30 seconds. Long recordings must be split yourself.

The GGUF route deserves its own paragraph. Architecture support has been submitted to llama.cpp as PR #31198, unmerged as of the October 28, 2026 recheck; GGUF weights currently run through llama-mtmd-cli (the example weight is a Q8_0 quant), while llama-server support awaits adaptation, and the official sub-documentation recommends waiting for a new llama.cpp release. Which GPU can run the quantized model: no minimum configuration is published anywhere official, and no community validation exists either. Any claim that "a single 4090 runs it" has no basis — that gap stays open until someone publishes numbers.

Tier three is SimulTrain: continued training on the downloaded checkpoint for domain adaptation, with model path, data, GPU, and output directory set in SimulTrain/.env. Teams with proprietary corpora that want custom terminology and style belong here; hardware details live in the sub-directory docs rather than the main README.

FAQ

Q1: Can I use it commercially?

A1: Yes. The license file states Hy-SimulMT is released under Apache-2.0 with a third-party component exception list (a single entry, Qwen3-ASR-0.6B, itself Apache-2.0) and no additional restrictions. The NOASSERTION badge on GitHub is a filename-detection artifact; the Hugging Face card is tagged apache-2.0. The license file is the authority.

Q2: Can a single RTX 4090 run it?

A2: Nobody official has said so. The only published hardware anchor is the SGLang-Omni path: BF16 weights at about 57 GiB, deployable on one H200. No minimum configuration for the GGUF quantizations has been published, and no community validation exists. Wait for PR #31198 to merge and for consumer benchmarks.

Q3: Is the roughly 3-second latency the wait before the listener hears the translated voice?

A3: No. The 3 seconds are the StreamLAAL metric on RealSI long audio, which the README itself says is not the time to first spoken output. The base model outputs translated text only; speech requires a separate TTS (the official recipe uses Qwen3-TTS). The full voice pipeline adds synthesis and playback time on top.

Q4: Which languages are supported?

A4: The README's commitment is the contract: bidirectional Chinese-English realtime interpretation. The 36 language tags on the Hugging Face card are not an interpretation capability promise. Do not greenlight a minor-language project based on tags.

Q5: Does it really beat human simultaneous interpreters?

A5: The official framing is far more restrained than some secondary coverage: a small-scale assessment with 18 clips of about 6 minutes and 6 CATTI-certified interpreters, with the disclaimer that conclusions apply only to this sample and setup and cannot be generalized to all scenarios. High-stakes settings still need humans.

Related reading

• Too Heavy or Too Old: Six Full-Duplex Voice Models Reviewed: the realtime-conversation lane, a different race from interpretation
• One RTX 4090, One Interactive World Model: Hy-Nomas Reviewed: the same Hunyuan stable, building interactive world generation
• Five Open Speech Models in One Drop: Ant's MIT-Licensed Ming-3.0: the full-stack setup guide for Ant Group's five speech models