French-to-English Only, 1,519 Stars: Why Hibiki Matters in 2026

Why a French-to-English Only Simultaneous Model Still Matters in 2026

Simultaneous translation and offline translation are different problems. Offline translation waits for a full sentence, then outputs one translation. Simultaneous interpretation demands the model emit target speech while the source speaker is still talking, with low latency and high accuracy. This space was long dominated by closed commercial systems and scattered papers. Open projects that release downloadable, locally modifiable weights remain rare, which is why Hibiki deserves attention.

Hibiki is a streaming speech translation model from Kyutai, a French non-profit open AI lab. The team describes it with restraint: a model for streaming speech translation, also known as simultaneous translation. Unlike offline translation, it adapts its generation flow to accumulate just enough context and emit a correct translation in real time, chunk by chunk. As the user keeps speaking French, Hibiki generates natural English speech, optionally with voice transfer and a written text translation.

Repository facts from the GitHub API as of 2026-09-21: full name kyutai-labs/hibiki, 1,519 stars and 119 forks, primary language Rust, created 2025-02-04, last push 2026-09-09, 11 open issues. At 1,519 stars it is midsize and clearly early stage, not comparable with Qwen-Image at over 8,300 stars or Codex-level projects, yet precisely because it is young, studying it is instructive.

Kyutai is a French non-profit lab whose famous prior work is Moshi, a real-time speech-to-speech model in arXiv 2410.00037. Hibiki reuses Moshi's multistream architecture and extends that capability into simultaneous translation; it should not be seen as an independent translation app. Its paper is arXiv 2502.03382, High-Fidelity Simultaneous Speech-To-Speech Translation, by Tom Labiausse, Laurent Mazaré, Edouard Grave, Patrick Pérez, Alexandre Défossez, and Neil Zeghidour (2025). A paper plus downloadable weights plus runnable code is uncommon here. For a broader local versus cloud view, see our local versus cloud voice AI review and speech-to-speech resource roundup.

Architecture: Decoder-Only, Moshi Multistream, Joint Source-Target Modeling

Hibiki is a decoder-only model, unlike architectures that stack a separate encoder, decoder, and alignment module. It models source and target speech jointly via Moshi's multistream architecture. Multistream means the model does not keep a single speech representation; source and target streams are modeled together in one sequence framework, so it can keep processing the input while already generating target speech instead of waiting for input to end.

Hibiki emits text and audio tokens at a constant 12.5Hz framerate. Constant means exactly 12.5 timesteps per second, letting downstream code assemble a continuous audio stream rather than broken segments. The text translation is timestamped and aligned to moments in the source audio, which is practical for subtitles, notes, and auditable interpretation.

Two variants are released, both French-to-English only. Hibiki 2B is the backbone with a depth transformer, using 16 RVQ channels per stream. Hibiki 1B uses 8 RVQ channels per stream, lighter and suited to on-device inference. Note 2B and 1B are parameter scale while 16 and 8 are RVQ channel counts; they are not the same.

Latency is the other axis that separates simultaneous systems from sequential ones. Because Hibiki commits to target tokens while source audio is still arriving, the delay a listener perceives depends on how much lookahead the alignment rule allows. That trade-off, rather than raw translation quality measured on a finished sentence, is what decides whether someone can actually follow a live talk. Encoder-decoder pipelines that finalize a sentence first cannot make this choice at all, which is why the streaming formulation matters more than the parameter count.

Weights ship on Hugging Face in four bf16 variants across PyTorch and MLX: kyutai/hibiki-2b-pytorch-bf16, kyutai/hibiki-1b-pytorch-bf16, kyutai/hibiki-2b-mlx-bf16, kyutai/hibiki-1b-mlx-bf16. Models were trained on sequences up to 120 seconds, with a 40-second inference context window, which sets how long a single utterance it can handle.

Training Is the Real Hard Part: Synthesizing Aligned Pairs

If architecture is clever, training is the real highlight. Hibiki needs supervised, same-speaker aligned data: within one recording, both source speech and target speech plus target text. Such word-aligned same-speaker data simply does not exist at useful scale; you will not find tens of thousands of recordings where one person speaks French and English simultaneously, word aligned. Building it by hand would also be self-defeating, since the model needs exactly the overlap patterns that human interpreters produce naturally and that no annotation guide fully captures.

With real data unavailable, the team went synthetic: construct large amounts of strictly aligned French-to-English speech pairs. The difficulty is word-level alignment, deciding when a source word should be translated in the target. They use contextual alignment, a weakly supervised word-level method, leveraging the off-the-shelf MADLAD machine translation system (google/madlad400-3b-mt) to match words between source and target transcripts.

This yields the alignment rule: a word should appear in the target only once predictable from the source. Simultaneous interpretation is not finishing the French sentence then reading it; it is emitting the English word the moment French provides enough to predict it. The rule is applied two ways: inserting silences where the target should wait, or synthesizing targets with a voice-controlled, alignment-aware TTS that is both natural and time-aligned. This synthetic pipeline bypasses the scarcity of real aligned data. For timbre control, see our AI voice cloning tools comparison review. Honestly, the training data is synthetic; Hibiki is strong in its French-to-English distribution and claims no cross-lingual generalization to German, Chinese, or Japanese.

Engineering Features: Temperature Sampling, CFG, and Multiple Backends

During inference Hibiki continuously encodes source speech and produces target speech. Its advantage: inference relies only on simple temperature sampling and is therefore compatible with batching. Many simultaneous models depend on complex inference policies that are hard to combine with standard batching, capping throughput; Hibiki avoids this, helping developers process multiple streams concurrently.

Second, adjustable voice fidelity. Hibiki supports voice transfer to retain the original speaker's voice. This is controlled by the classifier-free guidance (CFG) coefficient: larger is closer to the original voice, but the team warns excessive values degrade translation. Default is 1, typical is 3; you trade voice likeness against translation accuracy. Sweep the coefficient on a handful of your own recordings instead of copying a value from a blog post, because the useful range shifts with speaker, recording condition, and how much latency the deployment can tolerate.

Third, coverage must be stated truthfully: Hibiki currently supports only French to English; any multilingual hint is inaccurate. Fourth, on-device potential: the smaller Hibiki-M can run locally on smartphones, and Hibiki 1B with 8 RVQ channels suits on-device use, though real experience depends on backend and quantization, which is still in progress. Fifth, broad backend coverage, summarized below.

A real-time web interface exists: python -m moshi_mlx.local_web --hf-repo kyutai/hibiki-1b-mlx-bf16 runs browser-based interpretation. The team also provides a Colab notebook and the kyutai/hibiki-samples listening space for those who skip local setup.

Getting Started: Run It With the Official Commands

The safest path is the README's original commands. For PyTorch, first install moshi with -U to force an update:

pip install -U moshi

Then download a French sample and run inference to English wav:

wget https://github.com/kyutai-labs/moshi/raw/refs/heads/main/data/sample_fr_hibiki_crepes.mp3
python -m moshi.run_inference sample_fr_hibiki_crepes.mp3 out_en.wav --hf-repo kyutai/hibiki-1b-pytorch-bf16

Add --cfg-coef to adjust voice; default 1, typical 3, too high hurts translation, so start from 1 or 3.

For Rust, enter hibiki-rs, download the same sample, and run with cargo:

cd hibiki-rs
wget https://github.com/kyutai-labs/moshi/raw/refs/heads/main/data/sample_fr_hibiki_crepes.mp3
cargo run --features metal -r -- gen sample_fr_hibiki_crepes.mp3 out_en.wav

--features metal is for Mac; on Nvidia use --features cuda. The Rust backend fits the repo's primary language and is smoother than pure Python in performance-sensitive cases.

Crucial fact: Hibiki's implementation is not all in this repo. The README states its implementation is very close to Moshi's and the actual code is in kyutai-labs/moshi; this repo mainly carries weights and adaptation. So installing moshi and calling moshi.run_inference uses Moshi's code; docs must not claim everything is in the hibiki repo. For a quick listen, use the Colab notebook or kyutai/hibiki-samples. For comparison, see our real-time interpretation comparison review and the Qwen3 real-time translation hotspot.

A Sober Second Look

First, keep stars in perspective. At 1,519 stars it is not a breakout; it is early, with community and tooling still accumulating. Star counts also trail reality in a fast-moving field, so treat them as a rough signal of attention rather than a verdict on engineering quality. It is neither comparable with Qwen-Image nor the alleged standard of interpretation; it is a high-quality, reproducible early-stage research project.

Second, language coverage is a hard limit: only French to English. Chinese-English, Japanese-English, or German-English are unsupported, determined by training data, alignment pipeline, and target languages, not a minor polish gap. Any outward promise must state this one-pair limit.

Third, code shared with Moshi is both advantage and trap: you stand on a validated speech model, but newcomers may assume this repo is self-contained and fail to find core code. The correct model is that core inference is in Moshi and this repo provides weights and adaptation; read both.

Fourth, the most miswritten point: the license is three separate parts. Code: Python and web client under MIT, Rust backend under Apache-2.0, with LICENSE-APACHE and LICENSE-MIT both present at the root, a Rust convention. Weights follow a different path, CC-BY 4.0, which permits free use, distribution, and modification including commercial, but requires attribution. Do not loosely call it Apache-2.0 or MIT open source, conflating weights with code, nor claim zero commercial obligation, since attribution is mandatory.

Fifth, division of labor with cloud interpretation: Hibiki offers transparency, modifiability, local deployment, no per-call cost, and on-machine data, suiting research, privacy, and customization, but with single-language coverage and an effectiveness ceiling bounded by its synthetic French-to-English distribution. Cloud APIs win on coverage, stability, and integration at the cost of fees, data leaving premises, and no modifiability. They guard different needs, not replacements. For trade-offs see our local versus cloud voice AI review.

Conclusion: Hibiki is not a universal interpretation miracle but an open specimen laying bare one language pair, one architecture, and one synthetic pipeline. Its stars are few, languages limited, weights carry attribution, and precisely because those limits are plain, it deserves more trust than vaguely worded products. Understanding how it was built beats chasing metrics.

FAQ

Q1: Does Hibiki support Chinese or multiple languages?

A1: No. Hibiki currently supports only French to English. Both documentation and code limit it to FR to EN. It has no Chinese capability and no general multilingual simultaneous translation. If you need Chinese-English or other pairs in real time, Hibiki cannot directly help today; you would wait for possible expansion or use a broader cloud solution.

Q2: What exactly is Hibiki's license, and can I use it commercially?

A2: Three parts. Code: Python parts and web client under MIT, Rust backend under Apache-2.0, both permissive with no copyleft. Weights are CC-BY 4.0, permitting free use, distribution, and modification including commercial use, but requiring attribution. So commercial use is not zero obligation; the attribution clause must be honored.

Q3: Can Hibiki run on a phone?

A3: Yes, with caveats. The smaller Hibiki-M is described as able to run locally on smartphone hardware, and Hibiki 1B with 8 RVQ channels suits on-device inference. On iOS, MLX-Swift was tested on iPhone 16 Pro but is very much experimental and should not be treated as production-grade.

Q4: What does the repeatedly mentioned 12.5Hz mean?

A4: 12.5Hz is the constant framerate at which Hibiki emits text and audio tokens, exactly 12.5 timesteps per second. A constant framerate lets downstream code assemble a continuous, unbroken audio stream while the text translation is timestamped and aligned to source moments. It is not a sample rate or parameter count; it is the basic temporal unit of output rhythm.

Q5: Where does Hibiki's training data come from, and is it real?

A5: Same-speaker word-aligned data does not exist at scale, so Hibiki relies mainly on synthetic generation. The team uses contextual alignment, a weakly supervised word-level method leveraging the off-the-shelf MADLAD translation system to match words between source and target transcripts, following the rule that a word appears in the target only once predictable from the source. It is applied by inserting silences or by synthesizing with a voice-controlled, alignment-aware TTS. In short, the training foundation is synthetic rather than massive real simultaneous interpretation recordings.