Three gigabytes of memory, running a model with 35 billion total parameters. That is not a cloud inference bill — it is something happening on a phone. Around October 13, 2026, ModelBest and the OpenBMB team, which grew out of Tsinghua University's THUNLP lab, released MiniCPM-V 4.7; the official repository changelog is dated October 12, and Chinese media coverage followed in a burst. The headline claim can be compressed into one sentence: according to the official description, this is the first open-source MoE multimodal model to run on-device with lossless quality and smooth speed, shipped in two variants — 35B-A3B and 1B. The first one is the 35-billion-parameter heavyweight that fits in 3GB of RAM.
The project lives at OpenBMB/MiniCPM-V, repository address: github.com/OpenBMB/MiniCPM-V. As of a GitHub API snapshot taken on 2026-10-15, this repository, created in January 2024, holds 26,753 stars and 2,096 forks, with the latest push on 2026-10-14.
Zoom out and the trajectory matters: "put large models on phones" has been the MiniCPM series' home turf since 2024 — the first generation's technical report was titled "MiniCPM-V: A GPT-4V Level MLLM on Your Phone." What changed with 4.7 is the architecture class: earlier generations proved small dense models could be useful; this release takes on MoE, a design widely assumed to be unrunnable on a phone.
Why Running MoE on a Phone Is Hard
On paper, MoE (Mixture of Experts) looks like a free lunch: total parameters set the capability ceiling, while each token activates only a small slice of them. The 35B-A3B variant of MiniCPM-V 4.7 combines 35 billion total parameters with 3 billion activated per token. By activation count it does the work of a 3B model; by capability it draws on a 35B ceiling.
The catch is storage. The official documentation is candid about this: an MoE model's storage requirement is not its active-parameter count. The router can pick different experts at every layer for every token, and the full expert weights far exceed phone RAM, so they sit on flash storage and get loaded on demand. The documentation acknowledges this makes inference "possible" — but repeated loading stalls slow decoding until real-time conversation is off the table.
The industry's other approach is subtraction: fix a subset of experts for an entire prompt to reduce loading frequency. The cost, per the documentation: this pre-decides routing outcomes, experts outside the subset get excluded, and capability degrades. Faster answers, dumber answers.
So the real problem was never "can it run" but the team's repeated bar: lossless and smooth — no capability discount, sufficient throughput. It is a three-way optimization under memory capacity, storage bandwidth, and model capability constraints.
LES: Letting Expert Weights Start Early
ModelBest's answer is called LES — Lookahead Expert Scheduling. The official analogy is a relay runner who starts accelerating before the baton arrives.
LES does three things. First, cross-layer routing prediction: a predictor guesses which experts the next layer will need and preloads them into memory. Second, priority scheduling: when the real router makes its selection, computation goes first to the selected experts that are already prefetched, eliminating the wait. Third, overlapping compute and loading: experts that have arrived are being computed while the ones still in flight keep loading — two pipelines running in parallel.
Here is the design detail that decides success or failure, and that is easy to miss: prediction affects prefetching and scheduling only; it never changes the compute routing. The model picks whichever experts it would have picked anyway, and any expert the predictor missed is loaded and evaluated as usual. That is what "lossless" means in engineering terms — LES optimizes the timing of weight arrival, not the model's decision path. You gamble a little memory bandwidth on prediction hit rate, and losing the bet costs nothing but a slower load.
By the official account, LES delivers a 5x speedup on real devices, and the team describes it as the industry's only open-source solution of its kind. Two companions round it out: fine-grained layer-wise quantization shrinks expert weights' storage footprint and memory traffic so loading fits on-device constraints, and DSpark speculative sampling attacks decoding speed from the other direction — guessing several tokens at once. The three together produce that 3GB runtime figure for 35B-A3B.
One more piece deserves note: the vehicle for LES is OpenBMB/llama.cpp-omni (github.com/OpenBMB/llama.cpp-omni), described as "Omni inference in C/C++," a llama.cpp-based fork under the MIT license. The GitHub API snapshot of 2026-10-15 shows it was created on 2026-10-13 and sits at 16 stars and 2 forks — brand new, but this is the official main path for running 4.7 on devices.
Two Sizes and the Scoreboard
MiniCPM-V 4.7 ships in two sizes. The 35B-A3B is the series' first MoE model and the flagship on-device experience; the 1B is extreme compression for the long tail of existing devices.
The speed figure should be quoted exactly as the official documentation states it: 35B-A3B reaches a decoding speed of 13+ tokens/s with just 3GB of memory on an iPhone 17 Pro Max — the original wording, consistent across the README and the model document minicpm_v4dot7_en.md. Third-party write-ups have cited 15+ and 17+ tokens/s; neither matches the official document, so neither is used here. The documentation's examples include three unedited screen recordings captured directly on devices — 35B-A3B on an iPhone 17 Pro Max and an iPad (M4), and the 1B on an iPhone 17 Pro Max.
The scoreboard needs a rule up front: what follows is the official claim, not independently verified. Officially, 35B-A3B leads its tier across 34 authoritative benchmarks spanning multimodal understanding, STEM reasoning, OCR, and long-video understanding; the 1B sits in the first tier for its size. The official cross-model comparison: in Instruct mode, 35B-A3B scores higher overall than Qwen3.6 35B-A3B while spending less inference time, and the 1B substantially outperforms Qwen3.5 0.8B on both overall score and inference efficiency, while improving on MiniCPM-V 4.6's core capabilities. The team calls this advancing the performance-efficiency Pareto frontier — less per-token computation, undiminished capability.
The vision side has its own economy. Version 4.7 carries forward the series' mixed 4x/16x visual token compression and adds intra-ViT early compression, a technique from LLaVA-UHD v4 that compresses visual tokens inside the vision encoder itself, cutting computation in every downstream layer. Developers can switch between 4x (finer detail) and 16x (higher efficiency) per task via the downsample_mode parameter — full detail for document OCR, saved tokens for casual chat.
The Ecosystem: From Weights to Fine-Tuning
Half of any open-source project's value is ecosystem thickness, and 4.7 ships with a lot of it:
The full weight family. Hugging Face and ModelScope both carry the release, with FP8, int4, AWQ (W4A16), GPTQ (W4A16), and GGUF variants for each size, plus DSpark speculative-decoding draft heads — GPU users get FP8/AWQ, CPU-only users take GGUF, on-device users get the official layered quantization.
Inference and fine-tuning frameworks. The official documentation lists SGLang, vLLM, llama.cpp, and Ollama for inference, and SWIFT and LLaMA-Factory for fine-tuning. OpenVINO provides day-0 support on Intel platforms with notebook examples. A companion Cookbook covers six task guides: single-image, multi-image, video, OCR, PDF parsing, and visual grounding.
Self-Replay Fine-Tuning (SRFT). An officially open-sourced customization method designed to extend model capabilities at low cost while reducing forgetting of existing ones — a saner bet for vertical customization than brute-force full fine-tuning.
Platforms and chips. Official support covers iOS, Android, and HarmonyOS, with apps and source code in the companion repository OpenBMB/MiniCPM-V-Apps. The official day-0 chip list, per the documentation, covers Intel, AMD, Arm, Rockchip, and Houmo. Team announcements add that adaptation through FlagOS extends coverage to domestic chips including Huawei Ascend, Moore Threads, and MetaX (official claim).
Series-level credibility. Officially, the MiniCPM series has surpassed 60 million cumulative downloads across text, vision, and speech models — unverifiable independently, but the 26,753-star repository and two-plus years of steady generational updates suggest this is not a one-off release.
The Weight License, Straight from the Source
Licensing questions should not be guessed; they should be read. The main repository's LICENSE file is Apache-2.0 (verified against the original text). The README's "License & usage" section presents a three-part structure: Apache-2.0 for code, a Model license entry pointing to the Model License section of docs/project_en.md, and a Usage statement.
The final authority on weights is the Model License section of docs/project_en.md, whose key sentence reads: "The MiniCPM-o/V model weights and code are open-sourced under the Apache-2.0 license" — weights and code alike, under Apache-2.0. The original text is at github.com/OpenBMB/MiniCPM-V/blob/main/docs/project_en.md.
In practical terms: Apache-2.0 is a permissive license that allows commercial use, modification, and redistribution. The obligations are the standard ones — retain the license and NOTICE files, state changes — and the license includes an explicit patent grant. Commercial use requires no registration with the team. The Model License section also includes a registration questionnaire, worded in the original as "we would deeply appreciate it if you could consider optionally filling out" — the operative word is optionally: it helps the team understand their user base and affects no usage rights. The Usage statement is a standard disclaimer: model-generated content does not represent the developers' views, and users bear the risks of use.
The combination — code and weights under the same license, registration optional — is refreshingly clean by open multimodal model standards: no custom clauses, no tiered use gates, one license document to read before you know where the boundaries are.
Related reading
- 4GB VRAM Holds 2.8 Trillion Parameters
- Bilibili's 35B translation MoE: 150 languages, only 3B active
- EmbeddingGemma 2: The Open Embedding Model With Fine Print
FAQ
Q1: Why is running an MoE model on a device so hard?
A1: Because an MoE model's storage requirement is not its active-parameter count. Every token can route to different experts at every layer, and the full expert weights far exceed phone memory. Loading on demand from flash works, but repeated loading stalls make decoding too slow for conversation; restricting the expert subset speeds things up but pre-decides routing outcomes and costs capability. "Lossless and smooth" means neither can be sacrificed — a three-way constraint problem of memory, storage bandwidth, and capability.
Q2: What is LES (Lookahead Expert Scheduling), and how does it work?
A2: LES uses cross-layer routing prediction to preload experts the next layer will likely need; when the real router chooses, computation prioritizes the already-prefetched experts, and compute overlaps with loading the rest. Prediction guides only prefetching and scheduling, never the compute routing — missed experts are loaded and evaluated as usual, so no capability is lost. Officially, this yields a 5x speedup on real devices.
Q3: How should the "3GB of memory runs a 35B model" figure be understood?
A3: It is runtime memory, not full-weight size — and not a device requirement. The 35B-A3B activates only 3 billion parameters per token; with layer-wise quantization, expert weights stay on flash and stream into memory on demand. The official figure is 13+ tokens/s decoding with 3GB of memory on an iPhone 17 Pro Max. The official app's hardware table separately recommends 12GB+ device RAM for 35B-A3B (validated on a 12GB iPhone 17 Pro Max) and 4GB+ for the 1B — devices with less RAM get a smaller expert cache and slower speeds. Distinguish the two figures when citing.
Q4: How does MiniCPM-V 4.7 relate to the previous 4.6?
A4: MiniCPM-V 4.6, released on 2026-05-11, is a dense architecture and joined the official Ollama model library on 2026-06-25. Version 4.7 is the series' first MoE model — an architectural generational change. Officially, the 1B variant continues to improve on 4.6's core capabilities, while 35B-A3B pushes capability density in its tier to a new level. Both generations share series techniques such as mixed 4x/16x visual token compression.
Q5: Can the MiniCPM-V 4.7 weights be used commercially?
A5: Yes — based on the official document's original text, not inference. The Model License section of docs/project_en.md states that the model weights and code are open-sourced under Apache-2.0, which permits commercial use, modification, and redistribution, with the obligations of retaining the license and NOTICE and stating changes; the official registration questionnaire is optional and is not a precondition for commercial use. The "permissive code, separately licensed weights" split seen in some open models does not exist here — though for any specific project, treat the latest official document as the final word.
Closing Thoughts
Fitting a 35-billion-parameter MoE into 3GB of memory comes down to three techniques: lookahead expert scheduling, layer-wise quantization, and speculative sampling. The signal beyond the engineering: the barrier to running large models on devices is shifting from "how much VRAM do you have" to "how well can you schedule." With fully open weights, inference code, and fine-tuning code under a permissive license, this on-device multimodal path can finally be taken home and reused. If you want to install it yourself — phone to desktop — the hands-on deployment routes are covered in the companion SOP piece in this series.