MoE models already win the “more total parameters than you activate” bet on servers. Phones are a different constraint: total expert weight still has to live somewhere, device RAM cannot hold every expert, and naïve load-from-SSD stalls kill interactive decode. Restricting the expert pool per prompt can keep latency down — and quietly drop experts the router would have picked. MiniCPM-V 4.7’s pitch is that you do not have to choose: keep token-level routing intact, prefetch the next layer’s likely experts, and stream the rest from storage fast enough that a 35B-class multimodal feels interactive. For overseas engineers the useful checks are which files to pull, what LES actually guarantees, and which speed/score numbers are OpenBMB’s own benches.
What shipped
Per the OpenBMB docs and Hugging Face cards:
- MiniCPM-V 4.7 35B-A3B — first MoE in the MiniCPM-V series; ~35.2B total / ~3.5B active; 256 experts, top-8 per token; language backbone
qwen3_5_moe_text. - MiniCPM-V 4.7 1B — dense ~1.55B on
qwen3_5_text; same vision tower and processor family. - Context — 256K for both.
- Vision — LLaVA-UHD v4-style intra-ViT early compression; mixed 4× / 16× visual token compression so you can trade detail for cost per task.
- Thinking — one checkpoint per size; enable/disable via chat-template kwargs (no separate “-Thinking” weight dump).
- DSpark draft heads — BF16 and GGUF draft models released alongside both sizes for speculative decoding.
As weights + inference stacks — yes. Hugging Face: openbmb/MiniCPM-V-4.7-35B-A3B and MiniCPM-V-4.7-1B (plus GGUF / BNB / AWQ / GPTQ / FP8 variants). Native Transformers path needs transformers >= 5.18.0 with AutoModelForImageTextToText — no trust_remote_code. Serving: SGLang (fork branch noted in docs), vLLM, llama.cpp, Ollama. Fine-tune: SWIFT, LLaMA-Factory; OpenBMB also published an SRFT (Self-Replay Fine-Tuning) recipe. On-device demos for iOS, Android, and HarmonyOS live in MiniCPM-V-Apps (TestFlight / APK / HAP). Day-0 chip paths listed: Rockchip, Houmo, Intel, AMD, Arm; OpenVINO notebook for Intel. As a retail phone SKU or App Store product — no. This is open weights and developer demo apps.
LES · MoE-SSD · DSpark — the three pieces OpenBMB says make on-device MoE interactive without freezing the router. Asterisks mark vendor / OpenBMB on-device and bench figures.
LES, MoE-SSD, and what “lossless” means*
OpenBMB’s write-up is unusually concrete about the failure modes they are trying to avoid:
- LES (Lookahead Expert Scheduling) — cross-layer routing prediction prefetches experts the next layer is likely to need. Among experts the actual router selects, it prioritizes ones already prefetched and overlaps compute with loading the rest. Predictions guide prefetch and schedule only; computation still follows the model’s token-level router, and missed required experts are still loaded and evaluated.
- Layer-wise quantization — shrink expert storage and memory traffic so streaming fits phone bandwidth/RAM budgets without the team’s claimed capability cliff.
- DSpark — speculative sampling wired into the on-device path alongside LES and quant.
The phone-oriented GGUF split is called out in MiniCPM-V-Apps as MoE-SSD: routed experts are stripped from the main .gguf into an experts/ directory (40 shards + manifest for the demo pack) and streamed from SSD on demand. That is why a 35B-class multimodal can run with a small resident working set — OpenBMB’s headline number is ~3 GB memory and 13+ tokens/s decode on iPhone 17 Pro Max* with LES + layer-wise quant + DSpark. Plan storage separately: the demo’s on-device 35B pack is roughly ~16.6 GB download (main -les file + experts + mmproj) and the apps README recommends ≥ 12 GB device RAM for that variant; devices with less RAM fall back to a smaller expert cache and slow down. Prefer that OpenBMB “13+” figure over older media paraphrases that floated “20+” or “iPhone 18” wording.
Benches* (vendor / OpenBMB)
Prefer the HF card and OpenBMB docs over secondary blogs:
- 35B-A3B Instruct* — higher overall score in less inference time than Qwen3.6-35B-A3B on OpenBMB’s Pareto charts*.
- 1B Instruct* — higher overall score and better efficiency than Qwen3.5-0.8B on the same vendor framing*.
These are OpenBMB’s own comparisons, not independent re-runs. Treat them as directional signals for capability density, then verify on your tasks (OCR, multi-image, video) with the cookbook prompts.
What changes for an overseas stack
If you already ship llama.cpp / Ollama edge demos, or serve Qwen3.5/3.6 VLMs behind vLLM, MiniCPM-V 4.7 changes a few practical checks:
- On-device MoE is no longer “server-only folklore” — there is a public LES + MoE-SSD recipe and phone demos, not just a blog claim that experts “could” stream.
- Two sizes, one processor family — prototype on the 1B GGUF, then decide whether the 35B-A3B SSD split is worth the ~17 GB storage tax for the same multimodal interface.
- Install path is standardizing — Transformers 5.18+ without remote code; day-0 mentions for Rockchip / Houmo / Intel / AMD / Arm widen the industrial-box story beyond a single phone SoC.
- Distinct from compact OCR VLMs — this is general multimodal (image / multi-image / video) with an on-device MoE story, not a dedicated 0.8B document OCR checkpoint.
Caveats
- Vendor on-device numbers* — ~3 GB / 13+ tok/s on iPhone 17 Pro Max come from OpenBMB; not independently re-timed here. Storage (~16.6 GB) and recommended RAM (≥12 GB) for the demo pack are separate from the resident working-set claim.
- Vendor Pareto benches* — vs Qwen3.6-35B-A3B and Qwen3.5-0.8B are OpenBMB charts.
- “Lossless”* — means the router is not replaced by a frozen expert subset; it does not mean bit-identical logits under every quant / SSD miss pattern. Validate quality on your prompts after LES + Q3_K expert quant.
- Phone split ≠ desktop GGUF — official whole-model GGUFs keep experts inline; the
-les+experts/layout is the phone-oriented artifact documented for MiniCPM-V-Apps. - Not a retail phone SKU — open weights and developer demo apps (TestFlight / APK / HAP), not an App Store consumer product or handset feature you buy.
- SGLang — docs note a fork branch for 4.7 support while upstream catches up; check current status before production.
Bottom line: MiniCPM-V 4.7 is OpenBMB’s bid that a 35B-class multimodal MoE can be interactive on a phone without freezing the router — LES for prefetch, MoE-SSD for storage, DSpark for decode. Overseas teams can pull HF weights and the Apps demos today; keep asterisks on every speed and leaderboard claim, and budget disk as carefully as RAM.