Compact OCR VLMs have gotten good at turning a page into text. The harder product ask — for phone assistants, agent tools, and RAG pipelines — is turning a messy photo of a page into structure you can act on: reading order, nested tables, formulas, and schema-shaped fields. Xiaomi-OCR-0’s pitch is that one 0.8B model can cover that stack without splitting “parser” and “understander” into separate products. For overseas readers the concrete checks are license, install path, how the headline benches were measured, and whether community GGUF/Ollama packs match the paper numbers.
What the model actually does
On the Hugging Face card and technical report, Xiaomi-OCR-0 is framed as a unified OCR-specialized VLM:
- Document parsing — whole-page (or region) recovery into Markdown; tables as OTSL, formulas as LaTeX, ordered by reading order.
- KIE — schema-guided field extraction (receipts, IDs, forms) into JSON.
- OCR-VQA — question answering over documents, charts, and scene text — selective use of visual text, not exhaustive transcription alone.
Training is progressive: Q-Mask text anchoring for fine-grained location↔token alignment, then OCR-centric continued pretraining (CPT), then Mix-RL on hard examples with verifiable rewards (edit similarity, TEDS, CDM, ANLS, JSON field match). The data engine leans on heterogeneous-expert consensus, render-guided verification, and targeted synthesis across that ~170M-sample corpus.
As weights + Transformers inference — yes: huggingface.co/xiaomi-research/Xiaomi-OCR-0 (Apache-2.0). The card documents a verified transformers / torch combo and AutoModelForImageTextToText whole-page parsing prompts. Code, batch scripts, two-stage layout→region assembly, Agent Skill, and eval helpers live on GitHub xiaomi-research/Xiaomi-OCR-0. Paper: arXiv 2609.36136. Community GGUF / Ollama / NInfer packs exist (HF news cites several); Xiaomi explicitly has not evaluated those conversions for accuracy. Browser demo / MCP / Agent Skill are mentioned on the card and in 智东西 coverage. As a phone you buy — no. Super Xiaoai screen-understanding is the product narrative in Chinese coverage; the overseas deliverable is open weights.
Parse · KIE · OCR-VQA on one 0.8B stack; Q-Mask → CPT → Mix-RL. Asterisks mark vendor/paper benches; main OmniDoc numbers use a two-stage layout detector.
Headline benches* (vendor / paper)
Prefer the HF model card and arXiv report over secondary blogs for the numbers. Selected figures:
- OmniDocBench v1.6 Overall 96.83* — #1 among compared models under 1B; overall #2 by 0.04 behind TeleOCR 1.2B (96.87*).
- Real5-OmniDocBench 95.24* — #1 among compared methods (acquisition robustness: scan / warp / screen-photo / illumination / skew).
- Wild-OmniDocBench 87.94* — strong in-the-wild recapture score; trails TeleOCR’s 88.53* on the card’s comparison table.
- UniMER-Test 97.84* (formula); PubTabNet TEDS 92.27* (tables).
- Five OCR-VQA mean 83.2* (DocVQA / InfoVQA / ChartQA / OCRBench / TextVQA) — beats Qwen3.5-2B 80.9*; approaches Qwen3.5-4B 84.9* at roughly one-fifth the parameters.
- Chronicles-OCR Mature Scripts 73* — on par with Seed 2.0 Pro 72* on that mature-script subset (Clerical / Cursive / Running / Regular).
Read the protocol carefully: the report states that the main OmniDocBench v1.6 / Real5 / Wild comparisons use two-stage inference — PP-DocLayoutV3 detects regions, Xiaomi-OCR-0 recognizes crops, then pages are assembled. The advertised 0.8B count excludes that external layout detector. Formula CDM is a highlighted strength on OmniDocBench*; OvisOCR2 and TeleOCR still win some text-edit / table cells in the same tables*.
What changes for an overseas stack
If you already run PaddleOCR-VL, MinerU, GLM-OCR, dots.ocr, or a Qwen3.5-VL baseline for docs, Xiaomi-OCR-0 changes three practical checks:
- Parameter budget — a single 0.8B Apache-2.0 checkpoint that claims parsing + KIE + OCR-VQA without jumping to a 2B–8B general VLM for “understanding.”
- Install path — Transformers
AutoModelForImageTextToTextwith pinned versions on the card; task prompts for whole-page Markdown, region text/table/formula, and schema KIE are documented. - Agent wiring — GitHub Skill + MCP / browser demo references make it easier to bolt OCR into tool-calling agents than a bare checkpoint dump — still research software, not a managed OCR SaaS.
Caveats
- Vendor / paper benches* — scores above come from the HF card and arXiv report; not independently re-run here.
- Two-stage OmniDoc protocol* — PP-DocLayoutV3 is part of the reported OmniDoc / Real5 / Wild system; end-to-end single-pass page parsing is a different measurement.
- Community GGUF / Ollama* — convenient for local demos; Xiaomi has not validated them against the paper numbers.
- KIE in-house* — report cites 59.09%* on an internal extraction set; useful signal, not a public leaderboard.
- Not a retail phone feature SKU — open model weights for labs and app builders; Super Xiaoai stories are product context, not the downloadable artifact.
- Licensing — model card: Apache-2.0; still read third-party base-model and dataset notices.
Bottom line: Xiaomi-OCR-0 is a compact, openly licensed bet that document structure and OCR-centric understanding can share one 0.8B instruction model. Overseas teams can pull HF weights today, follow the Transformers path, and keep asterisks on every leaderboard claim — especially wherever a separate layout detector is in the loop.