{"id":118744,"date":"2026-09-19T01:55:14","date_gmt":"2026-09-19T01:55:14","guid":{"rendered":"https:\/\/youzum.net\/jina-ai-releases-jina-ocr-v1-a-3-4b-moe-document-parser-with-built-in-speculative-decoding-for-low-budget-gpus\/"},"modified":"2026-09-19T01:55:14","modified_gmt":"2026-09-19T01:55:14","slug":"jina-ai-releases-jina-ocr-v1-a-3-4b-moe-document-parser-with-built-in-speculative-decoding-for-low-budget-gpus","status":"publish","type":"post","link":"https:\/\/youzum.net\/it\/jina-ai-releases-jina-ocr-v1-a-3-4b-moe-document-parser-with-built-in-speculative-decoding-for-low-budget-gpus\/","title":{"rendered":"Jina AI Releases jina-ocr-v1: A 3.4B MoE Document Parser With Built-In Speculative Decoding for Low-Budget GPUs"},"content":{"rendered":"

Jina AI, part of Elastic, has released jina-ocr-v1<\/a>, an end-to-end visual document parser. It takes PDFs, scans, tables, charts or invoices and returns clean Markdown in 1 pass. The model has 3.4B total parameters, with about 570M decoder parameters active per token. A speculative decoding head ships inside the checkpoint. Jina AI built it to serve on low-budget GPUs such as the NVIDIA L4. The technical report<\/a> lists 91.14 on OmniDocBench v1.6 and 83.4 on olmOCR-Bench.<\/p>\n

Is it deployable?<\/strong> Yes, for research and non-commercial use. The open weights<\/a> are about 6.8 GB in BF16 and run on Transformers or vLLM. The CC BY-NC 4.0 license means commercial use requires contacting Jina AI<\/a>.<\/p>\n

What is jina-ocr-v1?<\/strong><\/h2>\n

The model post-trains DeepSeek-OCR<\/a> and keeps its 2 efficiency components. DeepEncoder has about 380M parameters and chains SAM, a 16x convolutional compressor and CLIP-L. It turns a 1024\u00d71024 page view from 4,096 patches into 256 visual tokens. A dynamic-resolution mode adds up to 9 local tiles at 100 tokens each. That caps a page at 1,156 visual tokens.<\/p>\n

The decoder is DeepSeek-3B-MoE with 12 layers, 64 routed experts and 2 shared experts. Top-6 routing activates about 570M parameters per token. The position limit is 32,768. Output is Markdown, with tables in HTML and formulas in LaTeX.<\/p>\n

How FastMTP Speculative Decoding Works<\/strong><\/h2>\n

OCR output is near-deterministic and locally structured. That makes it a good fit for speculative decoding<\/a>. Jina AI adds a FastMTP<\/a> head: 1 dense draft block applied recursively for K=3 steps. Draft parameters stay constant as depth grows.<\/p>\n

The decoder then verifies the drafts greedily. It accepts the longest prefix that matches its own choices and commits 1 more token itself. If all 3 drafts match, that extra token is a bonus. The committed text always equals plain greedy decoding, so the speedup is lossless. At K=3 the model commits 2.73 tokens per step on average.<\/p>\n

<\/p>\n

\n

Post-Training With Dense Verifiable Rewards<\/strong><\/h2>\n<\/p>

Post-training combines instruction alignment, robustness fine-tuning on degraded pages, and GRPO<\/a>. Every reward term is deterministic code scored against a reference transcription. The terms cover content, formulas, tables, structural validity, unit tests, repetition and format.<\/p>\n

The terms are multiplied, and each one is graded, so partly correct pages earn partial credit. Structural, unit-test and format terms are floored at 0.2, and the table term at 0.1. The repetition term has no floor, because loops can inflate the content score.<\/p>\n

On natural pages, the formula and table rewards apply to few samples. Jina AI therefore built JinaOCRSynth, synthetic pages packed with both, each carrying olmOCR-Bench-style unit tests. An agent also merges candidate checkpoints under a fixed evaluation budget. The draft head is trained last, against the frozen final verifier.<\/p>\n

Benchmarks and Throughput<\/strong><\/h2>\n
\n\n\n\n\n\n\n\n\n\n\n
Model<\/th>\nParams as listed in the paper<\/th>\nOmniDocBench v1.6<\/a><\/th>\nolmOCR-Bench<\/a><\/th>\n<\/tr>\n<\/thead>\n
jina-ocr-v1<\/td>\n3B\/570M<\/td>\n91.14<\/td>\n83.4<\/td>\n<\/tr>\n
DeepSeek-OCR<\/td>\n3B\/570M<\/td>\nnot listed<\/td>\n76.0<\/td>\n<\/tr>\n
DeepSeek-OCR-2<\/td>\n3B\/570M<\/td>\n90.25<\/td>\nnot listed<\/td>\n<\/tr>\n
PaddleOCR-VL-1.6<\/td>\n0.9B<\/td>\n96.34<\/td>\nnot listed<\/td>\n<\/tr>\n
chandra-ocr-2<\/td>\n4B<\/td>\nnot listed<\/td>\n85.8<\/td>\n<\/tr>\n
Qwen3-VL-235B<\/td>\n235B\/22B<\/td>\n89.78<\/td>\nnot listed<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/figure>\n

For MoE models, params show decoder total and active counts. The whole jina-ocr-v1 model is about 3.4B.<\/p>\n

The model does not lead on accuracy. PaddleOCR-VL-1.6 and HunyuanOCR-1.5 (94.74) score higher on OmniDocBench. chandra-ocr-2 and dots.mocr (83.9) score higher on olmOCR-Bench. Post-training does add 7.4 points over the DeepSeek-OCR backbone on olmOCR-Bench.<\/p>\n

Throughput is the main result. On 1 A100 40 GB at concurrency 32, jina-ocr-v1 parses 2.57 pages per second. That is the highest of 14 systems Jina AI measured, against 1.22 for olmOCR-2 and 0.38 for chandra-ocr-2. It emits 1,085 output tokens per page. Jina AI says that is the shortest output among systems scoring above 83.<\/p>\n

On an NVIDIA L4 at batch size 1, eager decoding rises from 42.7 to 83.1 tokens per second. That is a 1.95x speedup at a 57.6% acceptance rate. With CUDA graphs the baseline is already 158.3 tokens per second. There, K=1 works best at 185.6 tokens per second, a 1.17x gain.<\/p>\n

How to Run It<\/strong><\/h2>\n

The quickest route is Jina Reader<\/a>. Send a URL to r.jina.ai<\/code> with the header X-Respond-With: jina-ocr-v1<\/code>. Reader fetches the page or PDF, runs the model and returns Markdown. An X-Page<\/code> header transcribes 1 page of a longer document.<\/p>\n

Jina AI also hosts an OpenAI-compatible endpoint at https:\/\/api.jina.ai\/v1\/chat\/completions<\/code>. A hosted demo<\/a> is available for quick tests.<\/p>\n

For self-hosting, weights and custom code ship in 1 repository and load with trust_remote_code=True<\/code>. FastMTP requires vLLM<\/a> 0.21 or later and a one-time register()<\/code> call. The Transformers path runs the MoE decoder alone and ignores the draft weights.<\/p>\n

Key Takeaways<\/strong><\/h2>\n