Nemotron OCR v2: Fast Multilingual OCR with Synthetic Data

Nemotron OCR v2 demonstrates something many of us suspected but few had measured: with enough realistic synthetic data you can train a multilingual OCR that is both accurate and fast. How did they do it? By combining a rendered-data farm with an architecture that reuses features to avoid redundant work.

Qué lograron y por qué importa

They built a multilingual OCR model (English, Chinese, Japanese, Korean, Russian) that reaches near-zero errors on synthetic tests and processes 34.7 pages per second on a single A100 GPU. That’s not speed for speed’s sake: it means cheaper production pipelines and real-time responses for apps that need to read documents in many languages without pre-detecting the language.

The main lever was massive use of synthetic data: 12.2 million pages generated with pixel-perfect annotation (word-, line-, and paragraph-level boxes, plus reading-order graphs). The public dataset is nvidia/OCR-Synthetic-Multilingual-v1 and the model is available as nvidia/nemotron-ocr-v2.

Datos sintéticos: la receta y por qué funciona

Why synthetic data and not just scraping or human labeling? Because programmatic rendering gives you scale and pure labels. Every box, transcription and reading order is known exactly: you put it there.

Key ingredients:

• Text source: they use mOSCAR to sample texts with realistic per-language distributions.
• Font set: between 165 and 1,258 fonts per language (Google Fonts, Noto, etc.).
• Rendering engine: a heavily modified version of SynthDoG with extensions for multi-structure layouts.
• Extensive augmentations: glyph effects, dilation/erosion, blur, color shifts, shadows, background textures and more.

Central advantage: you can control which edge cases appear (vertical columns, tables, slides, scattered text) and produce millions of examples per day on a single machine.

Extensiones importantes sobre SynthDoG

• Multi-level annotations: word-, line- and paragraph-level boxes, with 4-point quads and hierarchical relations.
• Relationship graph: explicit reading order between lines and paragraphs, crucial for multi-column documents or tables.
• Varied layout modes: multi-column, vertical columns (important for Japanese and Chinese), tables, slides, etc.
• Line-level recognition for CJK: avoids word segmentation when spaces aren’t consistent.

All of this reduces the classic bottleneck in multilingual OCR: it wasn’t the architecture, it was the lack of representative data.

El dataset en cifras

The set contains 12,258,146 samples across six languages. Breakdown by language:

Arquitectura: diseño para velocidad y estructura

Nemotron OCR v2 uses a three-component end-to-end architecture:

• Text Detector with RegNetX-8GF backbone.
• Text Recognizer: a small pre-norm Transformer that decodes rectified crops.
• Relational Model: a compact Transformer that predicts logical groupings and reading order.

The idea is simple and efficient: the expensive convolution runs once and its feature maps are reused by the detector, recognizer and relational model. That feature reuse is what enables the efficiency jump: 34.7 pages/s on A100.

Two available variants:

Note: the multilingual recognizer is heavier because it handles a 14,244-token vocabulary and processes full lines, which reduces the need to segment in languages without spaces but affects throughput on very dense pages.

Resultados: precisión y velocidad

SintDoG (synthetic pages) NED: the multilingual model reduces NED to very low levels compared to Nemotron v1 and to specialized variants from other frameworks.

OmniDocBench (real-world documents) shows a clear tradeoff between speed and accuracy on mixed-language workloads:

The table makes a practical point clear: if your workload is multilingual and you don’t want to detect language first, the multilingual variant offers better overall results and a huge speedup compared to pipeline solutions that run separate detectors and recognizers.

Tradeoffs y decisiones de diseño

• Line-level recognition for CJK avoids word segmentation and improves robustness for Japanese and Chinese.
• A very large vocabulary (14,244 tokens) requires a deeper recognizer and reduces throughput on very text-dense pages.
• Reusing the backbone reduces latency, but requires care in how regions are extracted and rectified for the recognizer.

If you work in product: think about the balance between multilingual support and pages-per-second performance. For single-language documents with little text, the English variant may be more efficient; for mixed or international flows, the multilingual model simplifies operations.

Licencia, demos y cómo probarlo

• Model: nvidia/nemotron-ocr-v2 (NVIDIA Open Model License)
• Dataset: nvidia/OCR-Synthetic-Multilingual-v1 (CC-BY-4.0)
• Demo: Nemotron OCR v2 Space on Hugging Face to try it in your browser

This makes integration and quick evaluation in your data flow straightforward.

In the end, the most interesting thing isn’t just the pages-per-second number or a NED table: it’s the demonstration that, with a robust renderer and good augmentations, synthetic data stops being an approximation and becomes a practical solution to scale OCR across many languages without the prohibitive cost of human annotation.

Fuente original

https://huggingface.co/blog/nvidia/nemotron-ocr-v2