NVIDIA Nemotron 3 Embed leads RTEB and improves retrieval

NVIDIA releases Nemotron 3 Embed: a collection of open, production-ready embedding models designed to boost retrieval quality and make deployments at enterprise scale easier. Want better semantic search, agent memory, or code retrieval with practical options for latency, cost and throughput? This is for you.

What NVIDIA released and why it matters

The collection includes three main models with distinct roles and clear goals:

All models support a 32k token context window, multilingual retrieval and codebase search. NVIDIA also ships open weights, datasets and recipes for fine-tuning and distillation, so you get full control to adapt and optimize on your infrastructure.

Results and key metrics

Highlights from public benchmarks:

• Nemotron-3-Embed-8B-BF16 ranks #1 in RTEB with 78.5% and achieves 75.5% in MMTEB Retrieval.
• Nemotron-3-Embed-1B-BF16 scores 72.4% in RTEB, cutting error by 27% versus its 1B predecessor, and 71.0% in MMTEB Retrieval (error down 28%).

Additional evaluations use average NDCG@10 across RTEB, ViDoRe V3 Text, MMTEB Retrieval and LongEmbed. In agentic retrieval tests — where an agent uses Nemotron 3 Ultra to reason over what the retriever returns — better retrieval usually means fewer repeated searches, fewer reasoning steps and lower downstream token costs.

Practical tidbit: in a combined benchmark the 8B model delivers the highest average accuracy and the lowest estimated downstream token cost on ViDoRe V3, BRIGHT and BrowseComp-Plus.

Architecture and compression pipeline (how they did it)

Technically relevant but explained plainly:

• The 8B adapts the Ministral-3-8B-Instruct backbone by converting its causal decoder into a bidirectional encoder for full-sequence embeddings. It was trained with contrastive pretraining and then fine-tuned on multilingual and domain-specific datasets (legal, medical, finance, etc.).

• The 1B is not a tiny retriever built from scratch. They started from Ministral-3-3B-Instruct to create a 3B base, then applied structured pruning and two rounds of distillation using ModelOpt and a NAS (architecture search) pipeline to compress to 2B and finally to ~1.14B.

• Compression included topology search (width, FFN, attention, depth), distillation from the 8B teacher with combined losses (cosine distance + MSE) and a two-stage context-scaling training (1024 tokens then 4096 tokens) to retain recall on long sequences.

• For hardware efficiency, the NVFP4 variant quantizes weights and activations to NVFP4 on Blackwell GPUs (GB200). They use Quantization-Aware Distillation (QAD) to recover precision on long inputs and preserve 99%+ of BF16 accuracy while reducing memory footprint.

Deployment, stack and production performance

Options available from day one:

• Hugging Face: weights, model cards and examples for SentenceTransformers, Transformers and vLLM.
• NVIDIA NIM: an optimized microservice (Rust) for production; NIM matches or outperforms the vLLM checkpoint on GB200 and RTX PRO 6000 in ISLs of 256 and 1024.
• Integrations with AI Cloud and inference partners: Baseten, Bitdeer AI, DeepInfra, Friendli AI, OpenRouter, and more.

Practical performance notes:

• NVFP4 on Blackwell can offer up to 2x throughput versus BF16 in high-demand scenarios.
• The 1B-NVFP4 variant keeps almost all accuracy while cutting memory use — ideal when query volume is massive.

Which model to choose for your use case?

• If your priority is enterprise RAG quality, compliance or legal/medical answers: use Nemotron-3-Embed-8B-BF16.
• If you need to serve production with latency and budget constraints: Nemotron-3-Embed-1B-BF16 is the typical balance.
• If you run on Blackwell infra and need massive throughput and very low cost per query: Nemotron-3-Embed-1B-NVFP4.

Concrete example: for agent memory and code search in companies handling thousands of simultaneous conversations, the NVFP4 variant can be the difference between paying X or 0.5X in monthly infra costs.

Fine-tuning, distillation and real-world cases

NVIDIA opens NeMo AutoModel recipes for:

• Fine-tuning: adapt the model to your corpus. Example: in NV Docs, fine-tuning the 1B-BF16 raised NDCG@10 from 56.7% to 63.3% (+11.6%) and Recall@5 from 56.1% to 62.8% (+11.9%).

• Distillation: shrink footprint without losing relevant ranking, useful for edge deployments or strict latency requirements.

Customers and partners already testing the collection include Automation Anywhere, Boomi, IBM, Mem0, Palantir, ServiceNow, You.com, Zep and Zoom. Reports show substantial gains in snippet selection, agent memory and enterprise documentation retrieval.

Engineering considerations

• Keep an evaluation pipeline that measures not only NDCG or Recall but also downstream token cost in your agents. A better retriever can save tokens and inference time on expensive models.
• If you plan to use NVFP4, validate with QAD and long-tail tests (long documents, multi-file code) to ensure precision retention.
• Use the open recipes if you need compliance or control over sensitive data: having weights and recipes lets you audit and adapt without relying on a closed service.

If you’re building RAG, agent memory or code retrieval, Nemotron 3 Embed gives you a clear set of options: top quality, production-ready balances and compression paths to scale.

Original source

https://huggingface.co/blog/nvidia/nemotron-3-embed-wins-rteb