NVIDIA has recently released the Nemotron3Embed series of embedding vector models, specifically designed for production-level RAG, intelligent question retrieval, code retrieval, and agent memory scenarios. The 8B version ranks first on the retrieval embedding benchmark RTEB, becoming the most powerful open-source model in this field.

The series includes three open checkpoints: Nemotron-3-Embed-8B-BF16 (precision-focused), Nemotron-3-Embed-1B-BF16 (lightweight version), and Nemotron-3-Embed-1B-NVFP4 (4-bit version optimized for Blackwell architecture). All three use a Transformer encoder with bidirectional attention masking training, have a maximum sequence length of 32,768 tokens, support 34 languages, and are open-sourced under the OpenMDW-1.1 license. Notably, all base models are based on the Mistral architecture, with the 8B version derived from Ministral-3-8B-Instruct-2512, and the two 1B variants based on Ministral-3-3B-Instruct-2512.

image.png

In the 16 public tasks of the RTEB benchmark, the 8B-BF16 version ranked first with an average NDCG@10 score of 78.46. The 1B-BF16 version scored 72.38, improving by 10.4 points compared to the previous baseline llama-nemotron-embed-vl-1b-v2. The 1B-NVFP4 version optimized for Blackwell only lost 0.38 points, equivalent to retaining 99.5% of the precision, while achieving twice the throughput of BF16 on the Blackwell architecture.

The construction of the 1B model adopted a compression rather than a small-scale training approach. The research team first used NVIDIA ModelOpt's neural architecture search to prune the 3B base model to 2B, then performed knowledge distillation from the fine-tuned 8B embedding vector teacher model using cosine distance loss and mean square error loss, iterating to 1.14B parameters. The NVFP4 version further conducted quantization-aware distillation, calibrating with 512 samples and training with 20,000 samples, restoring precision in long input scenarios.

image.png

On the deployment level, the three versions differ: both the 8B and 1B BF16 versions support Transformers and Sentence Transformers frameworks, while the 1B-NVFP4 only supports the vLLM0.25.0 /v2/embed interface. In terms of microarchitecture coverage, the NVFP4 version is compatible with Ampere, Hopper, Lovelace, and Blackwell, whereas the BF16 versions mainly target Ampere, Hopper, and Blackwell. NVIDIA also released an optimized NIM microservice for the 1B model, built with Rust, achieving or surpassing the performance of the vLLM checkpoint on GB200 and RTX PRO6000.

In terms of application scenarios, the series supports multilingual enterprise search (cross-language retrieval), code retrieval (training data includes code datasets such as SWE-bench), and agent memory (32K token long context support for longer conversation summaries). For cost-sensitive scenarios, a hierarchical RAG strategy of "1B-NVFP4 handling high-capacity recall + 8B handling difficult queries" can be adopted.

NVIDIA also provides complete code examples, covering local inference based on Sentence Transformers and server-side deployment based on vLLM. Queries and documents are distinguished by the `query:` and `passage:` prefixes, and the dot product of the embedded vectors after L2 normalization equals the cosine similarity.