{"id":91688,"date":"2026-05-20T16:48:48","date_gmt":"2026-05-20T16:48:48","guid":{"rendered":"https:\/\/youzum.net\/nvidia-ai-releases-nemotron-labs-diffusion-a-tri-mode-language-model-with-6x-tokens-per-forward-over-qwen3-8b\/"},"modified":"2026-05-20T16:48:48","modified_gmt":"2026-05-20T16:48:48","slug":"nvidia-ai-releases-nemotron-labs-diffusion-a-tri-mode-language-model-with-6x-tokens-per-forward-over-qwen3-8b","status":"publish","type":"post","link":"https:\/\/youzum.net\/fr\/nvidia-ai-releases-nemotron-labs-diffusion-a-tri-mode-language-model-with-6x-tokens-per-forward-over-qwen3-8b\/","title":{"rendered":"NVIDIA AI Releases Nemotron-Labs-Diffusion: A Tri-Mode Language Model with 6\u00d7 Tokens Per Forward Over Qwen3-8B"},"content":{"rendered":"

NVIDIA researchers have released Nemotron-Labs-Diffusion, a language model family that unifies three decoding modes in one architecture. The model supports autoregressive (AR) decoding, diffusion-based parallel decoding, and self-speculation decoding. It is available in 3B, 8B, and 14B parameter sizes. The family includes base, instruct, and vision-language variants.<\/p>\n

Sequential Decoding Limits Throughput<\/strong><\/h2>\n

Standard autoregressive (AR) language models generate text one token at a time, left to right. Each token depends on all previous tokens. This sequential dependency limits GPU parallelism per generation step. The result is low hardware utilization at low batch sizes \u2014 the typical setting for single-user or edge deployment.<\/p>\n

Diffusion language models (LMs) offer a different approach. Instead of generating tokens sequentially, they denoise multiple tokens in parallel per forward pass. This enables higher throughput. The tradeoff has been accuracy: diffusion LMs have consistently lagged behind AR models on benchmarks, requiring substantially more data to reach comparable performance. A key reason is that diffusion training treats all token permutations uniformly, rather than leveraging the strong left-to-right prior inherent in natural language.<\/p>\n

\n
\"\"
https:\/\/d1qx31qr3h6wln.cloudfront.net\/publications\/Nemotron_Diffusion_Tech_Report_v1.pdf?VersionId=db8_EMO8B.vmU26.jr7Le9pN3MqcUDNL<\/figcaption><\/figure>\n<\/div>\n

What Is a Tri-Mode Language Model?<\/strong><\/h2>\n

Nemotron-Labs-Diffusion is trained on a joint AR-diffusion objective. At inference time, it operates in three modes depending on the deployment context. There are no mode-specific architectural modifications \u2014 the same weights serve all three modes.<\/p>\n

AR mode<\/strong> is standard left-to-right autoregressive decoding using causal attention. This mode is best suited for high-concurrency cloud serving.<\/p>\n

Diffusion mode<\/strong> denoises multiple tokens in parallel within a fixed-length block. The sequence is partitioned into contiguous blocks. Within each block, tokens attend bidirectionally. Across blocks, attention remains causal, so prior blocks can reuse their KV cache. A lightweight trained sampler predicts, per masked position, whether the model\u2019s top-1 prediction at the current denoising step is correct. Positions predicted as correct are committed in that step. This allows the model to commit multiple tokens per forward pass.<\/p>\n

Self-speculation mode<\/strong> uses the diffusion pathway to draft candidate tokens and the AR pathway to verify them, within the same single model. No auxiliary draft model or separate prediction head is required. The diffusion pathway generates a block of k candidate tokens in parallel. The AR pathway then runs a second forward pass over those candidates using causal attention, verifying the longest contiguous prefix that matches AR predictions. Each cycle produces between 1 and k+1 verified tokens. This contrasts with Multi-Token Prediction (MTP) methods such as Eagle3, which use small auxiliary draft heads attached to an AR backbone.<\/p>\n

Training<\/strong><\/h2>\n

The joint training objective combines an AR next-token prediction loss and a block-wise diffusion denoising loss:<\/strong><\/p>\n

\u2112(\u03b8) = \u2112_AR(\u03b8) + \u03b1 \u00b7 \u2112_diff(\u03b8)<\/strong><\/p>\n

The coefficient \u03b1 is set to 0.3 across all training stages. Ablation experiments varying \u03b1 from 0.1 to 1.0 show that both AR-mode and diffusion-mode accuracy peak at \u03b1 = 0.3. No value in the range [0.1, 0.5] improves one mode at the expense of the other \u2014 the two objectives rise and fall together.<\/p>\n

Two-stage training<\/strong> first trains the model purely on the AR objective for 1 trillion tokens, building strong left-to-right linguistic priors. Stage 2 then introduces the joint objective for 300 billion additional tokens. In ablations, two-stage training contributed +5.74% average accuracy. Adding the AR loss contributed the single largest gain at +7.48%. Global loss averaging \u2014 treating all tokens across a batch equally rather than averaging per-sequence first \u2014 contributed +2.12% by reducing gradient variance from variable diffusion masking ratios. Cumulatively, the full training pipeline improved the baseline by 16.05% average accuracy.<\/p>\n

All models are initialized from pretrained Ministral3 base models, not trained from scratch. Training was performed on 256 NVIDIA H100 GPUs. Instruct models are trained via supervised fine-tuning (SFT) on 45 billion tokens on top of the base models, using the same joint AR-diffusion objective with \u03b1 = 0.3. The training and inference pipeline is released through Megatron Bridge.<\/p>\n

LoRA-Enhanced Linear Self-Speculation<\/strong><\/h2>\n

The base diffusion-to-AR alignment in self-speculation can be improved with a LoRA adapter. This adapter is fine-tuned on the diffusion draft pathway to better align its output with the AR verifier. It targets only the o_proj layer of the attention module (rank 128, \u03b1 = 512, approximately 36M trainable parameters, 0.4% of the backbone). LoRA tuning improves tokens per forward (TPF) by 14.4%, 32.5%, and 27.6% at the 3B, 8B, and 14B scales respectively, with negligible accuracy change.<\/p>\n

Speed-of-Light Analysis<\/strong><\/h2>\n

The research team reports a speed-of-light (SOL) analysis \u2014 a theoretical upper bound on tokens per forward pass achievable by the diffusion mode, assuming an oracle sampler that correctly identifies all positions that can be safely committed in parallel.<\/p>\n

At block length 32, the SOL acceptance rate reaches 7.60\u00d7 on average, exceeding 10\u00d7 on coding and multilingual tasks. Current confidence-based sampling achieves approximately 3\u00d7 TPF at comparable accuracy, leaving a large gap to the SOL ceiling.<\/p>\n

Comparing against linear self-speculation: both approach similar acceptance rates (6.82\u00d7 for linear self-speculation vs. 7.60\u00d7 SOL). However, the real tokens per forward pass (TPF) gap is much larger \u2014 6.02\u00d7 for SOL versus 3.41\u00d7 for linear self-speculation, a 76.5% difference. Linear self-speculation requires two forward passes per cycle (one diffusion draft, one AR verify) and accepts only a contiguous prefix. These two constraints cap its real TPF well below SOL, even when drafter and verifier are well aligned.<\/p>\n

\n
\"NVIDIA
https:\/\/d1qx31qr3h6wln.cloudfront.net\/publications\/Nemotron_Diffusion_Tech_Report_v1.pdf?VersionId=db8_EMO8B.vmU26.jr7Le9pN3MqcUDNL<\/figcaption><\/figure>\n<\/div>\n

Benchmark Results<\/strong><\/h2>\n

On the 10-task instruct evaluation (HumanEval, MBPP, LiveCodeBench-CPP, GSM8K, Math500, AIME24, AIME25, GPQA, IFEval, MMLU):<\/strong><\/p>\n