NVIDIA Releases Nemotron-Labs-TwoTower, a Diffusion Language Model That Doubles Generation Throughput Without Retraining

Overview

NVIDIA has released Nemotron-Labs-TwoTower, an open-weight diffusion language model built on top of a frozen copy of the company’s existing Nemotron-3-Nano-30B-A3B backbone, according to MarkTechPost. Rather than training a new model from scratch, NVIDIA researchers adapted the pretrained autoregressive model to generate text in parallel blocks instead of one token at a time, and the resulting system retains 98.7% of the autoregressive baseline’s quality while offering 2.42X higher wall-clock generation throughput, according to the paper published on arXiv.

What We Know

The model, formally titled “Nemotron-Labs-TwoTower: Diffusion Language Modeling with Pretrained Autoregressive Context,” was authored by NVIDIA researchers Fitsum Reda, John Kamalu, Roger Waleffe, Mostofa Patwary, Mohammad Shoeybi, and Bryan Catanzaro, and first posted to arXiv on June 25, 2026, with a revised version following on June 29.

The paper frames the work around a specific limitation in existing diffusion language models: “existing approaches use a single network for both context representation and iterative denoising, forcing one model to serve both roles and limiting its capacity for either role,” according to the paper. TwoTower’s fix is architectural — the authors “propose TwoTower, a block-wise autoregressive diffusion model that decouples these roles into two towers: a frozen AR context tower that causally processes clean tokens, and a trainable diffusion denoiser tower with bidirectional block attention that refines noisy blocks via cross-attention to the context,” the paper states.

Both towers share the same 52-layer hybrid design as the underlying Nemotron-3-Nano-30B-A3B backbone — 23 Mamba-2 layers, 6 self-attention layers, and 23 mixture-of-experts layers, according to MarkTechPost and the model’s Hugging Face card. The released checkpoint carries roughly 60 billion total parameters, with about 3 billion active per token, and its mixture-of-experts layers route among 128 experts, activating 6 per token alongside 2 shared experts, according to the Hugging Face model card.

The key efficiency of the approach is that NVIDIA did not have to retrain the whole system. The AR context tower stays frozen, reusing the backbone’s original pretraining on roughly 25 trillion tokens, while only the denoiser tower was trained, on approximately 2.1 trillion tokens under a masked-diffusion objective, according to the Hugging Face model card. That denoiser training volume is a small fraction of the backbone’s original pretraining budget.

At its default inference settings — a confidence threshold of 0.8 and a block size of 16 — the model achieves “2.42× the AR baseline’s wall-clock generation throughput” while retaining “98.7% of the AR baseline’s aggregate benchmark quality,” according to the Hugging Face model card. MarkTechPost’s benchmark comparison shows the quality trade-off is uneven across tasks: on MMLU, the autoregressive baseline scored 78.56 versus TwoTower’s 78.24, a marginal gap, while on HumanEval the baseline’s 79.27 fell to 75.58 for TwoTower, and on GSM8K the baseline’s 92.49 dropped to 90.14, according to MarkTechPost.

The released checkpoint supports three inference paths — full two-tower diffusion, a “mock-AR” mode, and an AR-only mode — according to MarkTechPost. NVIDIA has published the weights on Hugging Face under the NVIDIA Nemotron Open Model License, with the model card describing it as ready for commercial use, according to the Hugging Face model card.

What We Don’t Know

Neither the paper nor the model card discloses the specific hardware configuration or wall-clock time used to measure the 2.42x throughput figure, so it is unclear how the gain scales across different GPU setups or batch sizes. It is also not yet clear whether NVIDIA plans to apply the same frozen-backbone, trained-denoiser approach to larger models in the Nemotron family beyond the 30B-parameter Nano variant.

Analysis

The TwoTower release is notable less for its raw performance numbers than for what it demonstrates about model economics: a lab with an existing autoregressive checkpoint can add diffusion-style parallel generation without the cost of training a new foundation model, by training only a second denoising network on a fraction of the original data budget. Given that the quality trade-off is uneven — near parity on knowledge benchmarks like MMLU but a more noticeable drop on coding (HumanEval) and math (GSM8K) tasks — the approach looks best suited to latency-sensitive applications where some quality loss on generation-heavy tasks is an acceptable trade for faster throughput, rather than as a wholesale replacement for autoregressive inference.