Nvidia has announced the Nemotron 3 Super, a 120-billion-parameter hybrid model designed to improve agentic reasoning workflows within enterprises. By combining state-space models, transformers, and a Latent mixture-of-experts design, Nemotron 3 Super offers specialized depth without the typical bloat of dense reasoning models. This innovation aims to address the challenge of handling long-horizon tasks efficiently, such as software engineering and cybersecurity triaging.
The core of Nemotron 3 Super lies in its Triple hybrid architecture, featuring a Hybrid Mamba-Transformer backbone that balances memory efficiency and precision reasoning. Additionally, the model introduces Latent Mixture-of-Experts (LatentMoE) for expert compression, enabling consultations with more specialists at the same computational cost.
Moreover, Nemotron 3 Super leverages Multi-Token Prediction (MTP) for accelerated structured generation tasks, predicting multiple future tokens simultaneously to enhance speed and efficiency.
One of the key advantages of Nemotron 3 Super is its optimization for the Nvidia Blackwell GPU platform, delivering 4x faster inference compared to previous architectures without compromising accuracy.
Released under the Nvidia Open Model License Agreement, Nemotron 3 Super offers commercial usability with specific provisions for enterprise users, emphasizing ownership of outputs and the ability to create derivative models with attribution.
This model has already gained traction among industry leaders, with companies like CodeRabbit, Greptile, Siemens, and Palantir adopting it for various applications, from large-scale codebase analysis to automating workflows in manufacturing and cybersecurity.
Source: VentureBeat