Today, with the rapid iteration of multimodal large models, audio processing capabilities are often "sacrificed" — many models enhance audio understanding while causing a decline in text logical abilities. Recently, the NVIDIA research team officially released a unified audio-text large language model called Nemotron-Labs-Audex-30B-A3B (referred to as Audex), aiming to solve this technical challenge.

The design concept of Audex is very simple and efficient. It is built upon a powerful pure-text MoE (Mixture of Experts) architecture, using a single Transformer decoder to achieve unified processing of text and quantized audio tokens. This architecture not only allows audio input to be smoothly projected into the text embedding space but also ensures that the model can seamlessly integrate with existing LLM infrastructure when handling multimodal tasks, achieving true deep integration.

To train this powerful model, the research team compiled massive data, covering 157.4 billion audio tokens and 320.5 billion text tokens. Through multi-stage supervised training, pure-text Cascade RL (Reinforcement Learning), and multi-domain in-policy knowledge distillation, Audex performs excellently on various metrics. It not only reaches industry-leading levels in tasks such as audio understanding, speech recognition, translation, and audio generation, but more importantly, it almost perfectly retains the core capabilities of the original LLM in reasoning, alignment, knowledge base, and long text processing, with minimal performance degradation.

As an open-source model, the release of Audex is undoubtedly a positive signal for the voice technology industry. It is no longer just a research demo that remains at the paper demonstration stage, but rather a mature tool that developers can directly evaluate and deploy. For product developers who need to handle complex audio interactions, Audex provides a new choice that balances performance and functionality, and also opens new doors for future research on multimodal intelligences.