The innovation pace in the open-source large model field has once again set a new record. Recently, the Tongyi Qianwen team from Alibaba officially open-sourced a new multimodal MoE model, Qwen3.8-Flash-Next. As an early preview version of the Qwen4 architecture, this model demonstrates exceptional performance while maintaining extremely low computational costs.

Regarding core architecture and parameter scale, Qwen3.8-Flash-Next adopts a highly sparse mixture-of-experts (MoE) design. The total number of parameters in the model is 125 billion, with an additional 51 billion parameters for the N-gram embedding table and 4 billion parameters for the multi-token prediction module. However, only 6 billion parameters are activated per token. The model features a 48-layer network structure, with MoE layers integrating 512 experts and using top-10 routing for allocation.

In terms of technological innovation, the model integrates multiple cutting-edge upgrades. It introduces a hybrid attention mechanism combining GDN and QSA, balancing efficiency in long-text processing tasks with information retrieval accuracy. The original 256K context window was successfully extended to 1M using YaRN technology. Additionally, by introducing a 51 billion parameter N-gram embedding, it significantly expands capacity without increasing computational costs, and supports keeping it in system slow DRAM memory through asynchronous prefetching. Furthermore, it adopts gated residual connections (GR) and sparse attention designs with micro-block granularity for context selection.

In terms of actual performance and cost, the model delivered impressive results. Benefiting from its extreme sparsity design, the training cost is only one-ninth of the previous generation, Qwen3.7-Plus, while achieving significant improvements in core tasks such as code writing and office work. Multiple evaluations and recent reports confirm that, with only 6 billion activated parameters, the model outperformed top-tier models like Claude Opus 4.6 Max.

In terms of ecosystem deployment and open-source rhythm, SGLang provided Day-0 level direct support on the day of the model release. Developers can obtain model weights through open-source channels and efficiently deploy them locally or in cluster environments. These innovations in underlying architecture not only provide cutting-edge technical exploration samples for the global open-source community but also further promote the evolution of high-performance AI large models toward cost-effective, efficient large-scale deployment.