Recently, the Qwen team officially released the multimodal MoE model Qwen3.8-Flash and simultaneously open-sourced the next-generation underlying architecture Qwen3.8-Flash-Next, which is also the prototype of the new generation Qwen4 series model.

The Qwen3.8-Flash has a total parameter count of 125B, with an additional 51B N-gram Embedding, but only 6B parameters are activated per token. The model natively supports a context of 260,000 tokens, and can be extended to 1,000,000 tokens using YaRN technology. It shows significant cost advantages, with a training cost that is only one ninth of its predecessor Qwen3.7-Plus. The API pricing is 1 yuan for every million input tokens and 3 yuan for every million output tokens. It has already been launched on the Qwen AI platform, and the "Qwen Office" feature has also been introduced.

QQ20260827-092838.png

In terms of architecture, the team completed four core upgrades. The attention module adopts a GDN+QSA hybrid architecture, where GDN is responsible for compressing historical information, and QSA selects key contexts, achieving up to 7.6 times and 4.9 times speedup in Prefill and Decode respectively in scenarios with a million tokens; the Gated Residual mechanism expands the residual flow into four parallel branches, optimizing cross-layer information transmission through dynamic gating, thereby improving training stability; the N-gram Embedding uses local context lookup tables to expand model capacity, adding 51B parameters without significantly increasing computational load; during the training process, a mixed optimization strategy of Muon and AdamW was adopted, re-fitting the Scaling Law and eliminating the batch warm-up process.

Multiple benchmark tests show that, relying on 6B activated parameters, the base model achieved the best results in 8 out of 14 evaluations. The fine-tuned version performed well in code, agents, and multimodal tasks, with SWE-bench Pro reaching 62.5 points, and leading significantly over other models in intelligent agent rankings such as CoWorkBench and Toolathlon.

Currently, the weights of Qwen3.8-Flash-Next are open-sourced on Hugging Face and ModelScope, and the complete technical report is also publicly available. The Qwen team has proactively opened the architecture, hoping to validate the innovative solutions with the community's help, continuously iterate the technology, and steadily advance the development of the Qwen4 series model.