At the recent AICon conference, Zhou Jun, vice president of Ant Group, delivered a notable speech focusing on how to improve the efficiency of trillion-parameter models. Zhou mentioned that the computing power consumption of modern AI models is astonishing, especially for trillion-parameter models, where the computing cost for 15 minutes of operation is equivalent to the price of a Tesla. This undoubtedly serves as a warning that improving efficiency has become a core issue that needs to be addressed in the era of intelligent agents.

To address this challenge, Zhou and his team proposed a revolutionary strategy: shifting from "more Tokens" to "higher Token density." They adopted a hybrid linear attention architecture called "7 parts Lightning Attention plus 1 part MLA," which significantly reduced the cost of processing 256K-length contexts from exponential to linear, allowing computing power to be used more efficiently for thinking.

image.png

In addition, the team introduced the Kpop algorithm to better distinguish tool calls from natural language Tokens, while combining techniques such as chain-of-thought pruning and self-distillation. These innovations reduced Token output by about four times without compromising model capabilities. Zhou pointed out that their billion-parameter model has exceeded some larger models on Agent tasks in multiple benchmark tests such as LongBench and BFCL. Excitingly, the flash throughput of the small model has reached 2.4 times, and the computing cost for five-round conversations has been reduced by more than ten times.

image.png

In short, Zhou Jun's speech not only brought new ideas to the industry but also provided a feasible reference for future intelligent agent design. With the increasing demand for computing power, improving the efficiency of large models has become particularly important, and the series of innovations proposed by Ant Group have undoubtedly brought new hope to the industry.

Key points:

🌟 Zhou Jun emphasized that the computing cost of trillion-parameter models is equivalent to that of a Tesla, highlighting the urgent need to improve efficiency.

🔑 The team proposed shifting from "more Tokens" to "higher Token density," with an innovative architecture significantly reducing the cost of context processing.

🚀 The Kpop algorithm and other technologies were adopted, reducing Token output without sacrificing capability, and the billion-parameter model performed well in multiple benchmark tests.