AIbase Report Latest news in Beijing Time, domestic AI unicorn MiniMax is about to launch its new large model M3. Skyler Miao, the AI engineering lead at MiniMax, recently released a teaser on social media, saying "Something BIG is coming!" which has attracted widespread attention in the industry.

image.png

M3 Core Architecture Innovation: Sparse Attention Mechanism

According to the information, M3 adopts a new sparse attention (Sparse Attention) architecture, combining fast indexing through the Index Branch with precise computation through the Sparse Branch, effectively solving the computational bottlenecks in ultra-long context scenarios.

Traditional Transformers face a quadratic growth in computational load when handling contexts of millions of tokens. However, M3's sparse design significantly reduces this cost, achieving a notable efficiency leap while maintaining high performance, providing strong support for applications such as long text understanding, long conversations, and multi-document analysis.

Test Performance Significantly Outperforms M2

Compared to its predecessor M2 (which supports 1M token context), M3 has achieved breakthrough improvements in key metrics:

  • Speed in the Prefill phase increased by 9.7 times
  • Speed in the Decoding phase increased by 15.6 times

This means that in practical deployment, M3 can efficiently process ultra-long contexts with minimal computational costs, significantly reducing inference costs and opening up new possibilities for more complex AI applications.

Industry Implications: A New Benchmark for Efficiency in the Era of Long Contexts

MiniMax's announcement of M3 once again highlights the competitiveness of domestic AI companies in architectural innovation. The breakthroughs in technologies like sparse attention are expected to shift the focus of large models from "competition in parameter scale" to "competition in efficiency and practicality," bringing more affordable and efficient experiences for enterprise-level applications and consumer use.

At present, MiniMax has not yet announced the specific release date or full parameter scale of M3. However, based on the engineer's teaser and the performance data, this model is expected to become a strong competitor in the field of long context processing. AIbase will continue to monitor the subsequent developments of MiniMax M3 and bring you the latest updates in a timely manner.