A startup company called Acrab has officially launched its first edge AI end SoC today - GΞLIX 1. This 5nm process chip is designed to deliver a hard metric: enabling large models with up to 100 billion parameters to run locally without sending data to the cloud.

On the hardware side, GΞLIX 1 uses a heterogeneous computing architecture, integrating 20-core Arm CPU, 3TFLOPS GPU, dedicated NPU, and audio and video processing engine into a single chip, achieving a total AI computing power of 650TOPS. Compared to others, it's clearer: the NPU power of Qualcomm Snapdragon X Elite is about 45TOPS, while Intel Core Ultra series NPU is around 40TOPS - GΞLIX 1's AI computing power is nearly ten times that of them.

To satisfy the insatiable appetite of large model inference, the cache and memory subsystems are specifically designed for high throughput requirements. The chip is equipped with an 8MB L1 cache, sharing a L2 cache bandwidth of 768GB/s, and is connected to a 256-bit wide LPDDR5X memory with a transfer rate of 8533MT/s. During large model inference, weight reading and KV Cache access are highly sensitive to memory bandwidth, and this configuration forms the foundation for running a 100B model on the edge.

This time, Acrab focuses on showcasing performance during the prefilling phase. Prefilling is the process where the large model fully encodes the entire input after receiving user input but before starting to generate output token by token. Its speed directly determines the first Token delay in long text input. Acrab accelerates the Transformer attention mechanism calculation using vector hardware units and makes targeted optimizations for the prefilling path. In the official test, under the Gemma 26B A4B configuration with 40k KV cache and 10k Token input, this chip achieved 1416.8 Token/s, which is 7.5 times faster than Apple M4 Pro Mac Mini.

On the software side, Acrab also released a complete software stack and an agent reference design, covering from the underlying drivers to the upper application framework. The first-party terminal device Agent Box was also unveiled, serving as a reference implementation of GΞLIX 1 for developers to evaluate and prototype verification.