RWKV: Small Team Aims to Be Android of AI Era with Big Model

Source:

Source:
Recently, Meta has announced a groundbreaking research achievement by developing a new type of memory layer technology that can significantly enhance the factual accuracy of large language models (LLMs) and achieve unprecedented expansion in parameter scale. This technology not only challenges traditional neural network expansion methods but also provides new directions for future AI architecture design. The core of this research lies in utilizing a trainable key-value lookup mechanism to add extra parameters to the model without increasing computational load (FLOPs). This method's core thinking is...
Recently, the National Development and Reform Commission, the National Data Bureau, and the Ministry of Industry and Information Technology jointly released the 'Guidelines for National Data Infrastructure Construction.' These guidelines aim to promote the active construction of government service large models across the country, facilitate the intelligent process of government services, and thereby improve the efficiency and quality of government services. The guidelines emphasize the importance of the data annotation industry and encourage regions to explore and innovate in the construction of data annotation ecosystems, capacity building, and application scenarios. The government will link public data, proactively disclose data of enterprises and individuals, and establish high-quality data assets.
On January 6, 2025, Kunlun Wanwei Group announced the official launch of its 'Tiangong Model 4.0' o1 and 4o versions, which are now available for free on the Tiangong website and app. The release of these two models marks another significant advancement for Kunlun Wanwei in the field of artificial intelligence. The 'Tiangong Model 4.0' o1 version (Skywork o1) is the first domestic model with Chinese logical reasoning capabilities. With comprehensive technology stack upgrades and model optimizations, it is capable of proficiently handling tasks involving mathematics, coding, logic, common sense, and ethical decision-making.
In 2024, the investment frenzy in generative artificial intelligence (Generative AI) remains unabated, with global investment reaching new heights. According to data from financial tracking firm PitchBook, generative AI companies raised a total of $56 billion in venture capital last year, with a total of 885 deals. This marks an impressive 192% increase compared to $29.1 billion in 2023. Prominent companies in this field include OpenAI and Anthropic.
In nature, animals communicate through various sounds, from dolphins' whistles to elephants' rumbles and birds' chirps. Each sound contains specific patterns and structures. These subtle differences in sound are difficult for humans to recognize, but the pattern recognition capabilities of artificial intelligence (AI) provide new possibilities for decoding these 'wild calls.' Shane Gero, a whale biologist from Carleton University in Canada, has spent 20 years studying how whales communicate. He discovered that the same...