[AIbase Report] The Meta Foundation AI Research (FAIR) team recently announced the launch of Omnilingual ASR, an innovative automatic speech recognition system capable of transcribing more than 1600 spoken languages. This move aims to bridge the significant gap in language coverage of existing AI tools, moving towards the goal of a "universal transcription system."

For a long time, most speech recognition systems have focused on a few languages with large amounts of audio recording resources, resulting in thousands of languages among the world's 7000 languages receiving almost no AI support. The release of Omnilingual ASR will change this situation. Meta pointed out that among the 1600 languages it supports, 500 languages had never been covered by any artificial intelligence system before.

Key Highlights: Emphasizing Accuracy and Scalability

The performance of Omnilingual ASR is impressive:

  • In tests of 1600 languages, the system achieved an error rate of less than 10 characters for 78% of the languages.

  • For "resource-rich" languages with at least 10 hours of training audio, this accuracy standard reached 95% coverage.

  • Even for "low-resource" languages with less than 10 hours of audio, 36% of the languages had an error rate below the threshold of 10 characters, providing practical speech recognition capabilities for these communities.

Situational Learning: Expanding Coverage to 5400 Languages

A key innovation of Omnilingual ASR is its "self-language" option, which draws on situational learning techniques from large language models. Users only need to provide a small number of audio and text pairing samples, and the system can directly learn new languages from these samples, without retraining or requiring significant computational resources.

Meta stated that theoretically, this approach has the potential to expand the coverage of Omnilingual ASR to more than 5400 languages, far exceeding current industry standards.

Open Source Ecosystem and Research Support

To support further research and applications, Meta has adopted a comprehensive open source strategy:

  1. Model Open Source: Omnilingual ASR is released under the Apache 2.0 license, allowing researchers and developers to freely use, modify, and build upon the model, including for commercial purposes. The model is built on the PyTorch-based fairseq2 framework, offering versions ranging from a 300 million parameter version suitable for low-power devices to a 7 billion parameter version aimed at achieving "top accuracy."

  2. Dataset Release: Meta also released the Omnilingual ASR Corpus, a large transcribed speech dataset containing 350 underrepresented languages. This dataset is released under the **Creative Commons Attribution License (CC-BY)**, aiming to help global developers adjust speech recognition models to meet specific local needs.