On the topic of local speech transcription, developers have long been stuck in an awkward dilemma: either use whisper.cpp or ONNX, and if they want to run on Apple devices, they also have to include MLX, which means maintaining two sets of engines and porting each model separately. On July 19th, sebjones, the author and maintainer of the well-known speech-to-text app Handy, turned the problems he encountered while distributing cross-platform applications into a new library called transcribe.cpp, and released version v0.1.0. This is a speech transcription library based on ggml, supporting all the latest transcription models, and every model released by the Hugging Face organization handy-computer has undergone numerical validation and word error rate (WER) testing, ensuring consistency with the official reference implementation. The author also admitted that this is a 0.1.0 version, and there may be rough edges that he hasn't discovered, and he welcomes the community to report issues for improvement.

The emergence of transcribe.cpp was driven by a series of frustrating realities. sebjones explicitly stated that using existing ASR inference stacks to distribute cross-platform applications results in a very poor experience. Although there are some scattered libraries claiming support for many models, their authors are unknown, and the tests are unclear, leaving him with a series of questions: When will this library stop being updated? Has the author considered official bindings so you can use it in real desktop or mobile applications? Is it essentially just demo code? Have any benchmark tests been done? Is it faster than ONNX? As someone who needs to integrate voice functionality into Handy, he wanted a library that allows you to download model files and run inference directly, with inference results matching the reference implementation; the inference should run on the GPU for optimal performance, and it should be easy to embed into Handy, without the burden of a large PyTorch framework; it must be compatible with Mac, Windows, and Linux. After careful consideration, ggml became his best choice, with a strong community and excellent distribution capabilities.

image.png

Regarding specific capabilities, transcribe.cpp delivers a fast and accurate inference engine with extensive model coverage. It supports 16 ASR model families, totaling over 60 models, with more models coming soon. In terms of acceleration, it moves inference to the GPU through four paths: Vulkan, Metal, CUDA, and TinyBLAS. The author especially considers Vulkan as the baseline for any local inference application, and has conducted benchmark tests for each supported model on a Ryzen4750U (CPU plus Vulkan) in the Fedora system and his own M4Max. Each model has undergone numerical validation and full WER scanning, meaning they have been refined through thousands of audio segments, and their outputs are very close to or even completely consistent with the reference implementation. These validation results are also publicly available in the transcribe.cpp repository and the corresponding model pages on Hugging Face. Functionally, it supports streaming transcription and batch transcription, and it can basically serve as a plug-and-play alternative to whisper.cpp — Handy originally ran on whisper.cpp, and the author is using transcribe.cpp to release an update to replace it. To this end, he specifically maintained compatibility with the popular .bin files from whisper.cpp that are distributed with Handy, allowing transcribe.cpp to run these files directly; although some flags and features are not yet aligned, its whisper implementation is sufficient and reliable for most use cases, with performance roughly equivalent.

What makes this library promising for real-world products is that it prioritized language bindings from day one. The library itself is written in C/C++, and the author not only needs Rust bindings but also understands that to widely distribute local speech transcription, there must be decent official bindings. He selected four languages with the most representative coverage: Python, JavaScript and TypeScript, Rust, and Objective-C and Swift. He also welcomes the community to contribute more bindings if they are willing to take on maintenance. These decisions were largely driven by Handy's requirements, and Handy's popularity itself has given the author the motivation to continue maintaining this library.

In the author's view, the ultimate goal of transcribe.cpp is to make local ASR easier to use. Speech transcription can achieve very high accuracy on most devices, so there's no need to send audio to the cloud. He gave a hard-core example: on the RK3566 chip, which has weak performance, transcribe.cpp can run the model faster than real-time on the CPU, and with the most advanced models, it can do real-time transcription with power consumption of just a few watts. He believes that in the future, due to various reasons, more inference will occur locally, making distribution issues extremely important. Although transcribe.cpp is far from solving this issue overall, it hopes to be a small step forward.

In the acknowledgments, the author specifically thanked several contributors: Mozilla AI and its BiR project and engineer Davide, who decided to support him when the project was still just a vague idea; ggml and its contributors, who formed the foundation of the entire project; Modal, which provided computing credits for WER testing and CUDA verification; Blacksmith, which handled part of the CI/CD; and Hugging Face, which is both a pillar of the local AI community and provided private space for model storage for the handy-computer organization. As for whether AI assisted development, the author answered straightforwardly: Of course it did, but it's impossible for one person to create an engine of this scale from scratch in a few months using only ggml. However, none of the text presented here was written by AI; everything was written by his own mouth and hand.

Original link: workshop.cjpais.com/projects/transcribe-cpp