nyosegawa/speech.cpp

Speech synthesis, speech recognition and voice activity detection for voice conversation, in C++ on ggml

Models · Install · Quick start · Documentation · C API

English · 日本語

speech.cpp is one command, speech, and one shared library, libspeech, behind one C API, on the CPU, Metal and Vulkan through ggml. The command speaks text, writes the text of recordings, finds where someone speaks in them, transcribes the microphone as someone speaks (speech asr --live), serves OpenAI's audio API and its Realtime transcription over HTTP, and runs a worker process for any program that keeps models loaded between requests, ASIST among them.

A time to first audio of a quarter of a second or less. On an Apple M5, Qwen3-TTS 0.6B sends its first audio 0.04 s after the text and Irodori-TTS 0.25 s after it, and the rest follows while it plays.

One executable, nothing else to install. No Python and no CUDA: speech fetches each model by name, one GGUF file with its codec, and runs it on Metal, Vulkan or the CPU.

OpenAI's API on your own computer. speech serve answers OpenAI's audio API and its Realtime transcription, so an OpenAI client works by changing its base URL.

Checked against the official implementations. Every stage of every model is compared with the model's official implementation, from the same inputs.

_{The page of speech serve --open: pick a model of the catalog, fetch it and try it.}

Transcribe writes a file or a recording, a region at a time where someone speaks.

Live writes the microphone as you speak; the grey text is the utterance still being said.

A command takes a model by its name and fetches its converted file from Hugging Face, from the repository the name links to, the first time. docs/models.md lists the models with their sizes, voices and languages, and each family has a page: Qwen3-TTS, Irodori-TTS, FastConformer, Qwen3-ASR and Silero VAD.

macOS on Apple silicon and Linux on x86-64:

curl -fsSL https://raw.githubusercontent.com/nyosegawa/speech.cpp/main/install.sh | sh

Windows x64, in PowerShell:

irm https://raw.githubusercontent.com/nyosegawa/speech.cpp/main/install.ps1 | iex

The installer checks the release's archive against its SHA-256, puts speech under your home folder, and adds it to PATH. Run it again to update. On Linux it installs the Vulkan build when the Vulkan loader is installed, and the CPU build otherwise; the Linux builds need glibc 2.34 or later. docs/install.md has the options, how to uninstall, and what each platform needs. On an Intel Mac or another system, build from source.

1. Speak a sentence. The model is fetched the first time (1.21 GB):

   speech tts qwen3-tts-0.6b --voice ryan -o hello.wav "Hello. It will be sunny in Tokyo tomorrow."

2. Write its text back (0.84 GB the first time):

   speech asr qwen3-asr-0.6b hello.wav

   Hello, it will be sunny in Tokyo tomorrow.
   

3. Open a page on which to pick models, speak and transcribe:

   speech serve --open

speech models lists every model and, for each language, the model to start with.

speech voice irodori-tts-mf me.wav me.voice.gguf
speech tts irodori-tts-mf --add-voice me=me.voice.gguf --voice me -o - < story.txt | ffplay -nodisp -autoexit -
speech asr reazonspeech-v2 --timestamps meeting.wav
speech asr qwen3-asr-1.7b --language ja --prompt "Claude Code、渋谷" meeting.wav
speech asr reazonspeech-v2 --vad silero-vad --live
speech vad silero-vad --max-speech-duration-s 10 meeting.wav

Every request option is a flag, and speech info MODEL lists the ones a model takes. docs/cli.md lists every subcommand and option.

speech serve holds a synthesis model, a recognition model and a detection model, and answers POST /v1/audio/speech and POST /v1/audio/transcriptions as OpenAI's audio API does, for a web app, a script or curl:

speech serve qwen3-tts-0.6b qwen3-asr-0.6b
curl http://127.0.0.1:8080/v1/audio/speech -H 'Content-Type: application/json' \
    -d '{"input": "Hello.", "voice": "ryan"}' -o hello.wav
curl http://127.0.0.1:8080/v1/audio/transcriptions -F file=@hello.wav

It listens on 127.0.0.1 and has no authentication. docs/server.md has the endpoints, the formats, the errors and the page.

A program links libspeech and includes speech.h, from a release archive:

speech_model * model = NULL;
speech_request * request = NULL;
speech_model_load("Qwen3-TTS-12Hz-0.6B-CustomVoice-Q8_0.gguf", NULL, &model);
speech_request_new(model, &request);
speech_request_set_text(request, "Hello.");
speech_request_set_string(request, SPEECH_OPT_VOICE, "ryan");
speech_synthesize(request, on_audio, NULL);   /* on_audio receives the samples as they are made */

Each call returns a status whose category says what failed. docs/c-api.md has the whole API.

A program that would rather not link a library starts speech worker MODEL and speaks JSON Lines with it, as ASIST does: one request per line on stdin, its audio or text and one terminal message on stdout. docs/worker.md has the protocol.

Every stage of each port is compared with tensors dumped from the official implementation. The recognizers write the official text on every check input in F32 and F16.

Measured with speech-bench on 2026-10-08, on an Apple M5 (Metal) and an RTX 2080 (Vulkan), one request at a time. Speech: 20 Japanese sentences, the p50 time to first audio, the RTFx (the seconds of audio over the seconds they took, so larger is faster), and the CER of the speech as Qwen3-ASR 1.7B hears it.

Recognition: the 4,483 utterances of Common Voice 8.0's Japanese test set, trimmed to the voice, with the CER that accepts other spellings of the same words in parentheses, and parakeet-tdt-0.6b-v3 on 300 English utterances of FLEURS. FastConformer's times were measured on 2026-10-12 by speech-bench, one request at a time, on the first 100 Japanese utterances (99 kept by VAD) and the 300 English ones (299 kept); its accuracy and timing use separate sample counts.

Each model's page has more.

MIT, see LICENSE. The model weights are their authors':

• Qwen3-TTS and Qwen3-ASR are the Qwen team's, under the Apache License 2.0.
• Irodori-TTS v4.1-Small and v4.1-Small-MF are Aratako's, under the MIT License with the ethical restrictions of their model cards: no voice cloning without consent, no deepfakes or misinformation.
• Semantic-DACVAE-Japanese-32dim, the codec in the Irodori-TTS files, is Aratako's under the MIT License. It derives from Meta's facebook/dacvae-watermarked, under the Apache License 2.0; the SAM License that its card's text also names was corrected to Apache-2.0 in facebookresearch/dacvae on 2025-12-19.
• The parakeet models are NVIDIA's, under CC-BY-4.0, and reazonspeech-nemo-v2 is reazon-research's, under the Apache License 2.0.
• Silero VAD is the Silero Team's, under the MIT License.