Filter by type:
Filter by tags:
audio-cpp-supertonic
Supertonic 3 (audio.cpp, GGUF orig) - fast English text to speech with no
reference clip required, served by the audio-cpp backend. Covers both the
TTS and the TTSStream RPC, so /v1/audio/speech works streaming and
non-streaming.
The orig (F32) package is deliberate. Upstream's tested-status table in
docs/gguf.md records supertonic as Pass on orig, untested on 16-bit, and
"No (unsupported weight dtype)" on q8_0; the f16 package aborted the
process inside ggml_concat when tested locally. The backend refuses
anything but f32/i64 weights for this family, so a wrong build fails at
load with a clear message rather than crashing.
Repository: localaiLicense: openrail
audio-cpp-chatterbox
Chatterbox (audio.cpp, Q8_0) - multilingual zero-shot voice cloning, served
by the audio-cpp backend.
A reference clip is REQUIRED. This build advertises only the cloning and
voice-conversion tasks, not plain TTS, so every request must set the TTS
voice field to the path of a WAV file on the server; a voice that is not a
file on disk is treated as a named preset and the request is refused with a
message saying so.
The voice-conversion half is reached through /audio/transform rather than
/v1/audio/speech: post the speech to convert as `audio` and the target
speaker's clip as `reference`. That is why this entry declares
audio_transform alongside tts. With both this and a source-separation model
installed, /audio/transform has two candidates, so name the model
explicitly rather than relying on the default pick.
Q8_0 rather than 16-bit because upstream's tested-status table records the
same result for both (Pass, ASR match with drift) and this build is half
the size.
Repository: localaiLicense: mit
audio-cpp-citrinet-asr
Citrinet ASR (audio.cpp, Q8_0) - compact English speech to text, served by
the audio-cpp backend through /v1/audio/transcriptions. 40 MB, offline
transcription only. Q8_0 is the only package upstream publishes and tests
for this family, and it is recorded as a clean Pass.
Licensing: the checkpoint is pulled from NVIDIA NGC
(nvidia/nemo/stt_en_citrinet_256), whose use is governed by the NGC Terms
of Use rather than a named open licence.
Repository: localaiLicense: other
audio-cpp-nemotron-asr
Nemotron 3.5 ASR Streaming 0.6B (audio.cpp, F16) - streaming English speech
to text, served by the audio-cpp backend. It serves the backend's live
bidirectional transcription RPC as well as ordinary
/v1/audio/transcriptions calls.
F16 rather than Q8_0 because upstream's tested-status table records Q8_0 as
"Pass (minor filler drift)" while 16-bit is a clean Pass, and drift in a
transcript is drift in the output a caller reads.
Repository: localaiLicense: other
audio-cpp-sortformer-diarization
Sortformer Diarization 4-speaker v1 (audio.cpp, Q8_0) - speaker diarization
for up to four speakers, served by the audio-cpp backend through
/v1/audio/diarization. Returns per-segment start, end and speaker label; it
does not transcribe, so pair it with an ASR model for text.
Q8_0 because upstream records it as a clean Pass, the same as 16-bit, at
two thirds of the size.
Licensing: the base checkpoint nvidia/diar_sortformer_4spk-v1 is
CC BY-NC 4.0, so commercial use is not permitted.
Repository: localaiLicense: cc-by-nc-4.0
audio-cpp-htdemucs
HTDemucs (audio.cpp, F16) - four-stem music source separation (drums, bass,
other, vocals), served by the audio-cpp backend through /audio/transform.
One run produces all four stems; ask for a single one with the
params[stem] form field, and read the X-Audio-Stems response header for the
URLs of the rest.
F16 rather than Q8_0 because upstream's tested-status table records Q8_0 as
"Pass (drift)" while 16-bit is a clean Pass, and separation drift is
audible in an isolated stem in a way it is not in a mix.
Repository: localaiLicense: mit
audio-cpp-stable-audio-sfx
Stable Audio 3 Small SFX (audio.cpp, Q8_0) - text to sound effects, served
by the audio-cpp backend through /v1/sound-generation. Set
duration_seconds to control the length.
Text prompt only. The family also has an audio-conditioned editing route
upstream, which is not wired here: it aborts the process on the pinned
revision, and no field on the sound-generation request can reach it in any
case.
Q8_0 because upstream records both 16-bit and Q8_0 as "Pass (drift)", so
the smaller build costs nothing extra.
Repository: localaiLicense: other
audio-cpp-forced-aligner
Qwen3 Forced Aligner 0.6B (audio.cpp, Q8_0) - aligns a transcript you
already have against the audio, served by the audio-cpp backend.
It is driven through /v1/audio/transcriptions, not a dedicated endpoint:
put the known transcript in the prompt field and the model returns it with
word timings instead of recognising anything. Both a transcript and a
language are required, so this entry defaults language to en; override it
with the language form field for other languages.
Q8_0 because upstream records both 16-bit and Q8_0 as a clean Pass.
Repository: localaiLicense: apache-2.0
audio-cpp-silero-vad
Silero VAD (audio.cpp) - voice activity detection over /v1/vad. The model
ships inside the audio-cpp backend package, so installing this entry
downloads nothing.
Repository: localaiLicense: mit
audio-cpp-marblenet-vad
MarbleNet VAD (audio.cpp) - NeMo voice activity detection over /v1/vad, an
alternative to Silero with different segment boundaries. The model ships
inside the audio-cpp backend package, so installing this entry downloads
nothing.
Repository: localaiLicense: other
audio-cpp-irodori-voicedesign
Irodori TTS 600M v3 VoiceDesign (audio.cpp, Q8_0) - describe the voice you
want in words instead of supplying a reference clip, served by the
audio-cpp backend.
Drive it through /v1/audio/speech with the `instructions` field: the
backend routes a request carrying instructions to audio.cpp's vdes task, so
"a calm elderly woman speaking slowly with a warm, gentle tone" shapes the
delivery of whatever you put in `input`. Without instructions this build
still answers as ordinary TTS, and with a reference WAV in `voice` it
clones instead; the request decides which of the three it gets.
The model is Japanese-first and handles English, so an English ASR round
trip recovers the words with accent drift rather than cleanly.
Repository: localaiLicense: mit
audio-cpp-seedvc-singing
Seed-VC (audio.cpp, Q8_0) - singing voice conversion, served by the
audio-cpp backend through /audio/transform. Post the performance to convert
as `audio` and the target singer's clip as `reference`.
This entry pins task:svc, and the pin is what makes it a singing model
rather than a speech one. Seed-VC advertises both svc and ordinary voice
conversion, and nothing in a request says "this input is singing", so
auto-routing always picks voice conversion; the pin is the only way to
reach the singing path. Drop the option from the model config to get plain
voice conversion from the same weights.
Q8_0 because the 16-bit and Q8_0 builds carry the same tested status
upstream (drift) and this one is half the size. Note the GPL-3.0 licence,
which is stricter than most of the gallery.
Repository: localaiLicense: gpl-3.0
audio-cpp-vevo2-speech-to-speech
Vevo2 (audio.cpp, Q8_0) - speech to speech, served by the audio-cpp backend
through /audio/transform. Post the source speech as `audio`, the target
speaker's clip as `reference`, and the words being spoken as
`params[text]`.
The text is not optional and not a hint. Vevo2's speech-to-speech route is
a text and prosody route: it reads the line it is resynthesising and
refuses the run without one. /audio/transform has no text field on the
wire, so the params map carries it and the backend lifts it into the
engine's text input; `params[target_text]` is the same thing under the name
Vevo2's own option table uses. Add `params[language]` when the line is not
English.
This entry pins task:s2s. Vevo2 also advertises plain voice conversion,
which needs no text and which auto-routing picks first, so without the pin
the text would be carried and then ignored. Drop the option to get voice
conversion from the same weights.
Q8_0 because it is the only build that fits alongside the rest of the
gallery; upstream records the original and 16-bit builds as the cleaner
ones, so expect drift on some routes.
Repository: localaiLicense: other