Repository navigation
feat: expose the encoder architecture, name and family string through the C API - #3
Merged
Merged
Conversation
A store of enrolled voices has to know which encoder made its embeddings. Equal embedding sizes do not mean the same embedding space: ECAPA-TDNN and CAM++ both give 192 values. A host that binds libvoicedetect could not tell the encoders apart, because the context did not report what the loader had read from the GGUF header. Add voicedetect_capi_encoder_arch (voicedetect.arch), voicedetect_capi_encoder_name (general.name) and voicedetect_capi_encoder_family, which returns "voicedetect:<arch>:<name>:<embedding_dim>". The pointers belong to the context and stay valid until voicedetect_capi_free; a NULL context gives NULL. The values are read once at load time, so they work for the path, memory and prefixed memory loaders. A prefixed load reads the prefixed keys, so a bundle component gives the same string as the standalone file it came from. The config gains name and family for C++ callers. The string is defined the same way as speaker_encoder_family in parakeet.cpp: a missing key gives an empty field, the embedding size is empty when it is 0, and the string is empty when general.architecture is not "voicedetect". The two must stay identical, and the header says so. A wrongly typed general.name reads as empty and does not fail a load that succeeded before. This is additive: the ABI version stays 1. test_encoder_family needs no model files: it checks the exact format, equal strings from the three loaders, missing and odd keys, bundle header keys that must not leak into a component, NULL and concurrent reads. test_encoder_family_models (set VOICEDETECT_TEST_GGUF_LIST) checks the ECAPA-TDNN, CAM++, WeSpeaker ResNet34 and ERes2Net files through the three loaders against the formula applied to the file header. Assisted-by: Claude:claude-sonnet-5-5 [Claude Code]
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What and why
parakeet.cpp records an encoder fingerprint in its speaker registry, so a registry made with one speaker encoder is refused when another encoder is used. Equal embedding sizes do not mean the same embedding space: ECAPA-TDNN and CAM++ both give 192 values. Its family string is built from the encoder GGUF header.
A host that binds
libvoicedetectthrough the C API (for example the LocalAI voice-detect backend) cannot build that string today, because the context does not report the architecture or the model name. Voices that such a host enrolls from audio stay unfingerprinted. This PR exposes what the loader already reads from the header.It is additive. Existing behavior and the ABI version (1) do not change.
API
C++:
vd::Model::config().arch,.nameand.family(VoiceDetectConfiggainsnameandfamily).voicedetect_capi_free. They are NULL for a NULL context. They can be read from several threads at once.<prefix>general.name,<prefix>voicedetect.arch, ...), so a bundle component gives the same string as the standalone file it was made from. The keys of the bundle header do not leak into a component.Family format
For example
voicedetect:ecapa_tdnn:speechbrain/spkrec-ecapa-voxceleb:192.general.architectureis notvoicedetect.general.nameof the wrong type reads as empty. It does not fail a load that worked before.This is the same formula as
speaker_encoder_familyin parakeet.cpp. The two must stay identical: the header and the README say so. Compared with that function (compiled from parakeet.cpp master) on the ECAPA-TDNN, CAM++, WeSpeaker ResNet34 and ERes2Net f32 files, the strings are identical:Tests
test_encoder_family(no model files, runs in CI): exact format, equal strings from the path, memory and prefixed loaders, missinggeneral.name, other or absentgeneral.architecture, a wrongly typed name, colons and UTF-8 in the name, two components in one bundle that each report their own identity, no leak from the bundle header, NULL context, stable pointers, 8 threads reading at once.test_encoder_family_models(needsVOICEDETECT_TEST_GGUF_LIST, a:-separated list of GGUF files; skips otherwise): for each file the three loaders give the same strings, equal to the formula applied to the header read straight from the file, and the last field equalsvoicedetect_capi_embedding_dim. Run with the four encoders above.Verification
Linux x86_64, GCC, CPU:
ctest -LE model: 5 passed, 1 skipped (test_capi_dim, needs a model), 0 failed.test_encoder_family_modelspasses.test_capi_dimandtest_load_memory_embedpass too.test_encoder_family,test_encoder_family_models(leak detection on),test_load_memoryandtest_load_memory_embedare clean.check_baseline(Python reference check) does not run in my environment because the reference Python packages are missing. It does not touch this code.Not run: macOS and Windows. The change uses only standard C++ and the already parsed header.
Limits
general.name(a file not made by the converter) gives an empty name field, so two such encoders with the same arch and size have the same family. The converter always writes it.🤖 Generated with Claude Code