diff --git a/AGENTS.md b/AGENTS.md index b405dec..eaa8f6c 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -154,7 +154,7 @@ docs/ parity.md , full model coverage matrix + per-stage tensor parity diarization.md , speaker diarization + speaker-attributed ASR: parity, C-API, speed sound.md , sound-event detection (CED) and the combined scene stream - speaker.md , speaker identification: enroll, scene naming, C-API v9, measured numbers + speaker.md , speaker identification: enroll, scene naming, C-API v9 and v10, measured numbers .github/workflows/ ci.yml , build job (per-push) + closed-loop job (pull_request + dispatch) ``` @@ -344,6 +344,12 @@ parakeet_capi_scene_stream_begin_speaker parakeet_capi_transcribe_and_diarize_named_json ``` +``` +# v10 (additive; not used by LocalAI yet) +parakeet_capi_speaker_registry_add_embedding # add an embedding computed by the host +parakeet_capi_diarize_named_pcm_json # diarization + "names", no ASR model +``` + Combined scene stream (ABI v8, additive; not used by LocalAI yet). One stream that carries any mix of an ASR context, a diarization context, and a tagger context, and emits speaker-attributed words/utterances plus sound-event diff --git a/README.md b/README.md index 46c881c..9020700 100644 --- a/README.md +++ b/README.md @@ -345,7 +345,7 @@ default) the scene stream can say who is talking instead of `Speaker 0`. Enroll each person from a short clip with `parakeet-cli enroll`, then pass `--speakers --registry ` to `scene`. Only one two-voice fixture has been measured so far. See [`docs/speaker.md`](docs/speaker.md) for -the models, the commands, the C-API (ABI v9) and what is still untested. +the models, the commands, the C-API (ABI v9 and v10) and what is still untested. --- diff --git a/docs/speaker.md b/docs/speaker.md index 566ae0e..b821509 100644 --- a/docs/speaker.md +++ b/docs/speaker.md @@ -169,7 +169,7 @@ command line with `--speaker-threshold` (only `accept_threshold`). encoder: see the starting threshold column in "Which GGUFs work" and the numbers below. -## C-API (ABI v9) +## C-API (ABI v9 and v10) Additive: no earlier signature changed, LocalAI does not use these yet. A speaker GGUF loads through `parakeet_capi_load` into a context of kind @@ -197,6 +197,71 @@ speaker part these fields are there from the first document on: `"names"` is change during the stream. Without a speaker part none of them appear. The offline named document is the SAS document plus those fields. +### v10: embeddings from the host, diarize-only naming + +Two more functions for a host that keeps speaker embeddings itself (LocalAI +stores them in its own voice registry) instead of keeping enrollment audio. + +``` +int parakeet_capi_speaker_registry_add_embedding(parakeet_speaker_registry* reg, const char* name, + const float* embedding, int dim); + +char* parakeet_capi_diarize_named_pcm_json(parakeet_ctx* diar, parakeet_ctx* speaker, + parakeet_speaker_registry* reg, const float* samples, + int n_samples, int sample_rate, float accept_threshold, + float margin); +``` + +`add_embedding` adds an embedding that was computed elsewhere, so it needs no +speaker model. The first successful call fixes the registry's size; after that +`dim` must match it. The values must be finite and not all zero, and the name +must not be empty. Adding the same name again averages the vectors, like +enrolling more clips. It returns 0 on success and nonzero on error, with the +message on the registry (`parakeet_capi_speaker_registry_last_error`), for +example `speaker embedding has 192 values, registry expects 256`. A NULL +registry returns nonzero with no message. + +`diarize_named_pcm_json` diarizes the clip and names the speakers, with no ASR +model. The result is the `parakeet_capi_diarize_pcm` document plus a `"names"` +key, for example `{"0":{"name":"ada","score":0.93}}`. `names` has one entry per +diarization slot that has a segment. `"name":""` (score 0) means no voice +matched, or the slot had too little clean speech (under the identifier's 2 s +minimum, or the audio overlapped another speaker). `accept_threshold` is a +cosine in [-1, 1] and `margin` is how far the best match must beat the +runner-up; it must be 0 or more (a negative value is rejected with "invalid +speaker options"). A value of 0 for either keeps the default (0.5 and 0.05), so +exactly 0 cannot be requested. Any sample rate works; the audio is resampled +to 16 kHz. It returns NULL on error. The message is on `diar` for a wrong +context kind or bad samples, and on `speaker` for a wrong context kind, a NULL +registry, bad options, or a registry whose embedding size differs from the +encoder's (`registry holds N-value embeddings, this model produces M`). The +registry must come from the same kind of encoder as `speaker`. An empty buffer +(`n_samples == 0`) is an error here, unlike `parakeet_capi_diarize_pcm`, which +returns an empty document. The call keeps a copy of the whole recording, so +memory grows with clip length (about 230 MB per hour of 16 kHz audio), the same +order as the speaker-attributed ASR call. Free the result with +`parakeet_capi_free_string`. + +```c +parakeet_speaker_registry* reg = parakeet_capi_speaker_registry_new(); +if (parakeet_capi_speaker_registry_add_embedding(reg, "ada", ada_emb, dim) || + parakeet_capi_speaker_registry_add_embedding(reg, "bob", bob_emb, dim)) { + fprintf(stderr, "%s\n", parakeet_capi_speaker_registry_last_error(reg)); + parakeet_capi_speaker_registry_free(reg); + return 1; +} +char* json = parakeet_capi_diarize_named_pcm_json(diar, speaker, reg, pcm, n_pcm, + 16000, 0.0f, 0.0f); +if (json) { + puts(json); + parakeet_capi_free_string(json); +} else { + fprintf(stderr, "%s / %s\n", parakeet_capi_last_error(speaker), + parakeet_capi_last_error(diar)); +} +parakeet_capi_speaker_registry_free(reg); +``` + ## Devices and threads voice-detect keeps its own backend and device selection, like ced.cpp does with diff --git a/include/parakeet_capi.h b/include/parakeet_capi.h index 04e4502..3516709 100644 --- a/include/parakeet_capi.h +++ b/include/parakeet_capi.h @@ -63,6 +63,10 @@ typedef struct parakeet_ctx parakeet_ctx; // parakeet_speaker_registry holds enrolled voices; the scene stream and // speaker-attributed ASR can name diarized speakers. Additive: no // existing signature changed. +// v10: raw-embedding enroll (parakeet_capi_speaker_registry_add_embedding) +// and diarize-only naming (parakeet_capi_diarize_named_pcm_json), for +// callers that keep speaker embeddings themselves. Additive: no +// existing signature changed. int parakeet_capi_abi_version(void); // Load a GGUF model. Returns an owning context, or NULL on failure. @@ -612,6 +616,16 @@ const char* parakeet_capi_speaker_registry_last_error(const parakeet_speaker_reg int parakeet_capi_speaker_enroll(parakeet_speaker_registry* reg, parakeet_ctx* speaker, const char* name, const float* pcm, int n, int sample_rate); +// Add one already-computed speaker embedding to `reg` under `name`, without a +// speaker model. `dim` must equal the registry's embedding size once it has +// one (the first successful call fixes it); the values must be finite and not +// all zero. Calling it again with the same name averages the vectors, like +// enrolling more clips. Returns 0 on success; nonzero on error, with the +// message on the registry (parakeet_capi_speaker_registry_last_error). A NULL +// `reg` returns nonzero with no message. ABI v10. +int parakeet_capi_speaker_registry_add_embedding(parakeet_speaker_registry* reg, const char* name, + const float* embedding, int dim); + // Binary file. 0 on success; nonzero on error (message on the registry). int parakeet_capi_speaker_registry_save(const parakeet_speaker_registry* reg, const char* path); // NULL when the file is missing or is not a valid registry. Free with _free. @@ -644,6 +658,27 @@ char* parakeet_capi_transcribe_and_diarize_named_json(parakeet_ctx* asr, parakee const float* samples, int n_samples, int sample_rate); +// Diarization with speaker names and no ASR model: the parakeet_capi_diarize_pcm +// document plus a "names" key, {"0":{"name":"ada","score":0.93},...}, one entry +// per diarization slot that has a segment ("name":"" with score 0 means no +// voice matched, or too little clean speech: under the identifier's 2 s +// minimum, or overlapped by another speaker). `accept_threshold` is a cosine in +// [-1, 1] and `margin` how far the best match must beat the runner-up; it must +// be 0 or more. 0 for either keeps its default (0.5 and 0.05). Any sample rate +// (resampled to 16 kHz like the other PCM entry points). Returns NULL on error, +// with the message on `diar` (a wrong kind, bad samples) or on `speaker` (a +// wrong kind, a NULL or wrong-sized registry, invalid options); both messages +// are cleared on entry, so only the ctx the failure belongs to has one. NULL +// `diar` or `speaker` returns NULL with no message. An empty buffer +// (`n_samples == 0`) is an error here, unlike parakeet_capi_diarize_pcm which +// returns an empty document. An empty registry gives every slot an empty name. +// `reg` is only read; it may be shared by concurrent calls as long as nothing +// adds to it meanwhile. Free with parakeet_capi_free_string. ABI v10. +char* parakeet_capi_diarize_named_pcm_json(parakeet_ctx* diar, parakeet_ctx* speaker, + parakeet_speaker_registry* reg, const float* samples, + int n_samples, int sample_rate, float accept_threshold, + float margin); + #ifdef __cplusplus } // extern "C" #endif diff --git a/src/parakeet_capi.cpp b/src/parakeet_capi.cpp index bb91682..8f54286 100644 --- a/src/parakeet_capi.cpp +++ b/src/parakeet_capi.cpp @@ -59,7 +59,11 @@ // v9: speaker identification (voice-detect.cpp): a speaker ctx kind, a speaker // registry, scene_stream_begin_speaker, transcribe_and_diarize_named_json; // additive. -#define PARAKEET_CAPI_ABI_VERSION 9 +// v10: raw-embedding enroll (parakeet_capi_speaker_registry_add_embedding) for +// callers that keep speaker embeddings themselves, and diarization with +// speaker names and no ASR model (parakeet_capi_diarize_named_pcm_json); +// additive. +#define PARAKEET_CAPI_ABI_VERSION 10 // The opaque context: a loaded model plus a buffer for the last error message. // Exactly one of `model` / `diar` / `tagger` / `speaker` is non-null: ASR models @@ -1013,7 +1017,7 @@ bool require_speaker(parakeet_ctx* ctx) { return true; } -char* diar_result_to_json(const pk::DiarizationResult& r) { +std::string diar_result_json_string(const pk::DiarizationResult& r) { std::string json = "{\"speakers\":"; pk::append_json_int(json, r.n_speakers); json += ",\"segments\":["; @@ -1028,7 +1032,25 @@ char* diar_result_to_json(const pk::DiarizationResult& r) { json += '}'; } json += "]}"; - return dup_to_c(json); + return json; +} + +char* diar_result_to_json(const pk::DiarizationResult& r) { return dup_to_c(diar_result_json_string(r)); } + +// {"0":{"name":"ada","score":0.9312},"1":{...}} for the slots the identifier has seen. +std::string names_to_json(const std::map& names) { + std::string s = "{"; + bool first = true; + for (const auto& kv : names) { + if (!first) s += ','; + first = false; + s += "\"" + std::to_string(kv.first) + "\":{\"name\":"; + pk::append_json_string(s, kv.second.name); + s += ",\"score\":"; + pk::append_json_float(s, "%.4f", kv.second.score); + s += '}'; + } + return s + "}"; } template @@ -1794,6 +1816,23 @@ extern "C" int parakeet_capi_speaker_enroll(parakeet_speaker_registry* reg, para return 1; } +extern "C" int parakeet_capi_speaker_registry_add_embedding(parakeet_speaker_registry* reg, const char* name, + const float* embedding, int dim) { + if (!reg) return 1; + try { + if (!name || !*name) { reg->last_error = "speaker name is empty"; return 1; } + if (!embedding || dim <= 0) { reg->last_error = "no embedding"; return 1; } + reg->reg.enroll(name, std::vector(embedding, embedding + dim)); + reg->last_error.clear(); + return 0; + } catch (const std::exception& e) { + reg->last_error = e.what(); + } catch (...) { + reg->last_error = "unknown error"; + } + return 1; +} + extern "C" int parakeet_capi_speaker_registry_save(const parakeet_speaker_registry* reg, const char* path) { if (!reg) return 1; auto* mreg = const_cast(reg); // only last_error is written @@ -1905,18 +1944,8 @@ extern "C" char* parakeet_capi_transcribe_and_diarize_named_json( const std::vector utts = pk::group_speaker_words(words); std::string s = "{\"speakers\":"; pk::append_json_int(s, n_speakers); - s += ",\"names\":{"; - bool first = true; - for (const auto& kv : names) { - if (!first) s += ','; - first = false; - s += "\"" + std::to_string(kv.first) + "\":{\"name\":"; - pk::append_json_string(s, kv.second.name); - s += ",\"score\":"; - pk::append_json_float(s, "%.4f", kv.second.score); - s += '}'; - } - s += "},\"utterances\":["; + s += ",\"names\":" + names_to_json(names); + s += ",\"utterances\":["; for (size_t i = 0; i < utts.size(); ++i) { if (i) s += ','; append_speaker_item(s, utts[i], "%.2f", &names); @@ -1937,3 +1966,68 @@ extern "C" char* parakeet_capi_transcribe_and_diarize_named_json( return nullptr; } } + +extern "C" char* parakeet_capi_diarize_named_pcm_json(parakeet_ctx* diar_ctx, parakeet_ctx* speaker, + parakeet_speaker_registry* registry, + const float* samples, int n_samples, int sample_rate, + float accept_threshold, float margin) { + try { + // Each failure below sets the one message that applies; clear both so the other ctx + // never shows an older, unrelated error. + if (diar_ctx) diar_ctx->last_error.clear(); + if (speaker) speaker->last_error.clear(); + if (!speaker) return nullptr; + if (!require_diar(diar_ctx)) return nullptr; + if (!require_speaker(speaker)) return nullptr; + if (!registry) { speaker->last_error = "speaker identification needs a registry"; return nullptr; } + if (!samples || n_samples <= 0) { + diar_ctx->last_error = "invalid samples buffer"; + return nullptr; + } + if (sample_rate <= 0) { + diar_ctx->last_error = "invalid sample rate"; + return nullptr; + } + const int rd = registry->reg.dim(); + if (rd != 0 && rd != speaker->speaker->dim()) { + speaker->last_error = "registry holds " + std::to_string(rd) + + "-value embeddings, this model produces " + + std::to_string(speaker->speaker->dim()); + return nullptr; + } + pk::SpeakerIdOpts o; + if (accept_threshold != 0.0f) o.accept_threshold = accept_threshold; + if (margin != 0.0f) o.margin = margin; + const std::string oerr = pk::validate_speaker_opts(o); + if (!oerr.empty()) { speaker->last_error = "invalid speaker options: " + oerr; return nullptr; } + + // Resample once; diarize_pcm and the identifier both take the 16 kHz signal. + std::vector pcm16k(samples, samples + n_samples); + if (sample_rate != 16000) pcm16k = pk::resample_linear(pcm16k, sample_rate, 16000); + pk::DiarizationResult dr; + try { + dr = diar_ctx->diar->diarize_pcm(pcm16k, 16000); + } catch (const std::exception& e) { + diar_ctx->last_error = e.what(); + return nullptr; + } + std::map names; + try { + names = pk::identify_offline(pcm16k, dr.segments, speaker->speaker->embedder(), registry->reg, o); + } catch (const std::exception& e) { + speaker->last_error = e.what(); + return nullptr; + } + std::string json = diar_result_json_string(dr); + json.pop_back(); // drop the closing '}' and add the names key + json += ",\"names\":" + names_to_json(names) + "}"; + diar_ctx->last_error.clear(); + speaker->last_error.clear(); + return dup_to_c(json); + } catch (const std::exception& e) { + if (speaker) speaker->last_error = e.what(); + } catch (...) { + if (speaker) speaker->last_error = "unknown error"; + } + return nullptr; +} diff --git a/tests/CMakeLists.txt b/tests/CMakeLists.txt index 245eb84..aae139e 100644 --- a/tests/CMakeLists.txt +++ b/tests/CMakeLists.txt @@ -86,6 +86,10 @@ set_tests_properties(test_speaker_identify PROPERTIES LABELS "model") pk_add_test(test_capi_speaker) target_compile_definitions(test_capi_speaker PRIVATE PK_SOURCE_DIR="${CMAKE_SOURCE_DIR}") set_tests_properties(test_capi_speaker PROPERTIES LABELS "model") +pk_add_test(test_capi_diarize_named) +target_compile_definitions(test_capi_diarize_named PRIVATE PK_SOURCE_DIR="${CMAKE_SOURCE_DIR}") +set_tests_properties(test_capi_diarize_named PROPERTIES LABELS "model") +pk_add_test(test_capi_speaker_registry) pk_add_test(test_combined_offline) pk_add_test(test_streaming_diarization) pk_add_test(test_sound_stream) diff --git a/tests/test_capi_diarize_named.cpp b/tests/test_capi_diarize_named.cpp new file mode 100644 index 0000000..e00edbb --- /dev/null +++ b/tests/test_capi_diarize_named.cpp @@ -0,0 +1,224 @@ +// Diarize-only naming through the C-API, with a registry filled by +// add_embedding from embeddings the folded encoder computes. +// PARAKEET_TEST_DIAR_GGUF, PARAKEET_TEST_VD_GGUF (required, else skip 77) +#include "audio_io.hpp" +#include "parakeet_capi.h" +#include "speaker_encoder.hpp" + +#include +#include +#include +#include +#include +#include + +static int failures = 0; +#define CHECK(cond) \ + do { \ + if (!(cond)) { \ + std::fprintf(stderr, "FAIL: %s (line %d)\n", #cond, __LINE__); \ + ++failures; \ + } \ + } while (0) + +static std::vector slice(const std::vector& x, double a, double b) { + return std::vector(x.begin() + (long)(a * 16000), x.begin() + (long)(b * 16000)); +} + +// Splits the "names" object of a document into its names and scores, in order. +static void parse_names(const std::string& doc, std::vector& names, std::vector& scores) { + size_t p = doc.find("\"names\":"); + if (p == std::string::npos) return; + const std::string key = "\"name\":\""; + while ((p = doc.find(key, p)) != std::string::npos) { + p += key.size(); + const size_t e = doc.find('"', p); + names.push_back(doc.substr(p, e - p)); + const size_t sp = doc.find("\"score\":", e); + scores.push_back(std::strtof(doc.c_str() + sp + 8, nullptr)); + p = sp; + } +} + +int main() { + const char* diar_path = std::getenv("PARAKEET_TEST_DIAR_GGUF"); + const char* vd_path = std::getenv("PARAKEET_TEST_VD_GGUF"); + if (!diar_path || !vd_path) return 77; + if (!pk::SpeakerEncoder::available()) return 77; + + pk::Audio wav; + if (!pk::load_audio_16k_mono(std::string(PK_SOURCE_DIR) + "/tests/fixtures/two_speakers.wav", wav)) return 1; + + // Embeddings of the two voices, computed outside the C-API on purpose: + // this is what LocalAI hands over (vectors, not audio). + auto enc = pk::SpeakerEncoder::load(vd_path); + if (!enc) { std::fprintf(stderr, "FAIL: load speaker model\n"); return 1; } + std::vector ea, eb; + const auto ca = slice(wav.samples, 0.6, 4.6), cb = slice(wav.samples, 6.9, 10.9); + CHECK(enc->embed(ca.data(), (int)ca.size(), ea)); + CHECK(enc->embed(cb.data(), (int)cb.size(), eb)); + + parakeet_ctx* diar = parakeet_capi_load(diar_path); + parakeet_ctx* spk = parakeet_capi_load(vd_path); + CHECK(diar && spk); + parakeet_speaker_registry* reg = parakeet_capi_speaker_registry_new(); + CHECK(parakeet_capi_speaker_registry_add_embedding(reg, "second_voice", eb.data(), (int)eb.size()) == 0); + CHECK(parakeet_capi_speaker_registry_add_embedding(reg, "first_voice", ea.data(), (int)ea.size()) == 0); + + // Defaults: slot 0 is voice A (first to speak), slot 1 is voice B. The registry order is + // reversed on purpose, so arrival-order naming would fail. + std::vector n1; + std::vector s1; + char* j = parakeet_capi_diarize_named_pcm_json(diar, spk, reg, wav.samples.data(), (int)wav.samples.size(), + 16000, 0.0f, 0.0f); + CHECK(j != nullptr); + if (j) { + const std::string s = j; + CHECK(s.find("\"segments\":[{\"speaker\":0") != std::string::npos); + CHECK(s.find("\"names\":{\"0\":{\"name\":\"first_voice\"") != std::string::npos); + CHECK(s.find("\"1\":{\"name\":\"second_voice\"") != std::string::npos); + parse_names(s, n1, s1); + parakeet_capi_free_string(j); + } + + // A registry built with enroll from the same two clips gives the same names and scores. + { + parakeet_speaker_registry* reg2 = parakeet_capi_speaker_registry_new(); + CHECK(parakeet_capi_speaker_enroll(reg2, spk, "first_voice", ca.data(), (int)ca.size(), 16000) == 0); + CHECK(parakeet_capi_speaker_enroll(reg2, spk, "second_voice", cb.data(), (int)cb.size(), 16000) == 0); + std::vector n2; + std::vector s2; + j = parakeet_capi_diarize_named_pcm_json(diar, spk, reg2, wav.samples.data(), (int)wav.samples.size(), + 16000, 0.0f, 0.0f); + CHECK(j != nullptr); + if (j) { + parse_names(j, n2, s2); + parakeet_capi_free_string(j); + } + CHECK(!n1.empty() && n1 == n2 && s1.size() == s2.size()); + for (size_t i = 0; i < s1.size() && i < s2.size(); ++i) CHECK(std::fabs(s1[i] - s2[i]) < 1e-3f); + parakeet_capi_speaker_registry_free(reg2); + } + + // The threshold reaches the identifier: above every genuine score, nothing is named. + j = parakeet_capi_diarize_named_pcm_json(diar, spk, reg, wav.samples.data(), (int)wav.samples.size(), + 16000, 0.999f, 0.0f); + CHECK(j != nullptr); + if (j) { + const std::string s = j; + CHECK(s.find("\"name\":\"\"") != std::string::npos); + CHECK(s.find("first_voice") == std::string::npos && s.find("second_voice") == std::string::npos); + parakeet_capi_free_string(j); + } + + // The margin reaches the identifier. Two registries whose only difference is a runner-up: + // "a1" and "a2" hold the very same vector (best minus runner-up is exactly 0, so the default + // 0.05 margin calls voice A unknown), while "a1" alone has no runner-up and is named. + // The construction is independent of the encoder. (A large margin such as 0.9 on the main + // registry is not robust: WeSpeaker scores impostors near 0, so best minus second best can + // exceed 0.9 and the slot is still named.) + { + parakeet_speaker_registry* tie = parakeet_capi_speaker_registry_new(); + parakeet_speaker_registry* solo = parakeet_capi_speaker_registry_new(); + CHECK(parakeet_capi_speaker_registry_add_embedding(tie, "a1", ea.data(), (int)ea.size()) == 0); + CHECK(parakeet_capi_speaker_registry_add_embedding(tie, "a2", ea.data(), (int)ea.size()) == 0); + CHECK(parakeet_capi_speaker_registry_add_embedding(solo, "a1", ea.data(), (int)ea.size()) == 0); + j = parakeet_capi_diarize_named_pcm_json(diar, spk, tie, wav.samples.data(), (int)wav.samples.size(), + 16000, 0.0f, 0.0f); + CHECK(j != nullptr); + if (j) { + const std::string s = j; + CHECK(s.find("\"names\":{\"0\":{\"name\":\"\"") != std::string::npos); // a tie is unknown + CHECK(s.find("a1") == std::string::npos && s.find("a2") == std::string::npos); + parakeet_capi_free_string(j); + } + j = parakeet_capi_diarize_named_pcm_json(diar, spk, solo, wav.samples.data(), (int)wav.samples.size(), + 16000, 0.0f, 0.0f); + CHECK(j != nullptr); + if (j) { + CHECK(std::string(j).find("\"names\":{\"0\":{\"name\":\"a1\"") != std::string::npos); + parakeet_capi_free_string(j); + } + parakeet_capi_speaker_registry_free(tie); + parakeet_capi_speaker_registry_free(solo); + } + // A negative margin is refused by the identifier's validation, on the speaker ctx. + CHECK(parakeet_capi_diarize_named_pcm_json(diar, spk, reg, wav.samples.data(), (int)wav.samples.size(), + 16000, 0.0f, -0.1f) == nullptr); + CHECK(std::strstr(parakeet_capi_last_error(spk), "invalid speaker options") != nullptr); + + // Invalid options and a non-16k input rate. + CHECK(parakeet_capi_diarize_named_pcm_json(diar, spk, reg, wav.samples.data(), (int)wav.samples.size(), + 16000, 2.0f, 0.0f) == nullptr); + CHECK(std::strstr(parakeet_capi_last_error(spk), "invalid speaker options") != nullptr); + { + std::vector twice; // the same audio at 32 kHz (each sample repeated): must still name both + for (float v : wav.samples) { twice.push_back(v); twice.push_back(v); } + j = parakeet_capi_diarize_named_pcm_json(diar, spk, reg, twice.data(), (int)twice.size(), 32000, 0.0f, 0.0f); + CHECK(j != nullptr); + if (j) { + const std::string s = j; + CHECK(s.find("first_voice") != std::string::npos && s.find("second_voice") != std::string::npos); + parakeet_capi_free_string(j); + } + } + + // Error paths, and which ctx carries each message. + CHECK(parakeet_capi_diarize_named_pcm_json(diar, nullptr, reg, wav.samples.data(), 16000, 16000, 0, 0) == nullptr); + CHECK(parakeet_capi_diarize_named_pcm_json(diar, spk, nullptr, wav.samples.data(), 16000, 16000, 0, 0) == nullptr); + CHECK(std::strstr(parakeet_capi_last_error(spk), "needs a registry") != nullptr); + // A speaker ctx passed as `diar`: the message is on that ctx. + CHECK(parakeet_capi_diarize_named_pcm_json(spk, spk, reg, wav.samples.data(), 16000, 16000, 0, 0) == nullptr); + CHECK(std::strstr(parakeet_capi_last_error(spk), "holds a speaker model; diarize_* needs a diarization model") != nullptr); + // A diarization ctx passed as `speaker`: the message is on that ctx. + CHECK(parakeet_capi_diarize_named_pcm_json(diar, diar, reg, wav.samples.data(), 16000, 16000, 0, 0) == nullptr); + CHECK(std::strstr(parakeet_capi_last_error(diar), "holds a diarization model; a speaker model is needed here") != nullptr); + // Bad samples: "invalid samples buffer" on diar. + CHECK(parakeet_capi_diarize_named_pcm_json(diar, spk, reg, nullptr, 16000, 16000, 0, 0) == nullptr); + CHECK(std::strstr(parakeet_capi_last_error(diar), "invalid samples buffer") != nullptr); + CHECK(parakeet_capi_diarize_named_pcm_json(diar, spk, reg, wav.samples.data(), 0, 16000, 0, 0) == nullptr); + CHECK(std::strstr(parakeet_capi_last_error(diar), "invalid samples buffer") != nullptr); + CHECK(parakeet_capi_diarize_named_pcm_json(diar, spk, reg, wav.samples.data(), 16000, 0, 0, 0) == nullptr); + CHECK(std::strstr(parakeet_capi_last_error(diar), "invalid sample rate") != nullptr); + // A registry made for a different encoder size is refused with both sizes in the message, on + // the speaker ctx, and before diarization runs (the whole fixture is passed, the diar ctx stays clean, although the call before it left a message there). + { + parakeet_speaker_registry* wrong = parakeet_capi_speaker_registry_new(); + const float small[3] = {1, 0, 0}; + CHECK(parakeet_capi_speaker_registry_add_embedding(wrong, "x", small, 3) == 0); + CHECK(parakeet_capi_diarize_named_pcm_json(diar, spk, wrong, wav.samples.data(), (int)wav.samples.size(), + 16000, 0, 0) == nullptr); + const std::string msg = parakeet_capi_last_error(spk); + const std::string want = "this model produces " + std::to_string(parakeet_capi_speaker_dim(spk)); + CHECK(msg.find("3-value") != std::string::npos); + CHECK(msg.find(want) != std::string::npos); + CHECK(std::strlen(parakeet_capi_last_error(diar)) == 0); + parakeet_capi_speaker_registry_free(wrong); + } + + // Only the ctx that the failure belongs to carries a message: a failure on one ctx must not + // leave an older message on the other. + { + parakeet_speaker_registry* wrong = parakeet_capi_speaker_registry_new(); + const float small[3] = {1, 0, 0}; + CHECK(parakeet_capi_speaker_registry_add_embedding(wrong, "x", small, 3) == 0); + // speaker fails, then diar fails: the speaker message is gone. + CHECK(parakeet_capi_diarize_named_pcm_json(diar, spk, wrong, wav.samples.data(), 16000, 16000, 0, 0) == nullptr); + CHECK(std::strlen(parakeet_capi_last_error(spk)) > 0); + CHECK(parakeet_capi_diarize_named_pcm_json(diar, spk, reg, nullptr, 16000, 16000, 0, 0) == nullptr); + CHECK(std::strstr(parakeet_capi_last_error(diar), "invalid samples buffer") != nullptr); + CHECK(std::strlen(parakeet_capi_last_error(spk)) == 0); + // diar fails, then speaker fails: the diar message is gone. + CHECK(parakeet_capi_diarize_named_pcm_json(diar, spk, wrong, wav.samples.data(), 16000, 16000, 0, 0) == nullptr); + CHECK(std::strstr(parakeet_capi_last_error(spk), "3-value") != nullptr); + CHECK(std::strlen(parakeet_capi_last_error(diar)) == 0); + parakeet_capi_speaker_registry_free(wrong); + } + + parakeet_capi_speaker_registry_free(reg); + parakeet_capi_free(diar); + parakeet_capi_free(spk); + if (failures) { std::fprintf(stderr, "%d failure(s)\n", failures); return 1; } + std::printf("test_capi_diarize_named: PASS\n"); + return 0; +} diff --git a/tests/test_capi_speaker.cpp b/tests/test_capi_speaker.cpp index 435a879..fb09566 100644 --- a/tests/test_capi_speaker.cpp +++ b/tests/test_capi_speaker.cpp @@ -32,7 +32,7 @@ int main() { const char* vd_path = std::getenv("PARAKEET_TEST_VD_GGUF"); if (!diar_path || !vd_path) return 77; - CHECK(parakeet_capi_abi_version() == 9); + CHECK(parakeet_capi_abi_version() == 10); parakeet_ctx* spk = parakeet_capi_load(vd_path); if (!spk) { std::fprintf(stderr, "FAIL: load speaker model (built without voice-detect?)\n"); return 77; } diff --git a/tests/test_capi_speaker_registry.cpp b/tests/test_capi_speaker_registry.cpp new file mode 100644 index 0000000..033becb --- /dev/null +++ b/tests/test_capi_speaker_registry.cpp @@ -0,0 +1,72 @@ +// Registry entry points that need no model: add_embedding, size, save, load, +// errors. Model independent. +#include "parakeet_capi.h" + +#include +#include +#include +#include +#include +#include + +static int failures = 0; +#define CHECK(cond) \ + do { \ + if (!(cond)) { \ + std::fprintf(stderr, "FAIL: %s (line %d)\n", #cond, __LINE__); \ + ++failures; \ + } \ + } while (0) + +int main() { + CHECK(parakeet_capi_abi_version() == 10); + + parakeet_speaker_registry* reg = parakeet_capi_speaker_registry_new(); + CHECK(reg != nullptr); + const float a[4] = {1, 0, 0, 0}; + const float b[4] = {0, 1, 0, 0}; + CHECK(parakeet_capi_speaker_registry_add_embedding(reg, "ada", a, 4) == 0); + CHECK(parakeet_capi_speaker_registry_add_embedding(reg, "ben", b, 4) == 0); + CHECK(parakeet_capi_speaker_registry_size(reg) == 2); + // Same name again: averaged into the same speaker, not a third one. + const float a2[4] = {0.9f, 0.1f, 0, 0}; + CHECK(parakeet_capi_speaker_registry_add_embedding(reg, "ada", a2, 4) == 0); + CHECK(parakeet_capi_speaker_registry_size(reg) == 2); + + // Errors, each leaves the registry unchanged and says why on the registry. + const float c3[3] = {1, 0, 0}; + CHECK(parakeet_capi_speaker_registry_add_embedding(reg, "cy", c3, 3) != 0); + CHECK(std::strstr(parakeet_capi_speaker_registry_last_error(reg), "expects") != nullptr); + const float nan4[4] = {std::numeric_limits::quiet_NaN(), 0, 0, 0}; + CHECK(parakeet_capi_speaker_registry_add_embedding(reg, "dee", nan4, 4) != 0); + const float inf4[4] = {std::numeric_limits::infinity(), 0, 0, 0}; + CHECK(parakeet_capi_speaker_registry_add_embedding(reg, "dee", inf4, 4) != 0); + const float zero4[4] = {0, 0, 0, 0}; + CHECK(parakeet_capi_speaker_registry_add_embedding(reg, "dee", zero4, 4) != 0); + CHECK(parakeet_capi_speaker_registry_add_embedding(reg, "", a, 4) != 0); + CHECK(parakeet_capi_speaker_registry_add_embedding(reg, nullptr, a, 4) != 0); + CHECK(parakeet_capi_speaker_registry_add_embedding(reg, "ee", nullptr, 4) != 0); + CHECK(parakeet_capi_speaker_registry_add_embedding(reg, "ee", a, 0) != 0); + CHECK(parakeet_capi_speaker_registry_add_embedding(reg, "ee", a, -1) != 0); + CHECK(parakeet_capi_speaker_registry_add_embedding(nullptr, "ee", a, 4) != 0); + CHECK(parakeet_capi_speaker_registry_size(reg) == 2); + // A success clears the message. + CHECK(parakeet_capi_speaker_registry_add_embedding(reg, "ben", b, 4) == 0); + CHECK(std::strlen(parakeet_capi_speaker_registry_last_error(reg)) == 0); + + // A registry filled this way saves and loads like any other. + const std::filesystem::path p = std::filesystem::temp_directory_path() / "pk_test_add_embedding.bin"; + std::filesystem::remove(p); + CHECK(parakeet_capi_speaker_registry_save(reg, p.string().c_str()) == 0); + parakeet_speaker_registry* back = parakeet_capi_speaker_registry_load(p.string().c_str()); + CHECK(back != nullptr && parakeet_capi_speaker_registry_size(back) == 2); + // And a loaded registry keeps its dimension: a wrong-sized vector is refused. + if (back) CHECK(parakeet_capi_speaker_registry_add_embedding(back, "cy", c3, 3) != 0); + parakeet_capi_speaker_registry_free(back); + std::filesystem::remove(p); + + parakeet_capi_speaker_registry_free(reg); + if (failures) { std::fprintf(stderr, "%d failure(s)\n", failures); return 1; } + std::printf("test_capi_speaker_registry: PASS\n"); + return 0; +}