feat: VAD-only GGUF files (head, front end, subsampler) and a slicer

AGENTS.md

@@ -113,6 +113,7 @@ examples/cli/        parakeet-cli binary
 scripts/             Python tooling
                        convert_parakeet_to_gguf.py, .nemo/.hf -> GGUF (--dtype f32|f16|q8_0)
                        convert_hf_parakeet_to_gguf.py, HF safetensors (moondream/parakeet-ultra, -redux) to GGUF (--template, --ternary keep|dequant, --vad keep|drop)
+                       slice_vad_gguf.py          , cut a VAD-only GGUF (parakeet.arch "vad") out of an Ultra or Redux GGUF
                        gen_nemo_baseline.py        , NeMo intermediates -> baseline.gguf
                        gen_stream_baseline.py      , NeMo cache-aware streaming encode+decode -> stream baseline.gguf
                        gen_diar_baseline.py        , NeMo offline + streaming diarization -> diar baseline.gguf
@@ -150,6 +151,7 @@ tests/               ctest targets
                        test_vad_head.cpp       , VAD head probabilities (PARAKEET_TEST_GGUF_ULTRA)
                        test_vad_segmenter.cpp  , segmenter cut rules, 32 ms grid, Silero defaults, event tracker (model-independent)
                        test_vad_options.cpp    , VAD option parser, NULL and bad-file C-API paths (model-independent)
+                       test_vad_only.cpp       , VAD-only slice: C-API VAD equals the parent, other calls refuse it (PARAKEET_TEST_VAD_ONLY_GGUF, PARAKEET_TEST_GGUF_ULTRA)
                        test_capi_vad_silero.cpp, Silero via the C-API: JSON, options, threads, stream (PARAKEET_TEST_SILERO_GGUF)
                        test_transcribe_vad_silero.cpp, transcribe with Silero segments on a model without a head (PARAKEET_TEST_SILERO_GGUF, PARAKEET_TEST_GGUF)
                        test_transcribe_vad.cpp , --vad path vs plain pass on long audio (PARAKEET_TEST_GGUF_ULTRA, PARAKEET_TEST_GGUF)

README.md

@@ -84,6 +84,7 @@ ternary ones.
 - Silero VAD (MIT, 32 ms frames, 16 kHz and 8 kHz) runs from its own small GGUF ([download](https://huggingface.co/mudler/parakeet-cpp-gguf)) through the same
   functions, as a stream (`parakeet_capi_vad_stream_*`), and as the cutter for any ASR model:
   `parakeet-cli transcribe --vad --vad-model silero.gguf`. See [`docs/vad.md`](docs/vad.md).
+- VAD-only slices of Ultra and Redux (6 to 10 MB, `redux-vad.gguf` and `ultra-vad-q8_0.gguf` in the same repo) hold just the head and its front end. They run `vad` and the `parakeet_capi_vad_*` calls and cannot transcribe. See [`docs/vad.md`](docs/vad.md).
 ---
 
 ## Performance
@@ -295,6 +296,11 @@ parakeet-cli transcribe --model ultra.gguf --input long.wav --vad
 # Tune with --threshold F (0.5), --min-pause SEC (0.2), --min-speech SEC (0.1), --max-segment SEC (30)
 parakeet-cli vad --model ultra.gguf --input audio.wav
 
+# Only the VAD head is needed? Use a 6 to 10 MB slice instead of the full model
+# (it cannot transcribe). Files and checksums: docs/vad.md.
+curl -LO https://huggingface.co/mudler/parakeet-cpp-gguf/resolve/main/redux-vad.gguf
+parakeet-cli vad --model redux-vad.gguf --input audio.wav
+
 # The same with a Silero VAD GGUF (frame_sec 0.032; defaults 250 ms min speech,
 # 100 ms min pause, 30 ms pad). Any ASR model can then cut long audio with it.
 # Download it from the collection repo (F16 is 1.3 MB, F32 is 2.2 MB). To make the

docs/vad.md

@@ -136,6 +136,48 @@ mode (frame period 0.032 s) and the same decode of each segment as the head
 path. Audio of at most 30 s is transcribed whole and the VAD does not run.
 Passing NULL as the second argument uses the model's own head.
 
+## VAD-only slice
+
+The VAD head of Ultra and Redux reads only the log-mel front end, the subsampler
+and the head, which is about 10 MB of the 213 MB to 1.4 GB file. `scripts/slice_vad_gguf.py`
+copies exactly those tensors, byte for byte, into a small GGUF:
+
+    python scripts/slice_vad_gguf.py ultra-q8_0.gguf ultra-vad.gguf
+
+Nothing is requantized, so the VAD output is the same as the parent's. The slice
+copies the `parakeet.encoder.*`, `parakeet.preprocessor.*` and `parakeet.vad.*`
+keys. It sets `parakeet.arch` to `vad` (`general.architecture` stays `parakeet`),
+and it records the parent in `parakeet.vad_only.parent_name`, `parent_file`,
+`parent_sha256`, `parent_bytes` and `parent_arch`.
+
+The `vad` marker is what the loader needs: a normal file has an encoder,
+a decoder and a vocabulary, and the slice has none of them. `parakeet-cli vad`
+and the `parakeet_capi_vad_*` functions accept the file (`Model::load_vad_only`).
+`Model::load` refuses it with a log message, so every other call (transcribe,
+diarize, stream) fails with "context holds a VAD-only model" or a load error.
+Files that are not slices load exactly as before.
+
+### Published slices
+
+Two slices are published in [`mudler/parakeet-cpp-gguf`](https://huggingface.co/mudler/parakeet-cpp-gguf).
+They are cut out of Moondream's models, not trained here, and carry the same
+CC-BY-4.0 license: credit Moondream and NVIDIA.
+
+| File | Parent | Size (bytes) | SHA-256 |
+|---|---|---:|---|
+| [`redux-vad.gguf`](https://huggingface.co/mudler/parakeet-cpp-gguf/resolve/main/redux-vad.gguf) | parakeet-redux | 9,939,488 | `588e1d6e2ee5b6cdfd9ec5ea98dc0993d5bea498d9cc4ec8d6077041eef8a34f` |
+| [`ultra-vad-q8_0.gguf`](https://huggingface.co/mudler/parakeet-cpp-gguf/resolve/main/ultra-vad-q8_0.gguf) | parakeet-ultra, Q8_0 | 6,007,328 | `8b891a4435e97438104ca07c72530d0c5fe62b986baee48b2dd4e1500c1d4758` |
+
+    parakeet-cli vad --model redux-vad.gguf --input audio.wav
+
+A slice cannot transcribe, diarize or stream.
+
+The two Redux parents (packed ternary and dequantized F16) give slices with the same
+tensors: the subsampler and the head are not ternary. The Ultra Q8_0 parent has a
+Q8_0 final subsampler projection, so its slice is 6.0 MB; the F16 parents give
+9.9 MB. A slice from a Q8_0 parent and a slice from an F16 parent give different
+probabilities because their weights differ, not because of the slicing.
+
 ## ABI
 
 All of this is additive. `parakeet_capi_abi_version` stays 10. A new symbol set

examples/cli/main.cpp

@@ -1901,7 +1901,7 @@ static int cmd_vad_probe(int argc, char** argv) {
 //   [--min-pause SEC] [--min-speech SEC] [--speech-pad SEC] [--max-segment SEC=30]
 //   [--mode speech|segments] [--probabilities] [--threads N]
 // Prints the same JSON as parakeet_capi_vad_path_json. The model is an ASR GGUF
-// with a VAD head (Ultra, Redux) or a Silero VAD GGUF. Unset options keep the
+// with a VAD head (Ultra, Redux), a VAD-only slice of one, or a Silero VAD GGUF. Unset options keep the
 // defaults of that model kind.
 static int cmd_vad(int argc, char** argv) {
     std::string model, input;
@@ -1974,7 +1974,8 @@ static int cmd_vad(int argc, char** argv) {
             std::printf("%s\n", pk::silero_vad_to_json(*sv, audio.samples, 16000, req).c_str());
             return 0;
         }
-        std::unique_ptr<pk::Model> m = pk::Model::load(model);
+        std::unique_ptr<pk::Model> m =
+            pk::gguf_is_vad_only(model) ? pk::Model::load_vad_only(model) : pk::Model::load(model);
         if (!m) { std::fprintf(stderr, "parakeet-cli: failed to load model %s\n", model.c_str()); return 1; }
         std::printf("%s\n", pk::vad_to_json(*m, audio.samples, req).c_str());
     } catch (const std::exception& e) {

models/MANIFEST.md

@@ -129,6 +129,13 @@ or AVX-512 VNNI and aarch64 with dotprod get SIMD kernels for the packed file. T
 original packed tensors resident next to the repacked planes, so its memory use is more than 213 MB.
 Ultra Q8_0 is measured in `docs/ternary.md`.
 
+VAD-only slices (made with `scripts/slice_vad_gguf.py`, cut from the Ultra and Redux files above, not retrained; CC-BY-4.0, credit Moondream and NVIDIA). They run VAD only and cannot transcribe:
+
+| File | Parent | Size (bytes) | SHA-256 |
+|---|---|---:|---|
+| [`redux-vad.gguf`](https://huggingface.co/mudler/parakeet-cpp-gguf/resolve/main/redux-vad.gguf) | redux | 9,939,488 | `588e1d6e2ee5b6cdfd9ec5ea98dc0993d5bea498d9cc4ec8d6077041eef8a34f` |
+| [`ultra-vad-q8_0.gguf`](https://huggingface.co/mudler/parakeet-cpp-gguf/resolve/main/ultra-vad-q8_0.gguf) | ultra, `--dtype q8_0` | 6,007,328 | `8b891a4435e97438104ca07c72530d0c5fe62b986baee48b2dd4e1500c1d4758` |
+
 Both carry the `parakeet.vad.*` KVs and `vad_head.*` tensors unless converted
 with `--vad drop`.
 

scripts/slice_vad_gguf.py

@@ -0,0 +1,108 @@
+#!/usr/bin/env python3
+"""Cut a VAD-only GGUF out of an Ultra or Redux parakeet GGUF.
+
+moondream/parakeet-ultra and -redux carry a voice-activity head on the
+encoder's subsampler output. The VAD path reads only the log-mel filterbank and
+window, the subsampler (`encoder.pre_encode.*`) and the head (`vad_head.*`).
+This script copies exactly those tensors, byte for byte, plus the config keys
+that path reads. Nothing is requantized, so the slice produces the same VAD
+output as its parent.
+
+The output sets `parakeet.arch` to `vad`. The loader needs that marker: a
+normal file has an encoder, a decoder and a vocabulary, and the slice has none
+of them. With the marker, `parakeet-cli vad` and the C-API VAD calls load the
+file, and every other entry point refuses it with a clear message.
+`general.architecture` stays `parakeet`.
+
+Provenance keys written (all strings or integers):
+  parakeet.vad_only.parent_name     general.name of the parent
+  parakeet.vad_only.parent_file     file name of the parent
+  parakeet.vad_only.parent_sha256   SHA-256 of the parent file
+  parakeet.vad_only.parent_bytes    size of the parent file
+  parakeet.vad_only.parent_arch     parakeet.arch of the parent (for example tdt)
+
+    python scripts/slice_vad_gguf.py ultra-q8_0.gguf ultra-vad.gguf
+"""
+import argparse
+import hashlib
+import pathlib
+import sys
+
+try:
+    import gguf
+except ImportError as e:  # pragma: no cover - env guard
+    print(f"slice_vad_gguf: missing dependency: {e}", file=sys.stderr)
+    sys.exit(2)
+
+TENSOR_PREFIXES = ("vad_head.", "encoder.pre_encode.", "preprocessor.featurizer.")
+KEY_PREFIXES = ("parakeet.encoder.", "parakeet.preprocessor.", "parakeet.vad.")
+# Packed ternary tensors cannot appear on the VAD path; refuse them loudly.
+PACKED_SUFFIXES = (".weight_packed", ".weight_scales")
+
+
+def sha256_file(path):
+    h = hashlib.sha256()
+    with open(path, "rb") as f:
+        for blk in iter(lambda: f.read(1 << 22), b""):
+            h.update(blk)
+    return h.hexdigest()
+
+
+def field_value(f):
+    return f.contents()
+
+
+def main():
+    ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
+    ap.add_argument("src", help="Ultra or Redux GGUF with a VAD head")
+    ap.add_argument("dst", help="output VAD-only GGUF")
+    a = ap.parse_args()
+
+    r = gguf.GGUFReader(a.src)
+    flds = r.fields
+    present = flds.get("parakeet.vad.present")
+    if present is None or not bool(field_value(present)):
+        sys.exit("slice_vad_gguf: source has no VAD head (parakeet.vad.present is not true)")
+    arch = flds.get("parakeet.arch")
+    if arch is not None and str(field_value(arch)) == "vad":
+        sys.exit("slice_vad_gguf: source is already a VAD-only file")
+
+    tensors = [t for t in r.tensors if t.name.startswith(TENSOR_PREFIXES)]
+    names = {t.name for t in tensors}
+    for need in ("vad_head.proj.weight", "vad_head.proj.bias", "vad_head.ctx.weight", "vad_head.ctx.bias",
+                 "vad_head.out.weight", "vad_head.out.bias", "preprocessor.featurizer.fb",
+                 "preprocessor.featurizer.window", "encoder.pre_encode.out.weight"):
+        if need not in names:
+            sys.exit(f"slice_vad_gguf: source lacks tensor {need}")
+    for t in tensors:
+        if t.name.endswith(PACKED_SUFFIXES):
+            sys.exit(f"slice_vad_gguf: packed tensor {t.name} on the VAD path; not supported")
+
+    parent_name = str(field_value(flds["general.name"])) if "general.name" in flds else pathlib.Path(a.src).name
+    w = gguf.GGUFWriter(a.dst, "parakeet")
+    w.add_name(parent_name + " (VAD only)")
+    w.add_string("parakeet.arch", "vad")
+    for k, f in flds.items():
+        if not k.startswith(KEY_PREFIXES):
+            continue
+        # Copy with the original GGUF value type so the C++ readers see the same kind.
+        vt = f.types[0]
+        sub = f.types[-1] if vt == gguf.GGUFValueType.ARRAY else None
+        w.add_key_value(k, field_value(f), vt, sub_type=sub)
+    w.add_string("parakeet.vad_only.parent_name", parent_name)
+    w.add_string("parakeet.vad_only.parent_file", pathlib.Path(a.src).name)
+    w.add_string("parakeet.vad_only.parent_sha256", sha256_file(a.src))
+    w.add_uint64("parakeet.vad_only.parent_bytes", pathlib.Path(a.src).stat().st_size)
+    w.add_string("parakeet.vad_only.parent_arch", str(field_value(arch)) if arch is not None else "")
+    for t in tensors:
+        w.add_tensor(t.name, t.data, raw_shape=t.data.shape, raw_dtype=t.tensor_type)
+    w.write_header_to_file()
+    w.write_kv_data_to_file()
+    w.write_tensors_to_file()
+    w.close()
+    n = sum(t.n_bytes for t in tensors)
+    print(f"wrote {a.dst}: {len(tensors)} tensors, {n} tensor bytes, from {a.src}")
+
+
+if __name__ == "__main__":
+    main()

src/diarization.cpp

@@ -3,6 +3,7 @@
 
 #include "audio_io.hpp"
 #include "backend.hpp"
+#include "common.hpp"
 #include "ggml_graph.hpp"
 
 #include <algorithm>
@@ -21,6 +22,8 @@ std::unique_ptr<DiarizationModel> DiarizationModel::load(const std::string& path
     const auto& cfg = m->loader_.config();
     if (cfg.arch != "diarization") {
         // Not a diarization model — caller should use Model for ASR.
+        if (cfg.arch == "vad")
+            PK_LOG("%s is a VAD-only file and cannot diarize", path.c_str());
         return nullptr;
     }
     if (!cfg.diarization.present) {

src/model.cpp

@@ -52,6 +52,11 @@ std::unique_ptr<Model> Model::load(const std::string& gguf_path) {
     if (!m->loader_.load(gguf_path)) {
         return nullptr;
     }
+    if (m->loader_.config().arch == "vad") {
+        PK_LOG("%s is a VAD-only file: it has no encoder or decoder, so it cannot transcribe or diarize; "
+               "use it with `parakeet-cli vad` or the parakeet_capi_vad_* functions", gguf_path.c_str());
+        return nullptr;
+    }
     // Model is the ASR entry point — reject diarization models so the C-API
     // can fall through to DiarizationModel::load.
     if (m->loader_.config().arch == "diarization") {
@@ -90,6 +95,28 @@ std::unique_ptr<Model> Model::load(const std::string& gguf_path) {
     return m;
 }
 
+std::unique_ptr<Model> Model::load_vad_only(const std::string& gguf_path) {
+    std::unique_ptr<Model> m(new (std::nothrow) Model());
+    if (!m) return nullptr;
+    if (!m->loader_.load(gguf_path)) return nullptr;
+    const ParakeetConfig& c = m->loader_.config();
+    if (c.arch != "vad") {
+        PK_LOG("%s is not a VAD-only file (parakeet.arch is \"%s\")", gguf_path.c_str(), c.arch.c_str());
+        return nullptr;
+    }
+    if (!c.vad.present) {
+        PK_LOG("%s: VAD-only file without parakeet.vad.present", gguf_path.c_str());
+        return nullptr;
+    }
+    // The slice never carries packed ternary tensors (the slicer refuses them).
+    if (c.ternary.present || m->loader_.has_tensor_with_suffix(".weight_packed")) {
+        PK_LOG("%s: a VAD-only file cannot hold packed ternary tensors", gguf_path.c_str());
+        return nullptr;
+    }
+    ensure_weights_realized(m->loader_);
+    return m;
+}
+
 std::shared_ptr<BackendPool> Model::pool_snapshot() const {
     std::lock_guard<std::mutex> lk(pool_mu_);
     return pool_;

src/model.hpp

@@ -42,8 +42,15 @@ struct NBestTranscription {
 class Model {
 public:
     // Loads the GGUF at `gguf_path`. Returns nullptr on failure (no throw).
+    // A VAD-only slice (parakeet.arch "vad") is refused here with a log message:
+    // it holds no encoder blocks or decoder. Use load_vad_only().
     static std::unique_ptr<Model> load(const std::string& gguf_path);
 
+    // Loads a VAD-only slice made by scripts/slice_vad_gguf.py. Returns nullptr
+    // on failure or when the file is not a slice. The result serves
+    // vad_probabilities() and config() only; every other call is invalid.
+    static std::unique_ptr<Model> load_vad_only(const std::string& gguf_path);
+
     // Transcribe raw mono float PCM. If `sample_rate != 16000` the audio is
     // linearly resampled to 16 kHz (via pk::resample_linear) before inference.
     // `target_lang` selects the language prompt for multilingual (nemotron)

src/model_loader.cpp

@@ -278,7 +278,8 @@ bool ModelLoader::load(const std::string& path){
     const int64_t nt = gguf_get_n_tensors(gguf_);
     for(int64_t i=0;i<nt;++i){ const char* nm = gguf_get_tensor_name(gguf_,i);
         ggml_tensor* t = ggml_get_tensor(ctx_, nm); if(t) tensors_[nm]=t; }
-    return cfg_.d_model>0 && (cfg_.vocab_size>0 || cfg_.arch=="diarization");
+    // A VAD-only slice (arch "vad", see scripts/slice_vad_gguf.py) has no vocabulary.
+    return cfg_.d_model>0 && (cfg_.vocab_size>0 || cfg_.arch=="diarization" || cfg_.arch=="vad");
 }
 ggml_tensor* ModelLoader::tensor(const std::string& n) const {
     auto it = tensors_.find(n); return it==tensors_.end()? nullptr : it->second;