Earshot

Ridiculously fast & accurate streaming voice activity detection, written in pure Rust and also available for Python.

Earshot achieves an RTF of 0.0003 (3,600x real time): 40x faster than Silero VAD v6 & TEN VAD - and more accurate, too!

Earshot operates on 16 millisecond frames of mono/stereo audio sampled at 16000 Hz & supports streaming. Earshot detects voice in any language and is resilient to most kinds of environmental noise with an SNR ≥ 3dB.

If you find Earshot useful, please consider sponsoring pyke.io.

Earshot, in black, performs markedly better than Silero VAD v6 and TEN VAD in blue and red.

Usage

• Python: pip install earshot

Rust

cargo add earshot

// Get per-frame probabilities from a real-time audio stream:
let mut detector = earshot::Detector::default();
let mut frame_receiver = ...
while let Some(frame) = frame_receiver.recv() {
	// `frame` is Vec<i16> with length 256.
	// Each frame passed to the detector must be exactly 256 samples (16ms) @ 16 KHz sample rate.
	// f32 [-1, 1] slices/vecs are also supported here.
	let score = detector.predict(&frame);
	if score.is_voice() {
		println!("Voice detected! Score: {:.1}%", score.raw * 100.);
	}

	// If the frame is interleaved stereo, use the Stereo wrapper:
	let score = detector.predict(earshot::Stereo(&frame));
}

// Get segments from a full audio buffer:
for segment in earshot::segments(audio, &earshot::SegmenterOptions::default()) {
	println!(
		"Voice detected from {:.2}s - {:.2}s ({:.2}s)",
		segment.start_secs(),
		segment.end_secs(),
		segment.duration_secs()
	);
}

Binary & memory size

Earshot is very embedded-friendly: each instance of Detector uses ~8 KiB of memory to store the audio buffer & neural network state. Binary footprint is ~95 KiB; the neural network is 40 KiB of that.

In contrast, Silero's model is 2 MiB, TEN's is 310 KiB, but both require ONNX Runtime, which adds an additional 8 MB to your binary (+ a whole lot more memory).

#![no_std]

Earshot supports #![no_std], but it does require the libm crate. The std feature is enabled by default, so add default-features = false and features = [ "libm" ] to enable #![no_std]:

[dependencies]
earshot = { version = "1", default-features = false, features = [ "libm" ] }