A lightweight client-server system for real-time audio processing with voice activity detection (VAD) and automatic speech recognition (ASR). This project demonstrates a complete pipeline from browser-based audio recording to real-time speech transcription.
- Browser-based Audio Capture: Record audio directly in the browser using Web Audio API
- Real-time Streaming: Efficient WebSocket communication for low-latency audio transmission
- Voice Activity Detection: Server-side VAD to detect speech segments and reduce unnecessary processing
- Speech Recognition: Integrated ASR for real-time transcription
┌─────────────┐ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐
│ Browser │ │ WebSocket │ │ VAD │ │ ASR │
│ Client │───▶│ Server │───▶│ Detection │───▶│ Engine │
│ (React) │ │ (Node.js) │ │ │ │ │
└─────────────┘ └─────────────┘ └─────────────┘ └─────────────┘
│ │ │ │
JavaScript Audio Frames Speech/Noise Transcription
Recording Transmission Detection Result
- Modern browser with Web Audio API support
- Python 3.8+ (for ASR engine, if using Python-based solution)
- Clone the repository
git clone https://github.com/HKAllThingsLink/streaming-asr.git
cd streaming-asr- Install server dependencies
pip install -r requirements.txt- Start the WebSocket server
cd streaming-asr
python main.py
# Server runs on http://localhost:9871- Open your browser and navigate to
http://localhost:9871 - Allow microphone permissions and start speaking
- Web Audio API - Audio capture and processing
- WebSocket - Real-time communication
- WebSocket - Real-time bidirectional communication
- webrtcvad - Voice activity detection
- SenseVoiceSmall - ASR engine
This project is licensed under the MIT License - see the LICENSE file for details.
Note: This is a demonstration project. For production use, consider additional optimizations for scalability, security, and reliability.