English · 中文 · LiveTalking Studio guide (Русский)
LiveTalking is a real-time talking-avatar engine. It turns text or speech into synchronized video and audio and can stream the result through WebRTC, RTMP, or a virtual camera. This fork also includes LiveTalking Studio, a Linux Electron app for running and using a local LiveTalking setup.
Studio brings avatar selection and creation, Qwen speech services, conversation, recording, and process control into one window. The Python server and its web pages remain available for users who prefer the command line or want to integrate through HTTP APIs.
| Start here | What it provides |
|---|---|
| LiveTalking Studio | Linux desktop interface for setup, avatars, voice conversation, recording, and projection |
| Python server and web UI | Direct access to LiveTalking transports, web pages, and APIs |
| API documentation | WebRTC sessions, avatar jobs, administration, and virtual camera integration |
Studio runs from this source checkout. It requires Linux with a graphical session and Node.js 22 or newer. There is currently no AppImage or .deb package in this repository.
git clone https://github.com/kmrov/LiveTalking.git
cd LiveTalking/desktop
npm ci
npm startIf installation scripts were disabled and Electron reports that it is missing, run node node_modules/electron/install.js from desktop/, then start again.
Studio finds the surrounding LiveTalking checkout and guides you through its Python path, avatar, speech services, and voice sample. The interface is currently in Russian: Проверить (Check) reports missing prerequisites, and Запустить (Start) launches compatible local services or connects to services already running. Studio downloads missing model weights when needed, but Python, CUDA, vLLM, vLLM-Omni, and other software dependencies must be installed separately. A local avatar requires an NVIDIA GPU even if speech servers run elsewhere.
For local speech, Studio uses Qwen/Qwen3-ASR-0.6B with vLLM and Qwen/Qwen3-TTS-12Hz-1.7B-Base with vLLM-Omni. It can also connect to existing ASR and TTS servers. A voice sample consists of a WAV file and a matching transcript; keep the source WAV in place because the profile refers to it. See the Studio setup guide for environment layout, model setup, ports, and first-run steps.
- Browse prepared avatars, or create MuseTalk avatars from an image or video and Wav2Lip avatars from video. Avatar preparation can happen before speech or conversation is configured.
- Preview the avatar over WebRTC, send text to an LLM or directly to TTS, speak through the microphone, interrupt speech, and save MP4 recordings.
- Use continuous voice conversation: Studio detects the end of a spoken phrase, sends it to ASR, and resumes listening after the avatar answers. Optional interruption during playback depends on microphone echo cancellation.
- Connect to an optional Persona service for persistent conversations, memory, and documents. Persona is a separate project with its own Python environment, PostgreSQL with pgvector, and Yandex AI Studio setup.
- Send the avatar to Head in Jar over WHIP. Discovery, connection, and the physical projection step are described in the Studio guide.
Studio stops only the processes and database containers it started. Already running compatible services are left running. Profiles live in Electron user data; secrets use the Linux keyring when available and otherwise remain in memory until the app closes.
From desktop/:
npm test
npm run build
npm run smokenpm run build creates renderer files in desktop/dist/; it does not produce an installer. The smoke test uses a fixture server and does not require model weights or a GPU, but it does need an environment capable of launching Electron.
The Python server can be run without Studio. Set up a Python environment and a compatible CUDA/PyTorch stack for the avatar model you choose, then install the project dependencies. The example below uses the prepared Wav2Lip avatar from the original LiveTalking quick start: it expects models/wav2lip.pth and data/avatars/wav2lip256_avatar1 to exist.
cd /path/to/LiveTalking
python3.12 -m venv .venv
source .venv/bin/activate
# Install a CUDA-compatible PyTorch build for your system first.
pip install -r requirements.txt
python app.py --transport webrtc --model wav2lip --avatar_id wav2lip256_avatar1Open http://127.0.0.1:8010/index-en.html for the English web client. The server also includes avatar creation, administration, and virtual camera pages. Server-side model and CUDA dependencies vary by avatar and speech backend; consult the relevant model instructions before starting the server.
For a Qwen voice setup without Studio, see the Qwen ASR/TTS avatar guide and Qwen TTS guide (both in Russian). The combined launcher exposes its options with python scripts/start_qwen_avatar.py --help.
| Guide | Covers |
|---|---|
| Studio guide | Setup, avatars, speech, Persona, projection, troubleshooting, and checks (Russian) |
| API reference | WebRTC, text and audio input, recording, and actions |
| Avatar API | Avatar creation jobs and status |
| Admin API | Configuration and session monitoring |
| Virtual camera guide | Virtual camera setup and troubleshooting |
The core flow is text or speech → optional LLM → speech synthesis → avatar rendering → audio/video output. LiveTalking supports Wav2Lip, MuseTalk, Ultralight, and other avatar backends; Studio's creation and selection options are documented in its own guide.
| Path | Purpose |
|---|---|
desktop/ |
LiveTalking Studio Electron app and tests |
app.py, server/ |
Python server, routes, and session handling |
avatars/, tts/ |
Avatar and speech adapters |
web/ |
Browser clients |
scripts/ |
Launchers and avatar/model preparation workers |
docs/ |
API and setup guides |
This repository builds on lipku/LiveTalking. The project is distributed under the Apache 2.0 license. For the original Chinese project overview, see README-ZH.md.
If you use the underlying LiveTalking framework in research, cite the upstream project:
@software{livetalking,
author = {Hengzhong Li},
title = {LiveTalking: Real-Time Interactive Streaming Digital Human Framework},
year = {2025},
publisher = {GitHub},
url = {https://github.com/lipku/livetalking}
}