Skip to content

About

No description, website, or topics provided.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Repository files navigation

Compass

Discover what to watch — by talking.

A voice-first entertainment experience built for the living-room TV.
Tell Compass your mood, occasion and constraints; get a small set of recommendations that actually make sense.

Working MVP SLNG challenge winner React 19 Python 3.12 Docker Compose

Why Compass · Quick start · Architecture · Product experience · Documentation

Compass brand and product overview

Why Compass

Streaming catalogues make people scroll when what they really want is help deciding. Compass replaces the endless grid with a short, natural conversation:

  • Speak naturally. Describe a mood, who is watching or how much time you have.
  • Refine in real time. Recommendations evolve as the conversation adds context.
  • Stay in control. Pause the conversation, explore a title and return without losing the session.
  • Designed for television. Every important action works with directional focus, OK/Back and a voice shortcut.

This repository contains the React TV interface, an independently deployable Pipecat voice agent and a FastAPI recommendation backend. Together they serve a unified catalogue of movies, television schedules and live football.

Quick start

Docker Compose (recommended)

Copy the environment template, add the two provider credentials and start the complete stack:

cp .env.example .env
# Fill in SLNG_API_KEY and NEBIUS_API_KEY in .env.
docker compose up -d --build --wait --wait-timeout 180

Open localhost:8080 and allow microphone access when prompted. The first build downloads dependencies, trains the sample models and compiles the voice runtime, so it can take a few minutes.

# Inspect the stack
docker compose ps
docker compose logs --tail=100 backend voice-agent frontend

# Stop it without deleting persistent data
docker compose down

See the deployment guide for production credentials, persistent data, automatic HTTPS and WebRTC networking.

Current status

Area Status Notes
TV interface Working MVP Browser simulation of the Titan OS experience
Remote navigation Working Directional focus, OK, Back and voice shortcut
Profile selection Integrated Backend usernames from GET /api/users
Voice conversation Working Real microphone, STT, LLM, TTS and speaker output
Recommendation UI Integrated Real backend items, session-cached details and animated refinement
Backend API Integrated FastAPI recommendation endpoints; profiles are backend usernames
Voice-driven recommendations Implemented Scoped tools emit structured events over the existing WebRTC connection
Titan OS device validation Pending The browser currently simulates the TV environment

Architecture

┌────────────────────────── TV / browser ──────────────────────────┐
│ React + TypeScript                                               │
│                                                                  │
│ Profile / Home / Recommendation UI     Pipecat JS client         │
│ TV focus + remote navigation            SmallWebRTC transport     │
└─────────────────────┬──────────────────────────┬─────────────────┘
                      │ HTTP API                 │ WebRTC audio/data
                      ▼                          ▼
              ┌───────────────┐       ┌────────────────────────────┐
              │ Product       │◄──────│ Voice agent                │
              │ backend       │ tools │ Unmute definition          │
              │               │       │ compiled to Pipecat        │
              │ profiles      │       └─────────────┬──────────────┘
              │ catalogue     │                     │
              │ recommendations│       ┌────────────┴─────────────┐
              └───────────────┘        │ SLNG STT → Nebius LLM   │
                                       │          → SLNG TTS       │
                                       └──────────────────────────┘

The browser never receives provider secrets. It starts a voice session against the local agent and exchanges microphone and speaker tracks over WebRTC. The agent owns provider credentials and is the only layer that calls SLNG and Nebius.

Technologies and responsibilities

Technology Where Responsibility
React + TypeScript + Vite frontend/ TV interface and application flow
Zustand frontend/src/store/ Profile, voice and recommendation state
Motion for React Frontend Shared-layout and staggered recommendation animations
Pipecat client + SmallWebRTC Frontend Real-time microphone, remote audio and agent events
Unmute voice-agent/ Declarative agent definition and Pipecat compilation
Pipecat Generated runtime Real-time voice pipeline and WebRTC server
SLNG Voice runtime Speech-to-text and text-to-speech execution
Nebius Token Factory Voice runtime Conversational reasoning model
Silero Voice runtime Local voice activity and turn detection

Product experience

The interface moves from a calm, voice-led home screen into a focused conversation and then reveals recommendations without leaving the session. Click any image to view it at full size.

Home Voice conversation
Compass home screen Compass listening screen
Start with one clear action: press to talk. Live visual states make listening and speaking explicit.

Recommendations that remain conversational

Compass recommendation board

Results appear as a TV-friendly board while the voice agent remains available to refine the selection — for example by changing genre, mood or runtime.

Hackathon challenge strategy

Titan OS: TV-first experience

Compass is designed as a ten-foot interface rather than a desktop application:

  • every important action is reachable with directional focus and OK/Back;
  • focus states are high-contrast and use the Compass cyan brand colour;
  • the primary Home action is voice, reducing remote-control typing;
  • the conversation has explicit listening, thinking, speaking, paused and recommendation states;
  • recommendations arrive progressively and can be frozen for calm exploration;
  • Titan-specific APIs will remain behind frontend/src/platform/ so browser simulation and device runtime can share the product code.

The current limitation is explicit: it has not yet been validated on a Titan OS television. The browser build simulates screen proportions, remote navigation and interaction flow until device access is available.

Best use of the SLNG platform 🥇 (winners of the challenge)

SLNG is in the critical audio path, not an optional add-on:

viewer speech
  → SLNG deepgram/nova:3 speech-to-text
  → Nebius conversational reasoning
  → SLNG deepgram/aura:2 text-to-speech
  → TV speakers

The agent uses the EU-West SLNG route. A small repository-owned adapter batches adjacent twenty-millisecond WebRTC frames, reducing approximately three thousand WebSocket audio messages per minute to approximately fifteen hundred. This stays below the observed two-thousand-message-per-minute bridge limit while adding at most one input-frame interval of buffering.

Unmute bonus

voice-agent/agent.yaml and voice-agent/instructions.md are the source of truth. Unmute validates this portable definition and compiles it into a Pipecat runtime that the team owns and can run locally. Provider and model choices stay declarative instead of being embedded throughout application code.

Build, adapt and ship with Nebius Token Factory

Nebius Token Factory provides the LLM for every conversational turn. The current model is Qwen/Qwen3-30B-A3B-Instruct-2507, accessed through Nebius' OpenAI- compatible endpoint. It interprets viewing intent, decides which clarification is valuable and produces concise spoken responses. The same model invokes recommendation and cached-detail tools using the active profileId.

This is a meaningful dependency: replacing or removing Nebius removes the reasoning layer of the live agent rather than a peripheral feature.

Manual local setup

1. Start the backend

Install the dependencies and train the models as described in BACKEND.md, then:

cd backend
uvicorn app.main:app --reload --port 8000

2. Configure the voice agent

Copy the example and add local credentials. Never commit or expose these values through a Vite variable.

Copy-Item voice-agent\.env.example voice-agent\.env

Required variables:

  • SLNG_API_KEY
  • NEBIUS_API_KEY
  • NEBIUS_BASE_URL

3. Start the voice runtime

.\voice-agent\scripts\dev.ps1

On Linux/macOS, with Unmute 0.4.2 and uv on PATH:

python voice-agent/run.py

The agent listens on http://localhost:7860. Leave it running while using the Compass frontend. An idle server does not hold an active conversation; disconnect voice sessions after testing to conserve SLNG credit.

4. Start the frontend

cd frontend
pnpm install
pnpm dev

Open http://localhost:5173. Allow microphone access when prompted.

TV controls

  • Arrow keys: move focus.
  • Enter: select.
  • Escape or browser Back: return.
  • V: activate the current voice control.

Documentation

Recommendation flow

Profile.id is the backend username. HTTP requests send X-Profile-Id, and the WebRTC start body carries { sessionId, profileId, mode }.

Frontend mode Backend endpoint
discover POST /api/content/preference
consensus POST /api/content/room
decide POST /api/content/decide

Voice tools merge structured preferences and emit recommendations.updated. The frontend adapts the event's catalogue items into the same session cache used by the board and detail page. Pause mutes microphone and remote audio; navigating to details and back keeps the connection, candidates and selection. Resume uses that connection. Leaving the session for Home or profiles clears its context.

The backend has no content-by-ID route: details are available only for items cached during this session. Refreshing the browser loses that cache. Playback and provider deep links remain future work.


Compass icon
Good movies. Better conversations.

About

No description, website, or topics provided.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages