WaiComputer - AI Second Brain

An ecosystem for task-centric dialogue, realtime voice, transcription, app generation, and deployment with AI. Native iOS + macOS apps with a Python cloud backend.

Architecture

┌─────────────────────────────────────────────────────────────────┐
│                    WaiComputer Architecture                      │
├─────────────────────────────────────────────────────────────────┤
│  ┌──────────────────┐      ┌──────────────────┐                 │
│  │    iOS App       │      │   macOS App      │                 │
│  │   (SwiftUI)      │      │   (SwiftUI)      │                 │
│  │  • Microphone    │      │ • Mic + System   │                 │
│  │  • Notes/Voice   │      │ • Zoom/Meet      │                 │
│  └────────┬─────────┘      └────────┬─────────┘                 │
│           │         WebSocket        │                          │
│           │      (Opus audio)        │                          │
│           └────────────┬─────────────┘                          │
│                        │                                        │
│  ┌─────────────────────▼─────────────────────┐                  │
│  │         Python FastAPI Backend            │                  │
│  │              (Self-hosted VPS)            │                  │
│  ├───────────────────────────────────────────┤                  │
│  │  ┌─────────────┐  ┌─────────────────────┐ │                  │
│  │  │ ElevenLabs  │  │ Cerebras/OpenAI AI │ │                  │
│  │  │ Realtime    │  │   Summarization     │ │                  │
│  │  │ Voice + STT │  │   App Generation    │ │                  │
│  │  │             │  │   Tool Execution    │ │                  │
│  │  └─────────────┘  └─────────────────────┘ │                  │
│  └───────────────────────────────────────────┘                  │
│                        │                                        │
│  ┌─────────────────────▼─────────────────────┐                  │
│  │              Data Layer                    │                  │
│  ├───────────────────────────────────────────┤                  │
│  │  PostgreSQL    │  pgvector     │  Redis   │                  │
│  │  + FTS         │  Embeddings   │  Agents  │                  │
│  └───────────────────────────────────────────┘                  │
└─────────────────────────────────────────────────────────────────┘

Features

iOS App

• Microphone recording with real-time transcription
• Speaker diarization (identify who's speaking)
• AI-generated summaries and action items
• Hybrid search (full-text + semantic)
• Offline support with sync

macOS App

• All iOS features plus:
• System audio capture (Zoom/Meet via BlackHole)
• Menu bar quick access
• Global keyboard shortcuts

Backend

• OpenAI gpt-realtime-whisper streaming speech-to-text for live dictation; Deepgram Nova-3 for recording captions
• ElevenLabs Scribe v2 file transcription for full recording processing
• ElevenLabs realtime voice sessions
• OpenAI gpt-5.6-sol (medium reasoning) for recording summaries
• Cerebras gpt-oss-120b for saved-content/digest summaries, entity extraction, and dictation cleanup
• OpenAI Responses API for Companion and embeddings
• PostgreSQL with pgvector for hybrid search
• Redis for agent scheduling and deploy/runtime coordination

Quick Start

Backend Setup

cd backend

# Create virtual environment
python -m venv .venv
source .venv/bin/activate

# Install dependencies
pip install -e .

# Copy environment file and configure
cp .env.example .env
# Edit .env with your API keys

# Start with Docker (recommended)
docker-compose up -d

# Or run locally (requires PostgreSQL with pgvector)
alembic upgrade head
uvicorn app.main:app --reload

# Run remote API smoke checks (default: https://wai.computer)
cd ..
./scripts/api-smoke.sh

# Run continuous QA loops for 8+ hours (default 8h)
# Iterates backend tests + shared tests + remote smoke repeatedly
./scripts/qa-loop.sh

# Example with explicit server-build deploy step after green P0/P1 loops
VPS_USER=your-user DEPLOY_CMD="./scripts/deploy-server.sh" ./scripts/qa-loop.sh

# Recommended detached mode (keeps running in tmux/nohup)
./scripts/qa-loop-start.sh
./scripts/qa-loop-status.sh
./scripts/qa-loop-stop.sh

iOS/macOS Development

# Open the iOS project
cd ios/WaiComputer
open WaiComputer.xcodeproj

# Or macOS project
cd macos/WaiComputer
open WaiComputer.xcodeproj

The apps use the shared WaiComputerKit Swift package for common code.

For a signed direct-download macOS installer, use scripts/build-macos-dmg.sh. The DMG builder generates a Finder-ready install window with drag-to-Applications guidance, signs the Sparkle appcast, and can also compose a custom background via MACOS_DMG_BACKGROUND=/absolute/path/background.png. For a strict production release, add MACOS_RELEASE_STRICT=1 so the build fails unless Gatekeeper, notarization, and Sparkle signing pass. Distribution details, appcast publishing, and Fastlane upload-all usage are documented in docs/macos-distribution.md.

Before running a release smoke on a developer machine, clear old desktop app state with scripts/reset-macos-app-state.sh so Debug/UI-test caches do not contaminate the Release app.

Web Development

cd web
pnpm install
pnpm dev

# Run unit tests (100% coverage gate on API/web client modules)
pnpm test:unit

# Run browser automation (Chromium + Firefox + WebKit)
pnpm test:e2e

Web app default routes:

• /login
• /register
• /dashboard
• /auth/verify?token=...

Environment Variables

# Required
DATABASE_URL=postgresql+asyncpg://postgres:postgres@localhost:5432/waicomputer
JWT_SECRET=your-secure-secret
OPENAI_API_KEY=your-openai-key
CEREBRAS_API_KEY=your-cerebras-key

# Optional - realtime voice sessions
ELEVENLABS_API_KEY=your-elevenlabs-key

# Optional - Email (magic links)
RESEND_API_KEY=your-resend-api-key
EMAIL_FROM=WaiComputer <noreply@mail.waiwai.is>
FRONTEND_URL=https://wai.computer
AUTH_COOKIE_DOMAIN=wai.computer

# Optional - Telegram bot integration
TELEGRAM_BOT_TOKEN=your-telegram-bot-token
TELEGRAM_BOT_USERNAME=waicomputer_bot
TELEGRAM_WEBHOOK_SECRET_TOKEN=your-random-webhook-secret
TELEGRAM_API_ID=your-telegram-api-id
TELEGRAM_API_HASH=your-telegram-api-hash
TELEGRAM_DOWNLOAD_MAX_BYTES=1073741824

API Endpoints

Auth

• POST /api/auth/register - Register new user
• POST /api/auth/login - Login
• POST /api/auth/magic-link - Request magic link
• POST /api/auth/refresh - Refresh token

Recordings

• GET /api/recordings - List recordings
• POST /api/recordings - Create recording
• GET /api/recordings/{id} - Get recording details
• GET /api/recordings/{id}/transcript - Get transcript
• POST /api/recordings/{id}/generate-summary - Generate AI summary

Search

• GET /api/search?q=query - Hybrid search
• GET /api/search/semantic?q=query - Semantic search only

WebSocket

• WS /api/ws/audio - Real-time audio streaming

WebSocket Protocol

// Client → Server (audio)
{ "type": "audio", "data": "<base64>", "timestamp": 123 }

// Server → Client (transcript)
{ "type": "transcript", "text": "...", "speaker": "Speaker 1", "is_final": true }

// Server → Client (status)
{ "type": "status", "status": "ready|processing|error" }

Project Structure

wai-computer/
├── backend/              # Python FastAPI
│   ├── app/
│   │   ├── api/          # Routes
│   │   ├── core/         # ElevenLabs, OpenAI, auth
│   │   ├── models/       # SQLAlchemy models
│   │   └── db/           # Database config
│   └── tests/
├── ios/                  # iOS SwiftUI app
│   └── WaiComputer/
├── macos/                # macOS SwiftUI app
│   └── WaiComputer/
└── shared/               # Shared Swift package
    └── WaiComputerKit/

BlackHole Setup (macOS System Audio)

To capture audio from Zoom/Meet/etc:

1. Install BlackHole 2ch
2. Open Audio MIDI Setup
3. Create Multi-Output Device with your speakers + BlackHole
4. Set Multi-Output as system output
5. Select BlackHole as input in WaiComputer

Tech Stack

• Frontend: SwiftUI (iOS 17+, macOS 14+)
• Backend: Python 3.11+, FastAPI
• Database: PostgreSQL 16 + pgvector
• Transcription: OpenAI gpt-realtime-whisper for dictation live STT, Deepgram nova-3 for recording live STT, ElevenLabs scribe_v2 for full file STT
• AI: OpenAI gpt-5.6-sol with medium reasoning for recording summaries; Cerebras gpt-oss-120b for saved-content/digest summaries, entities, and dictation cleanup; OpenAI Responses API for Companion and embeddings
• Audio Codec: Opus (16kHz, mono)
• Deploy: Cloudflare Pages + Workers

License

MIT