Skip to content

Live: a member's page watches this Mac now, over WebRTC - #2

Merged
jonastemplestein merged 1 commit into
mainfrom
live-view
Nov 22, 2026
Merged

jonastemplestein merged 1 commit into
mainfrom
live-view

Conversation

@jonastemplestein

Copy link
Copy Markdown
Contributor

What. The Mac app is now a WebRTC peer. When a member's page on record.<domain> asks
(device/live-signal { want } on /devices/<name>), the Mac offers each display it may show,
one camera (1280×720 at most) and the microphone. The tracks are send only, H.264 in hardware,
and the picture is shown at once with the sound in its own stream. The mind carries only the
signalling. The pictures and the sound go peer to peer.

  • Sources/Live holds everything that touches WebRTC (LiveKit's build, a binary target fetched
    from this repository's release, carried in Contents/Frameworks).
  • Record gets one seam: the sources hand each frame and sound buffer to sinks (CaptureSink).
    The record's own tracks are one sink, and live is a second. The capture runs for the union of
    what the record and live want. A watched screen captures 30 frames a second, and the record
    keeps its 10 by time.
  • The signalling's rules are a pure state machine (LiveWatchers, VoiceMeetCore).
  • bridge/device-live.mjs carries the signals, asks the facet record for ICE servers (STUN
    when it has none), and appends device/live-watch-started / -ended.
  • The owner's existing switches decide through the same gate as every use: Off is not offered,
    Ask asks once per watch, and Shared is sent. Turning a switch Off ends a watch. Activity shows
    "Watched live by " while it runs. The panel lists the watchers with End.
    live.state() and live.end() are on the endpoint. Info.plist gets
    NSLocalNetworkUsageDescription.

Why. The record's files are 30 to 60 seconds behind; a live view needs to be a few milliseconds behind, and it must not change the record. The same capture now feeds both.

Proof (one Mac, a headless watcher, signalling through the mind on
/devices/record-test-rtc-app, with the app's own libraries and bridge via live-run):

  • Drawn to shown: 6 / 7 ms (p50 / p95) with live alone, and 10 / 26 ms with the record running.
  • The first picture after 0.2–0.4 s. Sound +86 to +103 ms after the picture, with a 30–35 ms
    buffer.
  • A second watcher, and a watcher leaving.
  • CPU: 4.1 % for the record alone, 6.0 % for live alone, 7.9 % for both.
  • The record's files: old and new recorders run side by side give equal frame and packet counts
    and identical audio levels, the same as two runs of the old code. With a watcher at 30, the
    record kept the same 220/230 frames per slot.
  • Real screen (3024×1964 at 30), camera and microphone through the app's permissions: all three
    tracks arrived (numbers only).
  • swift test: 86 tests, 16 of them new.

Risk map.

  • The record's files: low. The seam is unchanged with no watcher. The comparison and the
    tests pass.
  • The record with a watcher: low to medium. A frame gate by time, measured.
  • The bundle: medium. A new framework in a hand-built bundle. --build-only plus render is
    proven; the first install is the real test.
  • Privacy: medium. The gate is the existing one, a watch is visible (act, panel, kept
    events), and Off ends a watch. Unproven in the installed app: see the report's checklist.
  • Local Network prompt / TURN / two networks: unproven.
  • Idle cost: a WebRTC factory is made at the app's start.

Session: claude b0eec5e0-9c0a-43e6-bf1b-01b7e83c523c on jonas-macbook-pro

🤖 Generated with Claude Code

The Mac app is a WebRTC peer. When a member's page asks (`device/live-signal { want }` on
/devices/<name>), the Mac offers each display it may show, one camera and the microphone, send
only, H.264 in hardware. The mind carries only the signalling; pictures and sound go peer to
peer. Measured with the app's own libraries: a picture is shown about 10 ms after it was drawn.

- `Sources/Live` holds everything that touches WebRTC (LiveKit's build, a binary target from
  this repository's release, carried in Contents/Frameworks).
- `Record` hands each frame and sound buffer to sinks (`CaptureSink`): the record's tracks are
  one, live is a second. A watched screen captures 30 frames a second; the record keeps 10.
- The owner's switches decide through the same gate as every use; a watch shows in Activity and
  in the panel, and `live.state()` / `live.end()` are on the endpoint.

Session: claude b0eec5e0-9c0a-43e6-bf1b-01b7e83c523c on jonas-macbook-pro
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
@jonastemplestein
jonastemplestein merged commit e5cf768 into main Nov 22, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant