Skip to content

Add a native macOS client - #5

Draft
dhh wants to merge 7 commits into
omacom:mainfrom
dhh:macos-client
Draft

dhh wants to merge 7 commits into
omacom:mainfrom
dhh:macos-client

Conversation

@dhh

@dhh dhh commented Sep 21, 2026

Copy link
Copy Markdown
Contributor

A native macOS client for gliff: Gliff.app connects to a Hyprland machine running gliff-server over ssh, like the GTK client does. The plan is in plans/mac-app.md.

What changes

  • gliff-client (new crate): the GTK client's session loop, moved out of crates/gliff/src/net.rs so both clients share it. The platform supplies the decoder through a Sink trait, and a session can be stopped from another thread. It has tests for one ack per frame (even when decoding fails), keyframe requests, a stream change mid-session, and stop.
  • gliff-ffi (new crate): that session as a C static library with a cbindgen header, for the Swift app.
  • macos/: a SwiftPM package.
    • GliffVideo: VideoToolbox decodes both H.264 streams in parallel into IOSurface-backed NV12, and a Metal port of recombine.comp rebuilds 4:4:4 without a CPU copy.
    • GliffInput: macOS key codes to evdev, including the Apple ISO key swap, plus keyboard-layout names.
    • The AppKit app: a CAMetalLayer view, pointer and scroll, clipboard, remote cursor, resize, reconnect, Shift+Esc to hand the keyboard back, and an opt-in event tap for Cmd-Tab and other system shortcuts.
  • Protocol version 3: Hello.keymap may carry rmlvo:layout=..;variant=.. names, which the server compiles, because a Mac cannot produce xkb keymap text. A v2 client and a v3 server refuse each other.
  • gliff-probe roundtrip --dump: writes the coded streams and the Vulkan decoder's output as fixtures for other decoders.
  • CI: a macos-15 job runs the portable Rust tests and the Swift tests and builds the app. release now waits for it and publishes the zipped app with the Linux binaries.
  • scripts/e2e.sh: now also runs the real GTK client against the nested server.

Testing

  • Linux: cargo test --workspace, clippy with -D warnings, cargo fmt --check, and scripts/e2e.sh all pass.
  • Mac (M4, macOS 15.5): the Rust tests and swift test pass.
  • Decoding: fixtures from the AMD Vulkan encoder decode with VideoToolbox and recombine in Metal to within ±1 of the Vulkan output, on every sample of every frame. This covers cropped sizes, a keyframe mid-stream and Single420. At 1080p, decode plus recombine takes 5.4 ms; at 4K, 13.7 ms.
  • Live, M4 Mac mini to an AMD Hyprland machine, over ssh and over TCP:
    • 60 fps Dual420 at 3.4 ms decode
    • mirror and headless modes
    • typed text arriving in a terminal on the remote
    • the clipboard in both directions

Not yet tested by a person at the Mac: physical key and mouse events, and the Accessibility-gated shortcut tap. The CI workflow has not run yet either.

🤖 Generated with Claude Code

dhh and others added 7 commits September 21, 2026 06:02
plans/mac-app.md lays out a native Mac client: the Rust session core
shared through a C ABI, VideoToolbox decode, a Metal port of the
recombine shader and an AppKit front end. scripts/mac-build.sh syncs
the working tree to a Mac (nyc-m4 by default) and builds and tests
there.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The first piece of the macOS client, from the plan's decode spike.
macos/ is a Swift package whose GliffVideo target splits Annex B access
units for VideoToolbox, decodes each stream into IOSurface-backed NV12,
and recombines main and aux into BGRA with a Metal port of
recombine.comp.

gliff-probe roundtrip --dump writes the coded streams and the Vulkan
decoder's output as fixtures. Against those, the Mac output matches to
within one step on every sample, at cropped sizes and across a forced
keyframe; 4K Dual420 decodes and recombines in 13.7 ms on an M4.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The GTK client's network worker becomes gliff-client: handshake, ack
pacing, clipboard and stats, with the decoder supplied through a Sink
trait so the macOS client can reuse it. The keymap and capabilities are
passed in rather than read from Hyprland, the session can be stopped
from another thread, and messages are read on their own task so the loop
can also forward ssh's stderr, which spawn_ssh can now capture.

Every video frame is still acked whether or not it decodes, now with a
test. The letterbox maths and cursor-visibility check move there too,
and e2e.sh runs the real GTK client against the nested server.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A client that cannot produce xkb keymap text, such as the macOS client,
can now send rmlvo:layout=..;variant=..;options=.. in Hello.keymap and
the server compiles the keymap itself. Protocol version 3, so an older
server refuses such a client clearly rather than failing to compile the
names as keymap text.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Gliff.app is an AppKit front end over the shared gliff-client session,
reached through a new gliff-ffi static library with a cbindgen header:
the session runs on its own thread and calls back into Swift to decode
each frame with VideoToolbox and the Metal recombine, which a
CAMetalLayer view then draws letterboxed at one stream pixel per device
pixel. Keys map from macOS key codes to evdev (with the Apple ISO swap),
the keyboard layout goes to the server as rmlvo names, and pointer,
scroll, clipboard, remote cursor, resize, reconnect and release on focus
loss follow the GTK client. While the screen has the keyboard, Cmd
chords go to the remote; an opt-in event tap also captures system
shortcuts, and Shift+Esc hands the keyboard back.

macos/build.sh builds and signs dist/Gliff.app on a Mac;
scripts/mac-build.sh now runs it by default, with Homebrew's rustup
because Homebrew's rust ships a standard library built for macOS 15.
Test hooks (--snapshot, --view-snapshot, --type, --exit-after) let a
remote run check decoding, drawing and input.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
README covers building and using Gliff.app, and the architecture notes
describe how it decodes and draws. A macos-15 CI job runs the portable
crates' tests and the Swift tests and builds the app; the release job
now waits for it and publishes the zipped app with the Linux binaries,
so the two jobs never race over the release. The plan records what was
verified live against a Hyprland machine and what still needs a person
at the Mac.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The app runs ssh without a terminal, so an unknown host key, a refused
key or a missing gliff-server all ended as a bare connection error, and
the app kept retrying. The status line now says what to fix, the full
message is in its tooltip, and setup problems are not retried.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@dhh
dhh marked this pull request as draft September 21, 2026 11:17
@tornikegomareli

Copy link
Copy Markdown
Contributor

I run gliff from a Mac to an Omarchy machine (RTX 4080) and want to help land the native client. #5 is now 125 commits behind main, and #13 changed the protocol under it. Would a PR that ports #5 onto main be welcome, with layout names in the optional Hello field that #13 describes? Or do you plan to update it yourselves?

@teomurgi

Copy link
Copy Markdown
Contributor

@dhh i just noticed this PR now... in the meantime i did this, it is working: https://github.com/teomurgi/mac-gliff

@LaurentFough

Copy link
Copy Markdown

So many of us... built my own gliff-mac (because I needed it).
Came back to find such progress official and varied versions for macOS.
Able to test and submit fixes as needed as well.

LaurentFough pushed a commit to LaurentFough/gliff that referenced this pull request Oct 28, 2026
Replace the per-commit list with what the fork adds: merged upstream PRs
(omacom#98, omacom#80) and the ones not merged (omacom#5, omacom#26), macOS work, and build and
check changes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G4VLoM6hjvYsFPeoe3rMWg
LaurentFough pushed a commit to LaurentFough/gliff that referenced this pull request Oct 29, 2026
Ports the VideoToolbox + Metal video path from upstream PR omacom#5 into the
viewer: hardware H.264 decode, Metal AVC444 recombine, CAMetalLayer
presentation, 4:4:4 preferred, with the OpenH264 path as fallback and
`--video cpu` / `decode = "cpu"` to force it. Adds the GPU video check
and benchmark (`make video-check`), which passed at 1080p and 4K in 4:2:0
and 4:4:4 (4K 4:4:4: 6.9 ms GPU vs 18.8 ms CPU per frame).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G4VLoM6hjvYsFPeoe3rMWg
LaurentFough pushed a commit to LaurentFough/gliff that referenced this pull request Oct 29, 2026
Brings GPU video, the Mac settings and ToDone.md into the PR omacom#80 test branch. The Mac client now counts AudioFrame payload bytes, so a server that sends sound to a Mac anyway cannot desynchronise the stream. README notes omacom#5 as ported and lists make video-check.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G4VLoM6hjvYsFPeoe3rMWg
@LaurentFough

Copy link
Copy Markdown

So many of us... built my own gliff-mac (because I needed it). Came back to find such progress official and varied versions for macOS. Able to test and submit fixes as needed as well.

My fork has merged a couple of PRs, and added some small quality of life features.

macOS fork of gliff: GPU video, settings and sound

Fork: https://github.com/LaurentFough/gliff (branch acrogenesis/macos-client-host)

Built on acrogenesis' macOS client and host (#47), with these additions:

Upstream PRs included

New in this fork

  • GPU video on the Mac viewer: VideoToolbox hardware H.264 decode, Metal 4:4:4 recombine, CAMetalLayer presentation. It asks for 4:4:4 first and falls back to OpenH264 (--video cpu forces it). 4K 4:4:4 decodes in about 6.9 ms per frame on the GPU vs 18.8 ms on the CPU.
  • Sound on the Mac viewer: plays Linux hosts' sound through Play the remote machine's sound #80's gliff-audio (CoreAudio). Machine ▸ Play Sound shares the Linux client's mute switch; --no-audio starts muted.
  • Host speaker mute: gliff-server --mute-local-audio mutes the Linux host's speakers while the client plays, and restores them on stop, disconnect or crash.
  • Shared settings: a gliff-settings crate; both clients read and write ~/.config/gliff/config.toml. On the Mac: a settings window (⌘,) and a Machine menu.
  • Local signing: no paid Apple Developer account needed (a self-signed gliff-local identity), so privacy permissions survive rebuilds.
  • Makefile: one front end for bin/ and scripts/. make macos builds, signs, installs and verifies; make install on Arch; also make check and make video-check.
  • Checks: make video-check checks GPU output against the CPU decoder and benchmarks both; the ssh login-shell test no longer depends on your shell setup.
  • Fixes: Mac install and signing (plutil, a custom cargo target-dir); the client ignores stray sound packets from a host.

Status

  • Tested: Linux host → Mac client (GPU video, sound, host mute).
  • Not yet: sound from a Mac host. Linux → Linux sound hasn't been retested on the merged branch.

Feedback and issues welcome on the fork.

Then... will test a bit more, then submit as PRs to omacom/gliff.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants