Skip to content

End capture before removing the headless output on every exit, and cap headless resizes - #64

Merged
dhh merged 4 commits into
omacom:mainfrom
TechLuddite:fix/headless-teardown-and-resize-cap
Oct 24, 2026
Merged

dhh merged 4 commits into
omacom:mainfrom
TechLuddite:fix/headless-teardown-and-resize-cap

Conversation

@TechLuddite

@TechLuddite TechLuddite commented Oct 16, 2026 •

Copy link
Copy Markdown
Contributor

Headless sessions had two gaps.

  1. The ordered teardown (release held keys, end the capture thread, end the input thread, then remove the headless output) ran only when the session loop ended normally. An early ? return or an unwinding panic dropped Session in field order instead. Such a return comes from a second Hello, a writer error when the ssh link drops, or an encoder failure. That order removed the output while the cursor capture session was still alive, which docs/hardware-quirks.md records as aborting Hyprland 0.56.2. The output and both threads now live in ServedOutput, whose Drop ends the threads before the output on every exit. The session's own Drop releases held keys first.
  2. Resize was bounded only by the size the client declared in its own Hello, so a client could request a 16384x16384 output and a four-buffer capture ring of that size. Resize is now capped at 4096 a side and 3840x2160 worth of pixels, keeping the aspect ratio. The capture thread refuses a headless ring past the same limit, since it follows size changes on its own. Mirrored outputs are unchanged.

Validation:

  • Unit tests cover the teardown order through ServedOutput (normal drop and panic), the resize cap and its aspect ratio, and the capture limit. Each fails with the code it covers removed.
  • scripts/e2e.sh (nested Hyprland, AMD Radeon 840M): the headless server and client steps, the probe, pipeline and CPU tier steps pass. The run then stopped at step 6, the e2e race that Wait for the client's clipboard item in e2e, not a MIME type #65 fixes.
  • scripts/check.sh passes.
  • Not reproduced: the compositor abort itself.

This conflicts in crates/gliff-server/src/session.rs with #66 and #73, and with #47, and merges cleanly with #57. I am happy to rebase on whichever lands first.

Opus 5.5 high, on behalf of TechLuddite.
Implemented with Claude Opus 5.5 at high effort and reviewed by GPT-6.1 (Codex) at high reasoning effort over three passes, then re-reviewed on 2026-10-19 by Claude, GPT-6.1 and Grok 4.7 at medium effort.

The screen session cleared its dmabuf and wl_shm format lists after every
constraints done. When the first GPU encode failed, UseShm dropped GBM and
reallocated the ring against those empty lists, so the fallback failed with
"no usable wl_shm format offered (got [])" and the session ended instead of
capturing into wl_shm.

Constraint events now fill a pending set, and done promotes it to a complete
set and starts the next batch empty, since the compositor sends the full set
in every batch. All allocation reads only the complete set, so a
reallocation between batches sees the last full set and never pairs its
formats with the size of a batch still in flight. The cursor session goes
through the same struct and now also sizes its buffer from the complete set.
--capture dmabuf still refuses the fallback.

The per-event constraint handling moves into Constraints::apply, and the
ring format choice into dmabuf_format and shm_choice, so the unit tests
drive the same code as the session handler and allocate_ring.
The ordered teardown at the end of the session loop (release held keys,
end the capture thread, end the input thread, then remove the headless
output) ran only when the loop ended normally. A `?` return from a
client message (a second Hello), a writer error (the ssh link dropping)
or the encoder, or a panic, dropped Session in field order instead.
`output` is declared before `input` and `capturer`, so the headless
output was removed while the cursor capture session was still alive,
which aborts Hyprland 0.56 (docs/hardware-quirks.md).

Hold the output with the capture and input threads in ServedOutput, a
small generic type whose Drop ends both threads before the output drops.
Session derefs through it to the output, so the call sites read as
before, and its own Drop releases held keys first, which runs before
any field drops. The normal exit now just drops the session. A test
drops a ServedOutput of stand-ins normally and under a panic and checks
the order.
Resize clamped a headless output only to max_width and max_height from
the client's own Hello, so a client could ask Hyprland for a 16384x16384
output and the capture ring would then allocate four buffers that size.
The 1920x1080 clamp in setup_output applies only to the first mode.

Cap a resize at 4096 a side and 3840x2160 worth of pixels before
clean_mode, scaling both sides by one factor so the aspect ratio holds.
The gliff client never asks for more than 3840x2160, and a larger output
would only be scaled down to the encoder maximum for the stream.

The capture thread follows size changes on its own, so CaptureConfig
gains max_size, a SizeLimit with a side and a pixel count, and
allocate_ring refuses a ring past it with an error. The server sets it,
from the same constant as the resize cap, for headless outputs only: a
mirrored monitor is the size it is. The resize size and the capture
config are built in headless_target and capture_config, which resize
and start_capture call and the tests exercise.
@greptile-apps

greptile-apps Bot commented Oct 22, 2026 •

Copy link
Copy Markdown

RetriggerView in GreptileConfidence Score: 5/5

[Medium impact] The PR appears safe to merge; no actionable issues were found.

Summary

The PR makes session cleanup run on normal exits, errors, and unwinding panics.

  • Session exits release keys and stop both threads before removing the output.
  • Headless outputs and their capture rings stay within a fixed size limit.
  • Capture rings use complete compositor size and format updates.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  A["Session exits"] --> B["Session::drop sends ReleaseAll"]
  B --> C["ServedOutput::drop"]
  C --> D["Capturer stops and joins"]
  D --> E["Input releases keys, stops, and joins"]
  E --> F["SessionOutput removes headless output"]
Loading

Reviews (1) · Last reviewed commit: "Integrate capture constraints with headl..." · Reviewed by Greptile

@dhh
dhh merged commit 44a3f01 into omacom:main Oct 24, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants