Skip to content

gfx: HiDPI windows, measured system-font text, cursor shapes - #1698

Open
godofecht wants to merge 6 commits into
mainfrom
feat/gfx-hidpi
Open

godofecht wants to merge 6 commits into
mainfrom
feat/gfx-hidpi

Conversation

@godofecht

@godofecht godofecht commented Nov 13, 2026 •

Copy link
Copy Markdown
Contributor

Intent

Linked issue or roadmap item: none. Found while building a gfx app on a Retina MacBook.

gfx_open(w, h) makes a w x h pixel framebuffer in a w x h point window. On a Retina display macOS scales it 2x with interpolation, so every line and every glyph from gfx_text is blurred. There was no way to ask for device pixels.

This adds four calls, on every backend:

  • gfx_open_hidpi(w, h, title, flags): a window of w x h points with a framebuffer in device pixels (2w x 2h on Retina), drawn 1:1 with interpolation off. Mouse and text coordinates use the same pixels. GFX_RESIZABLE makes the window resizable.
  • gfx_metrics(g): framebuffer width and height, pixels per point, and a generation that changes when the framebuffer is reallocated (resize, or a move to a display with another scale).
  • gfx_text_ex(g, x, baseline, text, size, rgb, style): system-font text on a baseline, with weight, tabular digits, monospace and tracking, returning the advance width. gfx_text_width measures without drawing, so labels can be right-aligned and centred exactly and paragraphs wrapped.
  • gfx_cursor(g, kind): arrow, hand, crosshair, resize and grab cursors.

Performance at 2x, where a full framebuffer is 4.7 million pixels:

  • gfx_text_ex caches shaped lines by text, size, style and colour, so labels redrawn every frame are not shaped again. In the PSOLA app text went from about 4 ms to 0.6 ms a frame.
  • gfx_blit_rgb converts 16 pixels per step with NEON where available.
  • gfx_present invalidates only the rectangle touched since the last present.
  • gfx_blit_rgb_region(g, src, src_w, x, y, w, h) copies one rectangle of a full-size canvas, so a program can send only what changed. The PSOLA app redraws by zone, and a recorded 40-frame session (hover, drag, stage changes, playback) is byte-identical to full redraws.

The macOS backend now calls [NSApp finishLaunching] once before opening a window. Events are pumped by hand, so nothing called it before, and accessibility clients saw no windows: System Events could not resize one. After the change, a resize from System Events reflows a gfx_open_hidpi window at the new size.

gfx_open and gfx_text are unchanged, so existing programs behave as before.

The recorder implements the new calls too. FLOW_GFX_RECORD_SCALE=2 records at Retina density, and text_ex draws the 5x7 font on the baseline with an exact width, so a layout that measures text records the same way it runs. The SDL and wasm backends get link-compatible fallbacks at scale 1. Neither draws text today, so text_ex there returns an estimated width and draws nothing.

Overlap check

  • I searched open pull requests and recent commits for overlapping work.
  • This does not duplicate another active implementation, or the relationship is explained below.

Overlap / dependency notes: none open. It builds on the existing mouse, text and blit ABI (b77dc30c4, c86575385, d79274b83).

Evidence

Baseline before this change: a 1380 x 860 window on a 3024 x 1964 display renders 1380 x 860 pixels, upscaled.

Targeted proof that exercises the changed behaviour:

  • tests/scripts/gfx_recorder.flow gains record_hidpi. It opens a 160 x 120 point window at FLOW_GFX_RECORD_SCALE=2, checks gfx_metrics (320 x 240, scale 2), checks gfx_text_ex and gfx_text_width widths, and reads glyph pixels back from the PPM to check the text sits on its baseline. It also blits a 10 x 10 region of an all-red canvas and checks the pixels on each side of it stay black. ./flow tool tests/scripts/gfx_recorder.flow: 2 passed.
  • On macOS, an app built on these calls (a PSOLA visualiser) captures at 2760 x 1784 for a 1380 x 860 point window, title bar included, with sharp CoreText and anti-aliased curves at device resolution. Hover, drag, scroll, resize and cursor changes were exercised in the window.
  • clang -fsyntax-only passes for gfx_macos.m, gfx_linux.c with SDL2, and gfx_linux.c and gfx_windows.c under -DFLOW_GFX_STUB.

Why that proof would detect the old failure or regression: without the recorder's scale, the PPM header is 160 x 120, and the metrics check exits 11. Without baseline placement, the lit and dark pixel checks around row 24 fail.

Repository gate(s) run: ./flow tool compiler/scripts/parity_mlir.flow. The four failures it reports (complex.flow, complex_real_scale, complex_types_arith, test_complex_scaling) also fail on a clean origin/main worktree. Re-recording test_complex_scaling there changes its digest.

Generated/bootstrap state, if any: docs/library/stdlib-api.md regenerated with ./flow tool gen_stdlib_docs. The MLIR digests of the 144 corpus programs that import gfx.flow were re-recorded with --record-case, two phase_portrait DSL fixtures among them. Nothing else in the corpus moved.

Risk

Compatibility, performance, security, lifetime, backend, or release implications: additive ABI. The HiDPI framebuffer is four times the pixels of the point-sized one. It is allocated only by gfx_open_hidpi. In HiDPI mode drawRect wraps the buffer in a CGImage without copying it. CTFont objects are cached per (size, style), up to 16 per window, and released at shutdown.

  • No unresolved language/API design decision is being implemented implicitly.
  • No unrelated generated artifacts or local state are included.
  • Docs changed when user-visible behaviour or syntax changed.
  • docs/project/Questions.md updated if this needs a language-design decision.

Compatibility

  • No user-visible compatibility change.
  • Compatibility change is documented in docs/project/CHANGELOG.md.

Security & AI Safety

  • Security-critical changes (compiler, runtime, tools/agent_sandbox, workflows) have been reviewed by a human CODEOWNER. (Runtime gfx backends: needs a CODEOWNER review.)
  • No unapproved changes to release policy, branch protection, or sandbox boundaries.
  • If this PR was AI-authored, the AI's claims have been independently verified against observable evidence.

Agent handoff

Agent/provider/session, when applicable: agent-assisted.

Any remaining blocker is recorded in GitHub rather than only in local/session state: none.

gfx_open made a framebuffer in points, which a Retina display scaled 2x
and blurred, text included. gfx_open_hidpi makes a window of w x h points
whose framebuffer is in device pixels, drawn 1:1 with no resampling.
Mouse and text coordinates follow the framebuffer. The window can be
resizable; gfx_metrics reports the size, the scale and a generation that
changes when the framebuffer is reallocated.

gfx_text_ex draws system-font text on a baseline with a weight, tabular
or monospaced digits and tracking, and returns its width; gfx_text_width
measures without drawing, so layouts can align text exactly. gfx_cursor
sets the cursor shape.

The recorder implements the same calls: FLOW_GFX_RECORD_SCALE=2 records
at Retina density, and text_ex draws the bitmap font on its baseline with
an exact width. SDL and wasm get link-compatible fallbacks at scale 1.
144 programs changed digest because gfx.flow gained externs and
functions, including the two phase_portrait fixtures, which lower to gfx
calls. Recorded with parity_mlir.flow --record-case. The four
complex-number failures the parity gate still reports also fail on
origin/main.
Events are pumped by hand, so [NSApp run] never calls finishLaunching.
Without it, accessibility clients see no windows: System Events could
not resize a Flow window. With it, resizing a gfx_open_hidpi window from
System Events works and the framebuffer follows.
A stdlib addition after the Python recording (ACCEPTED.md class 9).
Regenerated with tests/tools/lsp/run.flow --update; 9 passed.
At 2x a full framebuffer is 4.7 million pixels, and three costs showed up
in profiles of a real app:

- text_ex shaped every label again on every frame. Shaped CTLines are now
  cached by text, size, style and colour (2048 direct-mapped slots), and
  the bitmap context is set up once. Text went from about 4 ms to 0.6 ms
  a frame in the PSOLA app.
- blit_rgb converted RGB to RGBA a byte at a time. It now does 16 pixels
  per step with NEON where available.
- every present redrew the whole view. The backend now tracks the
  rectangle touched since the last present and invalidates only that.

gfx_blit_rgb_region copies one rectangle of a full-size canvas, so a
program can send only the regions that changed. The recorder, SDL and
wasm backends implement it too, and gfx_recorder.flow checks that only
the rectangle is copied.
Same 144 programs as before, recorded with --record-case. The parity
gate reports only the four complex-number failures that also fail on
origin/main.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant