Skip to content

FMV: bind HD replacement to guest movie timeline - #761

Merged
mstan merged 14 commits into
RetroPortingToolKit:masterfrom
tetrisgm:fix/fmv-guest-timeline-upstream
Oct 27, 2026
Merged

mstan merged 14 commits into
RetroPortingToolKit:masterfrom
tetrisgm:fix/fmv-guest-timeline-upstream

Conversation

@tetrisgm

@tetrisgm tetrisgm commented Oct 24, 2026 •

Copy link
Copy Markdown
Member

HD FMV replacement in #718 could select the wrong replacement after a seek or state load, and advanced subframes from host time even while the guest was paused. This continuation keeps the original contributor's HD movie feature while binding lookup and subframe timing to guest events.

The renderer now receives an immutable guest timeline snapshot. FMVHD003 packs identify a movie by its SetLoc range and each source frame by the delivered CD sector interval, with the native frame signature as a second check. Duplicate-looking frames are distinct. Guest cycles drive subframe phase; MDEC decode events, seeks, state loads, rollback, and movie exit reset it. An unmatched or failed lookup displays the native PS1 frame. Signature-only FMVHD002 packs require explicit PSX_FMV_HD_LEGACY=1 because they cannot resolve seek and repeated-frame ambiguity.

This supersedes the unresolved timeline design in #718. The original HD replacement implementation is by @yacineolivier-maker; this branch includes that work and the reviewer-requested safety changes. Please review this PR alongside #718. It does not modify or close the contributor's branch.

Verification:

  • Windows and macOS: full psx-runtime builds and focused gpu_fmv_hd_test passed. The synthetic test covers malformed packs, worker/decode lifecycle, pause, guest-cycle subframe advancement, duplicate frames, seek generation, movie exit, and load/rollback epochs.
  • R4 local-disc replay with an 80-frame local synthetic FMVHD003 pack: expected HD replacements at SetLoc 192822 and delivered LBAs 194127–194324, zero dispatch misses. First replacement at guest frame 1475/cycle 832608000, delivered LBA 194129, MDEC ordinal 260.
  • Host pause for 0.5 s held guest cycle, HD count, and audio counters. Save/load advanced execution epoch 1→2 and playback resumed.
  • Across one unpaused movie interval, guest time advanced 366.67 ms, SPU output advanced 16,170 samples/366.67 ms, CD input 16,464 samples/373.33 ms, and host output 15,876 samples/360.00 ms (44.1 kHz). This measures cadence within 6.67 ms, not sound-to-image content alignment.
  • TCP Start skipped the movie. At guest frame 1494, HD source tracking reset to -1; HD frame count/lookups stopped at 17/530 through frame 1514. CD input stopped while queued host audio drained. Zero dispatch misses.

R4's intro has no movie seek control. Local Rewind was off in this test configuration and netplay rollback inactive; the focused test exercises seek generation and rollback epoch changes. The replacement images are conspicuous synthetic frames, so content-level A/V lip-sync is not measured. No retail movie assets or synthetic pack are included.

@tetrisgm
tetrisgm marked this pull request as ready for review October 24, 2026 22:13
yacineolivier-maker and others added 13 commits October 26, 2026 16:04
An optional pack replaces a title's MDEC movies with high-definition frames
while the game keeps its own audio, timing and skip control:
- fmv_hd/<movie>/index.bin: header, then a luminance signature per source
  frame; fmv_hd/<movie>/frames/NNNNNN.jpg: `sub` HD frames per source frame.
- Every presented 24-bit frame is matched by signature (mean absolute
  difference), first around the last recognised frame, then over the whole
  pack when tracking is lost; a near global match is also tried a few lines
  up or down (each movie has its own vertical placement on screen) and the
  offset kept. Worker threads decode the upcoming frames ahead into a small
  RGBA ring.
- Format FMVHD002 records the native display the movie is shown in and its
  position, so the HD frame lands exactly where the original was.
- The last HD frame is held briefly when the next one is not ready or not
  recognised (no flicker back to the 4:3 original mid-movie), and is
  uploaded to the GPU only when it changes.
- Debug: TCP {"cmd":"fmv_hd"} reports the pack, the matches and lost frames.
No pack: no effect. Pack frames are the user's own upscale of their own disc;
nothing is distributed with the runtime.

Used for R4's intro and ending movies (SeedVR2 upscales, 25 fps sources).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Rkyma8rbyPiccs9n2bkyG6
Reject invalid index products and unsafe paths, cap encoded/decoded residency, recover allocation and partial worker-start failures, release GL movie textures with their context, and serialize diagnostics through the presenter. Real SDL decode/restart/failure test passes and SDL2 C seam compiles warning-free. Guest/present timestamp ownership remains a separate design gate. Tracking: beads-eio.3.388.
Replace host-time phase and hold with captured guest cycles, reset on state timeline changes and non-movie scanout, and fall back to native frames on misses.

Co-Authored-By: GPT-6.1-Sol <noreply@openai.com>
Co-Authored-By: GPT-6.1-Sol <noreply@openai.com>
FMVHD003 carries movie and per-frame LBA ranges; legacy signature-only packs require explicit opt-in. Capture CD stream generation and MDEC ordinal with each present so seeks and repeated images cannot reuse an old interpolation phase.

Co-Authored-By: GPT-6.1-Sol <noreply@openai.com>
Co-Authored-By: GPT-6.1-Sol <noreply@openai.com>
Co-Authored-By: GPT-6.1-Sol <noreply@openai.com>
Co-Authored-By: GPT-6.1-Sol <noreply@openai.com>
Co-Authored-By: GPT-6 <noreply@openai.com>
Co-Authored-By: GPT-6 <noreply@openai.com>
@mstan
mstan force-pushed the fix/fmv-guest-timeline-upstream branch from d16a937 to 9bd10cf Compare October 26, 2026 23:11

@mstan mstan left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rebased onto frozen current master1a3fc0db3, preserving the existing perf/hd_uploads commands and both postfx/FMV fixture adapters. Fixed two demonstrable findings and pushed with an exact d16a937 lease.

  1. Actual CD snapshot A/B/restore reproduced stale FMV movie identity: the canonical SetLoc restored150, while the added host copy still returned225. Removed that duplicate field; the FMV getter now uses the existing reset/serialized SetLoc. The real CD FIFO and resident suites fail before repair and pass afterward on Linux and Windows, including restored identity and subsequent controller init. No serializer format or guest timing changes.
  2. An overlong pack root could be truncated to another valid pack path. A real Linux pack-path regression failed before repair. Root/movie paths now reject truncation before loading and retain native fallback.

The full synthetic FMV bounds/worker lifecycle/guest-timeline suite passes on Windows and Linux after the repairs, with no new compile warnings in those builds. This is not a new retail-content A/V or actual network rollback claim. The compatible combined regression review is still in progress.

@mstan mstan left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rebased onto current master and pushed fixes through 9bd10cf. Actual CD controller restore tests reproduced stale movie identity: restoring SetLoc 150 after 225 left the duplicate field at 225. The getter now uses canonical serialized/reset SetLoc state. A real overlong-root test also reproduced truncation into a different valid pack path; truncated root/movie paths are now refused.

CD FIFO/resident and synthetic FMV pack/timeline/lifecycle tests pass on Windows and Linux, with red results retained. The combined Release runtime at bb0d6de builds and passes 16/16 focused tests. The immutable guest timeline and explicit legacy-format limits are clear. Author title replay evidence is retained with its stated seek/rollback and content A/V limits. Ready from this review; this supersedes #718.

@mstan
mstan merged commit f7070e1 into RetroPortingToolKit:master Oct 27, 2026
1 check passed
tetrisgm pushed a commit to tetrisgm/psxrecomp that referenced this pull request Oct 27, 2026
…t-timeline-upstream

FMV: bind HD replacement to guest movie timeline
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants