Steadier captions, on-target loudness, click-free tighten cuts (with benchmarks)

[JeremySNR]

What does this change?

Three export-quality fixes found while comparing Cutawan's pipeline with SupoClip's. Each has a new offline benchmark that renders through the real code path and shows numbers before and after. No API keys are needed and nothing is assumed.

1. Karaoke captions hold still (captions.ts). The export popped the active word with \fscx109. In libass that changes the word's advance width, so the centred line re-flows and every other word slides sideways on each word change. The word also settled at 109%, while the preview (a CSS transform from 0.94 to 1) settles at 100%. Each word event is now two layers built from the same text: a base layer with the active word transparent, and a pop layer that shows only the active word. Scaling a single word re-centres the line by half its growth, so that word's own centre doesn't move. SupoClip avoids per-word scaling for the same reason.
2. Loudness lands on target (loudness.ts, render.ts).
   • Mono sources were measured before being spread to stereo (3 dB down per channel). Ordinary mono speech therefore went down the limiter path and was limited about 3 dB too hard. Measurement and normalisation now run on the stereo mix the export actually carries.
   • When a clip still needs limiting, the limited result is re-measured (audio only, at most two passes) and the gain into the same limiter is raised to reach the target.
3. Tighten cuts no longer click (render.ts). Audio pieces were concatenated at arbitrary samples, leaving a waveform step under any music, game sound or room tone. Each internal join now fades out and back in over 5 ms. No durations change, so picture and captions stay in sync. SupoClip crossfades its joins; a duration-preserving crossfade was measured here too and was worse (see below).

Why?

The SupoClip review: on most axes Cutawan's pipeline is already more thorough. Examples:

• Sentence-level transcript and boundary snapping, versus SupoClip's fixed line labels.
• Clip-ending and opening review passes.
• LR-ASD active-speaker detection, versus SupoClip's dominant face; its own test reports say production falls back to Haar.
• Pause- and sentence-aware caption grouping, versus fixed word counts.
• Two-pass loudness, NVENC and VideoToolbox, versus single-pass dynamic loudnorm and CPU-only encoding.

SupoClip has no quality evaluation at all. Its test reports run one 19-second fixture.

The places where SupoClip made a better call were the caption pop and the crossfaded joins. These benchmarks confirmed both were real problems in Cutawan. The loudness issue turned up while measuring.

Benchmark	Before	After
bench-caption-stability (12 styles): other words' drift, settled	17–26 px	0 px
active word settled size vs preview	109.1%	100.0%
bench-loudness: mean error from −14 LUFS, mono / stereo	1.30 / 0.52 LU	0.26 / 0.26 LU
worst error, mono / stereo	2.4 / 1.6 LU	0.3 / 0.3 LU
highest true peak	−2.0 dBTP	−2.0 dBTP (unchanged)
bench-tighten-joins: worst join step ÷ material's 99.9th-pct step (music / bass / room)	12.7× / 118× / 14.2×	0.2× / 0.6× / 0.2×
timing drift from join handling	0 ms	0 ms

Cross-checks:

• Loudness was also run on flite TTS speech (mono and stereo). Every case is now within 0.3 LU, where mono was previously off by up to 2.4 LU.
• SupoClip's single-pass loudnorm, run on the same signals, missed by up to 2.8 LU and reached −0.9 dBTP.
• Join alternatives measured and rejected: crossfades of 10, 30 and 60 ms. They dip less (−2 to −8 dB against −12 to −20 dB, for about 10 ms) but leave join steps up to 1.5× the material's own. Longer ones cancel low notes that meet out of phase (−17 to −22 dB on a 55 Hz tone).

Release notes

• Exported karaoke captions now hold still while the highlight moves from word to word, and the highlighted word matches the preview's size.
• Exports land closer to the −14 LUFS loudness target, especially mono recordings and clips with laughs, claps or plosives.
• Tighten cuts no longer click where pauses are removed.

(Also in CHANGELOG.md under Unreleased.)

How did you test it?

• npm test (730 passed, including new tests that fail on the old code)
• npm run typecheck
• npm run lint
• Checked the change in the running app (not done; this session had no display or API key)
• scripts/test-pipeline.ts, scripts/test-quality.ts, scripts/test-captionsize.ts pass
• The three new benchmarks pass with --check; numbers above
• Rendered before/after caption frames for beast and pill and checked them visually: neighbours no longer shift, and the word pops in place

Anything to watch out for?

• Preview and export now agree on the caption pop: 94% → 100% over 90 ms, scaled about the word's centre. Each word now emits two ASS events instead of one, which is negligible for libass.
• The pop now plays once per word. Previously it replayed when a caption was split at a layout band boundary.
• Clips that need limiting get one or two extra audio-only ffmpeg passes over the clip span during export. Linear-path clips are unchanged.
• bench-caption-stability can't isolate words for styles whose highlight is the same colour as the other words. For those styles its "during pop" column includes the popping word itself; the settled column is still exact.

🤖 Generated with Claude Code

https://claude.ai/code/session_019QYatd8aXSLCVxPx1SggNZ

---

Generated by Claude Code

---

Note

Medium Risk
Changes core export paths (ASS caption generation, loudness measurement/normalisation, and tighten audio graphs); limited-path exports may run extra ffmpeg measurement passes, but behaviour is covered by new benchmarks and unit tests.

Overview
This PR tightens export quality in three areas—karaoke captions, loudness mastering, and tighten-cut audio—and adds offline benchmarks so regressions are measurable without API keys.

Karaoke captions (captions.ts) no longer scale the active word in a single layer (which re-flowed centred libass lines and shifted neighbours 17–26px). Each word now emits two ASS dialogues: a base layer with a transparent active slot, and a pop layer that only draws the highlighted word with a preview-matched 0.94→1 pop over 90ms. The pop runs once per word, not again when events split at layout bands.

Loudness (loudness.ts, render.ts) measures and normalises the stereo mix exports actually carry (fixing mono sources measured ~3 dB too hot before downmix). Clips on the limiter path get up to two extra measurement passes via refineLimiterGain so peak-limited material still lands near −14 LUFS.

Tighten cuts (render.ts) apply 5 ms fade in/out at internal audio joins only (JOIN_FADE_SEC), removing splice clicks without changing duration or video cuts. tightenGraph is exported for tests.

Docs: CHANGELOG and README document the fixes; new scripts bench-caption-stability, bench-loudness, and bench-tighten-joins (each with --check) exercise the real pipeline paths. Unit tests cover dual-layer captions, stereo measurement, limiter refinement, and join fades.

^{Reviewed by Cursor Bugbot for commit 8cc5bab. Bugbot is set up for automated code reviews on this repo. Configure here.}