Skip to content

Pro controls: manual exposure + Cinematic video (#206) - #223

Open
darioalessandro wants to merge 6 commits into
masterfrom
issue-206
Open

Pro controls: manual exposure + Cinematic video (#206)#223
darioalessandro wants to merge 6 commits into
masterfrom
issue-206

Conversation

@darioalessandro

@darioalessandro darioalessandro commented Aug 24, 2026

Copy link
Copy Markdown
Member

Closes #206 — a user asked for the controls a photographer expects from a real camera: shutter speed, ISO, and aperture. This PR adds them to the remote, free for everyone. Design notes live in Docs/pro-controls.md.

What you get

Tray tiles, one tap deep

Open the tray (grid glyph, top-right capsule) and the controls are right there, next to timer, aspect and quality — no sub-menu:

  • SHUTTER and ISO when the camera can do manual exposure (any mode). Each tile shows the camera's current value (1/125, 400) and lights up while Manual is on.
  • CINEMATIC in video mode on phones that support it; tap to toggle, like HDR. Once it's on, APERTURE appears next to it (f/2.8).

If the camera can't do any of this (a Mac's webcam, an older phone, an older app version on the other side), the tiles simply aren't there — the remote never offers a control the camera would ignore. This works on the screen you actually land on: a single camera opens the multicam director, and its tray carries the same tiles, driving the focused camera exactly like torch and zoom.

Sliders that work like zoom

Tap SHUTTER, ISO or APERTURE and the tray closes; that control's slider appears where the zoom pill sits — the same control, not a look-alike: a log-spaced ruler over the camera's range, photographic detents you can feel (1/8000 … 1/125 … 1s; ISO in ⅓-stops; f/1.4 … f/16), relative drag so a small correction stays small, scroll wheel on the Mac, VoiceOver-adjustable. The ruler shows the value you're dragging to and settles on what the camera confirms.

Dragging the shutter locks the shutter and keeps ISO where auto had it; dragging ISO does the reverse. So your first drag engages Manual from a correctly exposed frame instead of a black one. AUTO on the pill hands exposure back to the camera and closes it; × just closes it.

Two protections happen quietly:

  • While recording, the shutter can't go longer than one frame (1/30 at 30 fps). A longer shutter would change the clip's frame rate mid-take. In photo mode long shutters are allowed — the preview just slows down, which is expected.
  • Multi-lens phones. iPhones open on a "virtual" camera that blends its lenses, and Apple refuses manual exposure on those. When you engage Manual, the phone hops to the actual physical lens in use (the wide lens by default); AUTO restores the multi-lens camera. Zoom keeps working, within that one lens's range.

The camera phone shows a small chip at the top of its screen (M 1/125 · ISO 400) so whoever is holding it sees it's under manual control.

Cinematic video — simulated aperture (iOS 26)

Turning the CINEMATIC tile on applies Apple's Cinematic effect — shallow depth of field with subject tracking — to both the live preview and the recorded clip, and the APERTURE slider (f/1.4 … f/16) sets how shallow. Tap-to-focus on the remote becomes "track this subject" while the effect is on.

The aperture is set before a take: once recording starts the CINEMATIC and APERTURE tiles dim, because Apple rejects changes mid-clip — we don't pretend otherwise. Leaving video mode turns the effect off.

Debug console: tap a message to see what's in it

The in-app traffic console (ant icon, debug builds) listed message names only. Tapping a command row now unfolds its fields — nested state indented, arrays with counts, shutter durations also shown as fractions — so you can read straight off the device whether a CameraCapabilitiesResp carried supportsManualExposure: true and what exposure range the phone reported. It's reflection-based, so every command, including future ones, is inspectable without writing a describer.

Nothing to buy

Earlier drafts gated this behind an in-app purchase; that's gone. No new product, no store changes, no unlock screen.

How it's built

  • Wire protocol (FlatBuffers, append-only): two new commands, SetExposure and SetCinematic, each answered with the camera's actual state. Capabilities gain supports_manual_exposure and supports_cinematic_video. Old app versions ignore what they don't know.
  • Capability gates: the remote only sends a command to a peer that advertised support — same rule as device selection and tap-to-focus, pinned by loopback tests over the real wire.
  • One ruler control. The pro sliders are the zoom pill: its ruler, relative drag, scroll wheel, pending-value echo and VoiceOver element are now RulerPill, and its log-track math LogTrackZoomScale wraps it (its tests are unchanged and green), ZoomPill and ProSliderPill are thin configurations of the same view. Slider sends are throttled with the zoom throttle.
  • Pure policies decide what's allowed (ExposurePolicy, CinematicPolicy): range clamping, the recording cap, aperture lock, video-mode-only Cinematic. No AVFoundation types cross that boundary, so every rule has a table test.
  • One owner per setting in the capture engine, re-applied after a lens swap or a quality change so the setting survives both.
  • Two bugs found while testing on real devices, both of which hid the tile entirely:
    • iPhones advertised no manual exposure because the capability check read activePrimaryConstituent, which Apple documents as nil until the session is running — and capabilities are exchanged before that. The check now reads the constituent lens list, and the same function picks the hop lens, so advertising and behavior can't disagree.
    • The tile was only wired into the classic 1:1 monitor, but a single camera lands on the multicam director (MULTICAM_FOR_SINGLE_CAMERA), whose tray rendered nothing for it. The panel now takes plain values, so both screens share it; the director gets its own per-camera routing and capability gate.

What I checked

  • Manual exposure and Cinematic commands round-trip between two real coordinators over the in-process transport, and are never sent to a peer that didn't advertise them.
  • On the director: exposure/Cinematic commands go only to the camera they were rendered for, and only if it advertised the capability; the lane shows the camera's echo and a refusal surfaces as a toast; the rig's mode reaches linked cameras and late joiners once, never repeatedly.
  • Tray composition covered for every mode on both screens (photo, video, recording, shorts): tiles appear only with the capability, Cinematic only in video, Aperture only once Cinematic is on and adjustable; on the director the tiles and an open slider follow the focused camera and hide when it drops. The slider pill renders; the ruler math is pinned by the existing zoom tests plus the shutter/ISO/aperture ranges.
  • The console dump shows nested capabilities fields, intents with their values, array elements, and error domains.
  • iOS device and Mac Catalyst builds are clean; the Catalyst unit suite passes apart from two failures that reproduce on master (keychain entitlement on this machine, and local purchase state).
  • On a Mac: both cameras report no .custom exposure support, so the PRO tile stays hidden — the gate works end to end.

Still needs a real iPhone

  • Open the remote's tray (grid glyph, top right): SHUTTER and ISO tiles should be there as soon as the camera's capabilities land. If not, tap the CameraCapabilitiesResp row in the debug console and check supportsManualExposure / supportsCinematicVideo; on the camera phone the 🌗 EXPOSURE PROBE lines should show the virtual camera custom=false and its lenses custom=true.
  • Tap SHUTTER, drag the slider, then ISO: the phone's chip and the picture should change, the tiles light up, and AUTO on the pill restores the multi-lens camera.
  • iOS 26: tap CINEMATIC, drag APERTURE, record a clip — the effect should be in the file and both tiles dim while recording.
  • Manual and Cinematic together — the policy allows it; the hardware may not.

🤖 Generated with Claude Code

https://claude.ai/code/session_01K5uHRuiiLu5FEBiHZkBWo3

Adds ExposurePolicy (pure clamp/recording-cap/unsupported decisions),
SetExposure=33 + ExposureState on the wire, supports_manual_exposure
capability, one-owner apply in CaptureEngine re-applied on device swap and
format change, coordinator gate pinned by loopback tests, and a hardware
probe in CaptureIntegrationTests. No UI yet. Design: Docs/pro-controls.md.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Aug 24, 2026

Copy link
Copy Markdown
Contributor

📊 Test Coverage Report

914/928 tests passed · ⏭️ 14 skipped

Metric Coverage
App code (excl. generated + vendored Theater) ██████████░░░░░░░░░░ 48.4% 🔴
Whole target ██████████░░░░░░░░░░ 49.4% 🔴

🎥 Capture & session core

File Coverage Lines
CaptureEngine.swift ██░░░░░░░░░░░░░░░░░░ 8.9% 177/1984 🔴
RecordingPipeline.swift ████████████████░░░░ 80.6% 697/865 🟢
FrameStreamingCoordinator.swift ░░░░░░░░░░░░░░░░░░░░ 0.0% 0/154 🔴
FrameStreamer.swift ███████████████████░ 93.4% 113/121 🟢
RemoteCmdFlatBuffers.swift ███████████████████░ 95.1% 1560/1641 🟢
📁 Full report — 106 files
File Coverage Lines
DebugLog.swift █████████████████████ 104.8% 22/21 🟢
CountdownTimer.swift ████████████████████ 100.0% 48/48 🟢
JPEGFrameEncoder.swift ████████████████████ 100.0% 12/12 🟢
CameraPreviewMode.swift ████████████████████ 100.0% 13/13 🟢
AppActivityMonitor.swift ████████████████████ 100.0% 31/31 🟢
AppTheme.swift ████████████████████ 100.0% 33/33 🟢
Messages.swift ████████████████████ 100.0% 7/7 🟢
MultiCamChrome.swift ████████████████████ 100.0% 37/37 🟢
ZoomScale.swift ████████████████████ 100.0% 56/56 🟢
CinematicPolicy.swift ████████████████████ 100.0% 72/72 🟢
ZoomScaleSeed.swift ████████████████████ 100.0% 14/14 🟢
MonitorChrome.swift ████████████████████ 100.0% 72/72 🟢
PeerSessionCore.swift ████████████████████ 100.0% 26/26 🟢
ClockOffsetEstimator.swift ████████████████████ 100.0% 24/24 🟢
ExposurePolicy.swift ████████████████████ 100.0% 30/30 🟢
StreamingConfig.swift ████████████████████ 100.0% 34/34 🟢
MultipeerMessages.swift ████████████████████ 100.0% 12/12 🟢
FocusedCameraControlState.swift ████████████████████ 100.0% 6/6 🟢
Locked.swift ████████████████████ 100.0% 19/19 🟢
FrameCreditWindow.swift ████████████████████ 100.0% 9/9 🟢
AppDelegate.swift ████████████████████ 100.0% 17/17 🟢
CameraLink.swift ████████████████████ 100.0% 56/56 🟢
HEICFrameEncoder.swift ████████████████████ 100.0% 17/17 🟢
PeerCompatibility.swift ████████████████████ 100.0% 42/42 🟢
RigQualityMenu.swift ████████████████████ 99.3% 138/139 🟢
DeviceScannerViewModel.swift ████████████████████ 98.9% 184/186 🟢
FrameSender.swift ████████████████████ 98.4% 187/190 🟢
ZoomPill.swift ████████████████████ 97.8% 91/93 🟢
FocusPointMapping.swift ████████████████████ 97.6% 40/41 🟢
RemoteCmds.swift ███████████████████░ 96.7% 384/397 🟢
CaptureSyncMetadata.swift ███████████████████░ 95.5% 64/67 🟢
RemoteCmdFlatBuffers.swift ███████████████████░ 95.1% 1560/1641 🟢
DiscoveredPeers.swift ███████████████████░ 94.4% 17/18 🟢
WatchSharedTypes.swift ███████████████████░ 94.0% 236/251 🟢
VP9FrameEncoder.swift ███████████████████░ 93.9% 153/163 🟢
WatchPreviewStreamer.swift ███████████████████░ 93.6% 44/47 🟢
FrameStreamer.swift ███████████████████░ 93.4% 113/121 🟢
WelcomeView.swift ██████████████████░░ 89.3% 492/551 🟢
MulticamController.swift ██████████████████░░ 89.3% 1363/1527 🟢
MulticamViewModel.swift ██████████████████░░ 88.9% 120/135 🟢
MediaProcessors.swift █████████████████░░░ 84.6% 11/13 🟢
VideoTransferProgressView.swift █████████████████░░░ 84.0% 295/351 🟢
UIAlertController.swift █████████████████░░░ 83.0% 44/53 🟢
UIImage+gif.swift ████████████████░░░░ 81.6% 129/158 🟢
ProSliderPill.swift ████████████████░░░░ 81.4% 92/113 🟢
CameraDeviceDescriptor.swift ████████████████░░░░ 81.0% 34/42 🟢
RecordingPipeline.swift ████████████████░░░░ 80.6% 697/865 🟢
RulerPill.swift ████████████████░░░░ 79.8% 320/401 🟡
CameraProgressOverlayView.swift ████████████████░░░░ 78.5% 186/237 🟡
FlatBufferSchemas_generated.swift (excluded from app-code metric) ████████████████░░░░ 78.1% 939/1203 🟡
MonitorView.swift ████████████████░░░░ 77.8% 1207/1552 🟡
RolePickerView.swift ███████████████░░░░░ 76.7% 330/430 🟡
RecordingTimer.swift ███████████████░░░░░ 75.0% 39/52 🟡
StreamLog.swift ███████████████░░░░░ 75.0% 15/20 🟡
WelcomeViewModel.swift ███████████████░░░░░ 74.8% 104/139 🟡
CameraRecordingTimerView.swift ███████████████░░░░░ 73.4% 127/173 🟡
SettingsViewModel.swift ██████████████░░░░░░ 72.2% 96/133 🟡
SessionCoordinator.swift ██████████████░░░░░░ 72.1% 2422/3360 🟡
DeviceScannerView.swift ██████████████░░░░░░ 70.1% 1143/1631 🟡
MonitorPresenter.swift ██████████████░░░░░░ 68.1% 173/254 🟡
WelcomeViewController.swift █████████████░░░░░░░ 64.1% 41/64 🟡
ScannerLobby.swift ████████████░░░░░░░░ 62.5% 5/8 🟡
UIAlertPresenter.swift ████████████░░░░░░░░ 60.5% 26/43 🟡
MonitorViewModel.swift ████████████░░░░░░░░ 57.7% 173/300 🟡
ViewfinderGestureLayer.swift ███████████░░░░░░░░░ 54.8% 92/168 🟡
StoreManager.swift ███████████░░░░░░░░░ 54.6% 83/152 🟡
UICmds.swift ███████████░░░░░░░░░ 54.1% 113/209 🟡
UIViewController+SwiftUIHosting.swift ██████████░░░░░░░░░░ 48.4% 15/31 🔴
FrameCodecs.swift █████████░░░░░░░░░░░ 47.1% 24/51 🔴
SceneDelegate.swift █████████░░░░░░░░░░░ 45.8% 11/24 🔴
FrameStreamReceiver.swift █████████░░░░░░░░░░░ 45.5% 234/514 🔴
CameraViewModel.swift █████████░░░░░░░░░░░ 43.5% 127/292 🔴
PurchaseManaging.swift ████████░░░░░░░░░░░░ 42.0% 21/50 🔴
CameraScreenView.swift ████████░░░░░░░░░░░░ 39.1% 262/670 🔴
MonitorViewController.swift ███████░░░░░░░░░░░░░ 37.4% 85/227 🔴
HEVCFrameEncoder.swift ███████░░░░░░░░░░░░░ 36.5% 91/249 🔴
EnumExtensions.swift ███████░░░░░░░░░░░░░ 33.9% 40/118 🔴
DeviceScannerViewController.swift █████░░░░░░░░░░░░░░░ 24.5% 119/486 🔴
SoundManager.swift █████░░░░░░░░░░░░░░░ 23.4% 11/47 🔴
CameraRig.swift █████░░░░░░░░░░░░░░░ 23.1% 157/681 🔴
SessionDebugConsole.swift █████░░░░░░░░░░░░░░░ 22.6% 143/633 🔴
WatchSessionManager.swift ████░░░░░░░░░░░░░░░░ 20.8% 37/178 🔴
MonitorViewController+SwiftUI.swift ███░░░░░░░░░░░░░░░░░ 17.1% 84/491 🔴
SwiftConstants.swift ███░░░░░░░░░░░░░░░░░ 13.3% 8/60 🔴
RolePickerController.swift ██░░░░░░░░░░░░░░░░░░ 10.2% 23/226 🔴
CaptureEngine.swift ██░░░░░░░░░░░░░░░░░░ 8.9% 177/1984 🔴
PeerLinkStatus.swift █░░░░░░░░░░░░░░░░░░░ 7.1% 10/141 🔴
CameraCountdownTorch.swift █░░░░░░░░░░░░░░░░░░░ 4.9% 3/61 🔴
WatchRemoteCameraController.swift █░░░░░░░░░░░░░░░░░░░ 3.2% 9/279 🔴
MulticamView.swift ░░░░░░░░░░░░░░░░░░░░ 0.9% 12/1285 🔴
CameraPermissionsView.swift ░░░░░░░░░░░░░░░░░░░░ 0.0% 0/640 🔴
CameraHostController.swift ░░░░░░░░░░░░░░░░░░░░ 0.0% 0/203 🔴
RemoteShutterHelpView.swift ░░░░░░░░░░░░░░░░░░░░ 0.0% 0/1373 🔴
CameraPermissionErrorView.swift ░░░░░░░░░░░░░░░░░░░░ 0.0% 0/416 🔴
PhotosAccessDeniedView.swift ░░░░░░░░░░░░░░░░░░░░ 0.0% 0/614 🔴
FrameStreamingCoordinator.swift ░░░░░░░░░░░░░░░░░░░░ 0.0% 0/154 🔴
MicrophonePermissionPromptView.swift ░░░░░░░░░░░░░░░░░░░░ 0.0% 0/347 🔴
SettingsView.swift ░░░░░░░░░░░░░░░░░░░░ 0.0% 0/788 🔴
Photos.swift ░░░░░░░░░░░░░░░░░░░░ 0.0% 0/12 🔴
MulticamViewController.swift ░░░░░░░░░░░░░░░░░░░░ 0.0% 0/317 🔴
LocalNetworkPermissionView.swift ░░░░░░░░░░░░░░░░░░░░ 0.0% 0/747 🔴
PermissionManager.swift ░░░░░░░░░░░░░░░░░░░░ 0.0% 0/159 🔴
OrientationUtils.swift ░░░░░░░░░░░░░░░░░░░░ 0.0% 0/18 🔴
PhotosAccessHelper.swift ░░░░░░░░░░░░░░░░░░░░ 0.0% 0/347 🔴
MultipeerService.swift ░░░░░░░░░░░░░░░░░░░░ 0.0% 0/123 🔴
Data+MD5.swift ░░░░░░░░░░░░░░░░░░░░ 0.0% 0/4 🔴

Generated from fb4335d · workflow run

- CinematicPolicy + SetCinematic=34 + CinematicState on the wire, gated on
  supports_cinematic_video; engine owns isCinematicVideoCaptureEnabled/
  simulatedAperture in one apply function (iOS 26 availability-guarded),
  Cinematic tap-to-focus routing, mode/recording locks per policy.
- Manual-exposure lens hop: entering Manual on a virtual device swaps to
  activePrimaryConstituent at runtime only when the device refuses .custom.
- Monitor UI: PRO tray tile -> ProControlsPanel (Auto/Manual segmented,
  detented shutter/ISO/aperture dials, echo-driven values, VoiceOver
  adjustable), camera-side readout chip; behind ENABLE_PRO_CONTROLS.
- IAP "10" Pro Controls mirroring tap-to-focus, 15-locale strings.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@darioalessandro darioalessandro changed the title Manual exposure: engine, wire protocol, capability gate (#206, PR 1 of 4) Pro controls: manual exposure + Cinematic video (#206) Aug 24, 2026
@darioalessandro
darioalessandro marked this pull request as ready for review August 24, 2026 06:02
darioalessandro and others added 4 commits August 23, 2026 23:19
The PRO panel is included for every user; the only gate left is the
capability gate (button appears only when the camera offers the feature).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…onstituent

activePrimaryConstituent is nil until the virtual device is used in a
running session (AVCaptureDevice.h), and the first capabilities exchange
fires before the session starts. Every modern iPhone opens on a virtual
device, so the camera advertised supports_manual_exposure = false and the
PRO tray tile never appeared. The lens choice is now one function
(manualExposureLensLocked) used by both the capability and the hop; the
tray predicate moved into MonitorTray so it is unit-tested.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5uHRuiiLu5FEBiHZkBWo3
…essage

A single camera lands on the multicam director (MULTICAM_FOR_SINGLE_CAMERA),
whose tray rendered nothing for .proControls — so the PRO tile only ever
existed on the classic 1:1 monitor nobody reaches. The director now offers
the tile for the FOCUSED camera (like torch and zoom): CameraLink carries
that camera's echoed exposure/Cinematic truth, RigTray lists the tile when
the focused lane advertised a capability, and MulticamController sends
SetExposure/SetCinematic only to a camera that advertised them. The rig's
photo/video mode is pushed to cameras (SyncMonitorSettings, incl. late
joiners) so Cinematic is not refused as photo-mode. ProControlsPanel takes
plain values so both monitors share it.

Debug console: TrafficEntry carries a reflective MessageDump of the
message; tapping a command row unfolds its fields, so a capabilities
message shows supportsManualExposure / exposure ranges on the device.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5uHRuiiLu5FEBiHZkBWo3
… zoom and pro

The PRO tile → panel → dials path was two taps too deep. Shutter, ISO,
Cinematic and Aperture are now first-class tray tiles (with the camera's
current value on each); SHUTTER / ISO / APERTURE open a slider in the zoom
pill's slot, Cinematic toggles in place like HDR. The slider IS the zoom
control: the ruler, relative drag, scroll wheel, pending-value echo and
VoiceOver adjustable element are extracted from ZoomPill into RulerPill,
and the log-track math from ZoomScale into LogTrack (ZoomScale wraps it;
ZoomScaleTests unchanged and green). ZoomPill and ProSliderPill are thin
configurations. Slider sends are throttled with the zoom throttle. Both the
1:1 monitor and the multicam director get the same tiles and slider.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5uHRuiiLu5FEBiHZkBWo3
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Suggestions

1 participant