Skip to content

Simulator apps get camera frames via an injected library (fixes #7) - #8

Merged
dautovri merged 3 commits into
mainfrom
feat/simulator-injection
Oct 15, 2026
Merged

dautovri merged 3 commits into
mainfrom
feat/simulator-injection

Conversation

@dautovri

Copy link
Copy Markdown
Owner

Fixes #7.

Root cause

The iOS Simulator's capture stack is a stub on Xcode 26/27: FigCaptureSourceSimulator answers every property with error -12784 and has no streaming path. AVCaptureDevice lists zero cameras in iOS 26.5 and iOS 27 simulators — not even the Mac's built-in camera. So a CMIO virtual camera can never reach a simulator app, whatever the extension does.

Fix

  • SimulatorInjection/libSimCamInject.dylib — loaded into the app in the Simulator only (DYLD_INSERT_LIBRARIES in the scheme). Your app imports nothing. It supplies a "SimulatorCamera" device through the normal AVFoundation API: discovery (default, DiscoverySession, devices, deviceWithUniqueID), AVCaptureDeviceInput, session running state + notifications, AVCaptureVideoDataOutput (BGRA sample buffers), AVCaptureVideoPreviewLayer, AVCaptureMetadataOutput (QR), AVCapturePhotoOutput (JPEG). Built-in test pattern when no Mac feed.
  • Frame transport — the Mac app writes the newest frame to /private/tmp/SimulatorCamera/frame.bin (64-byte header + BGRA, seqlock). Simulator processes read host paths directly.
  • Mac app writes every source's frames there (sandbox temporary exception for that folder; camera entitlement added) and feeds its test pattern from launch. The dylib is bundled in Contents/Resources.
  • simcamctl feed --pattern | --qr TEXT | --image PATH | --camera for CI / headless use.
  • README: setup section for iOS Simulator apps.

Verified (Xcode 27, Mac Studio)

iOS 27.0 iOS 26.5
Camera found SimulatorCamera SimulatorCamera
Video frames / 3 s 90 76
Preview layer shows feed shows feed
QR from Mac decoded yes yes
Photo capture JPEG JPEG

Also verified the sandboxed Mac app writes the frame file at 30 fps and a simulator app receives it.

Known limits

  • Simulator only; the extension still serves macOS apps.
  • The temporary sandbox exception is fine for Developer ID / build-from-source; the Mac App Store usually rejects it.
  • Not yet: barcode types other than QR, depth, multi-cam, movie file output. Video is always BGRA.

🤖 Generated with Claude Code

dautovri and others added 3 commits October 15, 2026 17:46
iOS Simulator capture is a stub on Xcode 26/27 (FigCaptureSourceSimulator
returns -12784; AVCaptureDevice lists no cameras, not even the Mac's), so a
CMIO virtual camera can never reach a simulator app.

- SimulatorInjection/libSimCamInject.dylib (DYLD_INSERT_LIBRARIES, Simulator
  only): device discovery, device input, session state, video data output,
  preview layer, QR metadata output, photo output; built-in test pattern
  when no Mac feed.
- Frames shared via /private/tmp/SimulatorCamera/frame.bin (seqlock header).
- Mac app writes every frame there (sandbox exception for that folder,
  camera entitlement); the dylib is bundled in Contents/Resources.
- simcamctl feed --pattern|--qr|--image|--camera.
Verified on iOS 27.0 and 26.5 simulators: 76-90 frames/3 s at 1280x720,
preview shows the feed, QR payload decoded, photo JPEG returned.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Simulator apps get frames as soon as SimulatorCamera is open, before any
source is picked. Verified headless: the sandboxed Debug app writes the
frame file at 30 fps (temporary-exception path works) and an iOS 27
simulator app receives 90 frames/3 s, preview + photo.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Simulator frame file now fed from the canonical CVPixelBuffer pipeline.
Test pattern goes to simulator apps only (no sink, no camera access);
other sources always write the frame file and push to the sink when the
extension is reachable, so simulator apps work even before the extension
is approved. Re-verified on iOS 27: app pattern 31 fps -> 90 frames/3 s,
simcamctl feed --qr decoded by AVCaptureMetadataOutput.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@dautovri
dautovri merged commit fc6071f into main Oct 15, 2026
2 checks passed
@hflexgrig hflexgrig mentioned this pull request Oct 17, 2026
6 of 9 tasks
@dautovri
dautovri deleted the feat/simulator-injection branch October 29, 2026 12:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Not working for IOS 27 simulators

1 participant