refactor(sensor-oak): pure Rust on the depthai safe wrapper — drop the C++ shim - #13
Merged
Merged
Conversation
…e C++ shim sensor-oak no longer bundles oak_bridge.cpp/.h, CMakeLists.txt, build.rs or ffi.rs. It builds its pipelines through the `depthai` crate (kornia/depthai-rs: depthai-sys pure-C shim + safe node-graph wrapper) and keeps every OAK-specific decision in Rust: - src/policy.rs: OAK_* env knobs, steady->epoch clock shift, IMU rotation gate (finite / det>0 / orthonormal / not-identity), even-dim depth sizing — unit tested - src/graph.rs: connect (USB speed cap), H.264 encoder attach + degrade rule, IMU preflight/attach + extrinsics gate, intrinsics + full stereo calibration read with explicit (calibrated, metres) arguments - imu.rs: zero-timestamp hole gate, pending buffer, rotation apply — unit tested - rgbd.rs: decoupled / video-only builds, stride-honouring depth repack (tested) - stereo.rs: Sync'd GRAY8 pair; Arc<ImgFrame> replaces the retain-handle FFI for the zero-copy kornia Image keepalive Public API (OakSource, OakStereoFrame, ImuSample, OakStereoCalib, kick_wedged_oak, ...) is unchanged; unsafe impl Send is gone (auto-derived). New example oak_rgbd_probe drains rgb/depth/video/imu and prints rates for before/after comparison. `depthai` is a path dependency on ../depthai-rs until kornia/depthai-rs is published; switch to a git rev/tag pin before merging. vendor/depthai-core and scripts/build_depthai.sh stay for now (the prefix is consumed by depthai-sys). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XpkL1ayku6dKhdSTstGJJX
…s; extract build_stereo Review findings: the shim's 'if (fps < 1) fps = 30' had become fps.max(1), turning a zero sentinel into a 1 fps stream (and a 500 ms Sync threshold). Restore the default via policy::fps_or_default. The IR dot-projector RPC error is now logged like every other degrade path instead of discarded. open_stereo's inline closure becomes OakSource::build_stereo, mirroring rgbd.rs's named builders. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XpkL1ayku6dKhdSTstGJJX
Cargo does not propagate a dependency's rustc-link-arg, so sensor-oak gets a two-line build.rs reading DEP_DEPTHAI_CORE_RPATH (exported by depthai-sys, now a direct dependency for that purpose). Verified natively on the Orin against depthai-core v3.7.1: clippy clean, 12 unit tests pass. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XpkL1ayku6dKhdSTstGJJX
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XpkL1ayku6dKhdSTstGJJX
…o double copies
Applied the /simplify review and the depthai-rs API changes (header-once
messages: timestamps come from the Message trait):
- graph::Session owns device/pipeline/sockets/calibration/knobs; both open
paths share one preamble and OakSource::from_parts replaces the 8-argument
assembler; ImuAttach { queue, rot: Option } replaces the tuple + parallel
imu_aligned flag
- policy::Knobs parses the OAK_* env once (testable through a map); builders
are pure functions of their arguments
- OakError + Ctx give every depthai failure one context shape; StereoCalibError
is typed and no longer re-wrapped on each stereo_calib() call
- repack_depth builds the owned Vec directly (no scratch + clone per depth
frame); OakStereoFrame stores plain ImgFrames (Arc only when an Image is made);
next_imu keeps a packet scratch buffer, takes the clock offset lazily and
spills only over-budget samples into the pending deque
- one pop() for every queue with a single first-error latch; tight_len shared
by the GRAY8 and RGB888 checks; frame_epoch_ns shared by both modalities;
add_stereo_depth / try_add_stereo_depth mirror the H.264 helpers;
build_video_only uses the mandatory encoder attach (no contradictory log);
kick_wedged_oak is a find
- pins depthai-rs at 27a307a
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XpkL1ayku6dKhdSTstGJJX
Parity with the removed shim (found by /code-review):
- OAK_* knobs parse like C atoi/atof again: OAK_IR= / OAK_IR=off disables the
projector (was silently 0.8), OAK_DEPTH_FPS=7.5 is 7, OAK_H264_KBPS=1500k is 1500
- an IR-projector RPC failure and an IMU-node build failure fail the open again
(only the getConnectedIMU preflight degrades)
- a stereo poll error ends the stream at once instead of being retried
- kick_wedged_oak(Some("")) means any device, like open_*
- the steady-clock read is never silently substituted
Review fixes:
- per-queue first-error latch instead of one shared flag
- try_add_stereo_depth removes the nodes it created before failing
(Pipeline::remove), so the degraded RGBD pipeline still starts
- padded RGB888 rows are repacked like depth instead of dropped + logged per
poll; repack accepts an unpadded last row; one repack_rows for both
- Tensor::from_borrowed_host_readonly replaces the hand-built storage
- can_do_depth via read_intrinsics; probe example uses abs_diff; dead cmake dep
and stale doc claims removed; pins depthai-rs eb9b8a9
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XpkL1ayku6dKhdSTstGJJX
3 tasks
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TmCRdrYoyj2zokHSv5EvvM
edgarriba
added a commit
that referenced
this pull request
Sep 2, 2026
…eaders in calib.rs Simplify + review pass over the whole rewrite (#13 + #14) before merging: - Q<M>: one output queue with its own "first poll failure logged" latch, replacing the Which bitmask enum, the Cell<u8> on OakSource and the queue clones that dodged the borrow checker. OakSource holds the Queues the builder returned instead of re-flattening them. - Session::finish is the one shared tail of both open paths (attach IMU last, start, read the reference camera's intrinsics); Built/from_parts are gone. Knobs are passed into Session::connect, so the session is a function of its arguments. - Calibration readers (intrinsics, stereo calibration, IMU rotation) move from graph.rs to calib.rs next to the types they fill; one pinhole extractor. - One row-pitch normaliser feeds the stereo tightness check and the RGB/depth repacks; depth repacks straight into u16 (one allocation instead of two). - eye_image uses kornia's Image::from_borrowed_host_readonly directly; the kornia-tensor dependency goes. - add_stereo_depth tracks the nodes it made and removes them all on failure (one shape with add_h264_encoder); the no-CAM_A degrade lives in try_add_h264_encoder for both modalities; clamp_imu_hz and the device-id normalisation live in policy.rs with the other rules. - Docs say each thing once; README's open_stereo example has the right arity; CLAUDE.md points at the right files. No public API change; behaviour unchanged (the IMU keeps its per-drain clock offset as the shim always had). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TmCRdrYoyj2zokHSv5EvvM
2 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
sensor-oakbecomes pure Rust on top of thedepthaisafe wrapper from kornia/depthai-rs#1. The 937-line C++ shim (oak_bridge.cpp/.h),CMakeLists.txt,build.rsandsrc/ffi.rsare gone.Every OAK-specific decision the shim used to hide now lives in Rust, unit-tested:
src/policy.rs—OAK_*env knobs (same names/defaults),fps=0 → 30, steady→epoch clock shift, the IMU chip→camera rotation gate (finite / det>0 / orthonormal / not-identity), even-dim depth sizing.src/graph.rs— device connect (USB speed cap), H.264 encoder attach + degrade rule, IMU preflight/attach + extrinsics gate, intrinsics and full stereo calibration read with the calibrated/metres choices passed explicitly.imu.rs— zero-timestamp hole gate, pending buffer, rotation apply.rgbd.rs— decoupled / video-only builds, stride-honouring depth repack.stereo.rs— Sync'd GRAY8 pair;Arc<ImgFrame>replaces the retain-handle FFI for the zero-copy korniaImagekeepalive.Public API is unchanged (
OakSource,OakStereoFrame,ImuSample,OakStereoCalib,kick_wedged_oak, …); the existing examples compile untouched.unsafe impl Send for OakSourceis gone (auto-derived). New exampleoak_rgbd_probedrains rgb/depth/video/imu and prints rates for before/after comparison.Behavioural parity with the shim was reviewed line by line against
5debd73; two regressions found in review are fixed in-branch (fps=0default, IR-projector error logging).Build
depthai-syslinks against a depthai-core prefix: keep usingvendor/depthai(the submodule +pixi run depthai-buildstay in this PR; a follow-up drops them in favour of depthai-rs's script) or pointDEPTHAI_PREFIXat one built by depthai-rs.sensor-oak/build.rsbakes the rpath into the examples fromDEP_DEPTHAI_CORE_RPATH.DEPTHAI_SYS_SKIP_NATIVE=1gives a check-only build.Test plan
cargo clippy -p sensor-oak --lib -- -D warnings,cargo fmt --checkcargo test -p sensor-oak --libnatively against depthai-core v3.7.1 on the Orin — 12 tests (rotation gate, hole gate, repack, env defaults, calib)oak_stereo_probe/oak_rgbd_probe/oakd_xfeat_stereowith an OAK-D attached, numbers compared against a5debd73build (no camera on the build machine yet)🤖 Generated with Claude Code
https://claude.ai/code/session_01XpkL1ayku6dKhdSTstGJJX