Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .agents/skills/g6-orientation/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ interface.
|---|---|
| **reiserlab/maDisplayTools** | **MATLAB** tools (G4+). Pattern math reference — **web output must match MATLAB exactly** (CI validates, tol 1e-4). Its protocol **runner is G4.1-only and frozen** (decision 2026-10-26); protocol authority = `webDisplayTools/docs/development/v3-spec.md` + conformance fixtures. (Its `docs/development/G4G6_ROADMAP.md` is largely historical now — use the umbrella docs for current G6.) |
| **reiserlab/LED-Display_pyDisplayTools** (pyDisplayTools) | **Python** tools for the displays (G4 pattern + experiment-folder tools). **Accepted destination** for the Python arena library and the **headless runner** (`arena-run`), a peer of the web runner (decision 2026-10-26). |
| **reiserlab/rig-nodes** | **Accepted destination, not yet created:** shared ROS 2 infrastructure for every lab rig (`rig_interfaces`, device nodes for FicTrac / Kinefly / BIAS / ScanImage / SiPM, MCAP recording and supervision primitives). Today that code lives in `webDisplayTools/ros2-poc/` on the 2026-10-20 release candidate. Record: `webDisplayTools/docs/development/rig-software-repos-and-runners.md`. |
| **reiserlab/rig-nodes** | **Accepted destination, not yet created:** shared ROS 2 infrastructure for every lab rig (`rig_interfaces`, device nodes for FicTrac / Kinefly / BIAS / ScanImage / SiPM, MCAP recording and supervision primitives). Today that code lives in `webDisplayTools/rig/` on the 2026-10-20 release candidate. Record: `webDisplayTools/docs/development/rig-software-repos-and-runners.md`. |

### G6 firmware (what the web connects to over USB)
| repo | role |
Expand Down Expand Up @@ -124,7 +124,7 @@ interface.
(WebSocket `ws://localhost:8765` ← UDP `:60000`) → browser. `pixi run bridge` + `pixi run
sim` (start the bridge first). The web runner executes only the `fictrac` and `log` plugins;
`backlight`/`camera`/`temperature` are declared for MATLAB and **skipped on the web**.
- **ROS 2 graph (LAB-303, candidate 2026-10-20)** — `ros2-poc/` replaces the bridge with a graph
- **ROS 2 graph (LAB-303, candidate 2026-10-20)** — `rig/` replaces the bridge with a graph
(rmw_zenoh, MCAP record, BIAS + ScanImage connectors) behind the SAME WebSocket; the Studio is
unchanged. Where the pieces will live from here: `rig-software-repos-and-runners.md` (above).

Expand Down
Original file line number Diff line number Diff line change
@@ -1,30 +1,30 @@
# Build ros2-poc/interfaces/arena_interfaces (the typed ROS 2 messages of the arena bridge graph,
# Build rig/interfaces/rig_interfaces (the typed ROS 2 messages of the arena bridge graph,
# LAB-303) ONCE per platform in CI and publish the result as a conda channel, so a rig PC installs
# typed messages as a download — no MSVC, no build backend, no MAX_PATH workaround. Decision and
# the consumer-side manifest edits: ros2-poc/docs/typed-distribution-decision.md; the measurements
# behind it: ros2-poc/docs/typed-windows.md.
# the consumer-side manifest edits: rig/docs/typed-distribution-decision.md; the measurements
# behind it: rig/docs/typed-windows.md.
#
# Runs: build-only on a PR that touches the package or this file (proves the three platforms still
# build); the `arena-interfaces-channel` artifact on every run; the hosted-channel upload only on an
# `arena-interfaces-v*` tag AND only when the repository variable ARENA_INTERFACES_CHANNEL names a
# build); the `rig-interfaces-channel` artifact on every run; the hosted-channel upload only on an
# `rig-interfaces-v*` tag AND only when the repository variable RIG_INTERFACES_CHANNEL names a
# prefix.dev channel (with the PREFIX_API_KEY secret). Until the lab owns one, the channel artifact
# is committed into ros2-poc/channel/ by hand (the decision note has the recipe).
name: arena-interfaces
# is committed into rig/channel/ by hand (the decision note has the recipe).
name: rig-interfaces

on:
pull_request:
paths:
- 'ros2-poc/interfaces/**'
- '.github/workflows/arena-interfaces.yml'
- 'rig/interfaces/**'
- '.github/workflows/rig-interfaces.yml'
push:
tags: ['arena-interfaces-v*']
tags: ['rig-interfaces-v*']
workflow_dispatch:

permissions:
contents: read

concurrency:
group: arena-interfaces-${{ github.event.pull_request.number || github.ref }}
group: rig-interfaces-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true

env:
Expand All @@ -48,7 +48,7 @@ jobs:
defaults:
run:
shell: bash
working-directory: ros2-poc
working-directory: rig
steps:
- uses: actions/checkout@v7
with:
Expand All @@ -57,16 +57,16 @@ jobs:
with:
pixi-version: ${{ env.PIXI_VERSION }}
run-install: false
# MAX_PATH: the runner's checkout (D:\a\webDisplayTools\webDisplayTools\ros2-poc, 45
# MAX_PATH: the runner's checkout (D:\a\webDisplayTools\webDisplayTools\rig, 40
# chars) plus the ~211 chars the rosidl build adds crosses 260. detached-environments
# moves .pixi (build workspace included) to a 4-char root — the ww18 recipe.
- name: Windows build root
if: runner.os == 'Windows'
run: pixi config set --local detached-environments 'C:\px'
- name: pixi build arena_interfaces
- name: pixi build rig_interfaces
run: |
rm -rf ../dist
pixi build --path interfaces/arena_interfaces/pixi.toml --output-dir ../dist
pixi build --path interfaces/rig_interfaces/pixi.toml --output-dir ../dist
mkdir -p ../dist/${{ matrix.subdir }}
mv ../dist/*.conda ../dist/${{ matrix.subdir }}/
ls -la ../dist/${{ matrix.subdir }}
Expand All @@ -84,7 +84,7 @@ jobs:
EOF
- uses: actions/upload-artifact@v5
with:
name: arena_interfaces-${{ matrix.subdir }}
name: rig_interfaces-${{ matrix.subdir }}
path: dist/${{ matrix.subdir }}/*.conda
if-no-files-found: error
retention-days: 30
Expand All @@ -102,13 +102,13 @@ jobs:
# dist/<subdir>, so the subdir is restored here from the artifact name).
- uses: actions/download-artifact@v5
with:
pattern: arena_interfaces-*
pattern: rig_interfaces-*
path: artifacts
- name: assemble + index the channel
run: |
mkdir -p channel
for d in artifacts/arena_interfaces-*; do
subdir=${d#artifacts/arena_interfaces-}
for d in artifacts/rig_interfaces-*; do
subdir=${d#artifacts/rig_interfaces-}
mkdir -p "channel/$subdir"
cp "$d"/*.conda "channel/$subdir/"
done
Expand All @@ -121,20 +121,20 @@ jobs:
for name, rec in pk.items():
print(rd.parent.name, name, rec["sha256"], [d for d in rec["depends"] if d.startswith("ros2-distro-mutex")])
EOF
# The deliverable: a complete three-platform channel. Commit it into ros2-poc/channel/
# The deliverable: a complete three-platform channel. Commit it into rig/channel/
# (decision note §"Releasing a new build") or let the step below upload it.
- uses: actions/upload-artifact@v5
with:
name: arena-interfaces-channel
name: rig-interfaces-channel
path: channel
if-no-files-found: error
retention-days: 90
# Hosted channel (prefix.dev), only when the lab has one: set the repository variable
# ARENA_INTERFACES_CHANNEL to its name and the secret PREFIX_API_KEY. Tags only, so a
# RIG_INTERFACES_CHANNEL to its name and the secret PREFIX_API_KEY. Tags only, so a
# PR build can never publish.
- name: upload to the hosted channel
if: startsWith(github.ref, 'refs/tags/arena-interfaces-v') && vars.ARENA_INTERFACES_CHANNEL != ''
if: startsWith(github.ref, 'refs/tags/rig-interfaces-v') && vars.RIG_INTERFACES_CHANNEL != ''
env:
PREFIX_API_KEY: ${{ secrets.PREFIX_API_KEY }}
ARENA_INTERFACES_CHANNEL: ${{ vars.ARENA_INTERFACES_CHANNEL }}
run: pixi exec rattler-build upload prefix --channel "$ARENA_INTERFACES_CHANNEL" channel/*/*.conda
RIG_INTERFACES_CHANNEL: ${{ vars.RIG_INTERFACES_CHANNEL }}
run: pixi exec rattler-build upload prefix --channel "$RIG_INTERFACES_CHANNEL" channel/*/*.conda
8 changes: 4 additions & 4 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -108,7 +108,7 @@ When editing web tools, audit all buttons and controls in the modified code sect
- Web outputs must match MATLAB outputs exactly

**Repository boundaries (decision 2026-10-26):** `docs/development/rig-software-repos-and-runners.md`
is the record. The browser tools and their rig-side gateway (`ros2-poc/nodes/bridge_node.py`) stay
is the record. The browser tools and their rig-side gateway (`rig/nodes/bridge_node.py`) stay
here; the Python arena library + headless runner go to `LED-Display_pyDisplayTools`; shared ROS 2
infrastructure (interfaces, device nodes, recording, supervision) goes to `rig-nodes`, which does not
exist yet. Until the extraction session, every file is where the record's inventory says it is today.
Expand Down Expand Up @@ -369,17 +369,17 @@ fix flows to every page automatically; two hand-written HTML pages never will.
colocated < uploaded — every write names its layer, arrival order never decides). The awaits
before the load are allowlisted in `tests/test-studio-preview-sources.js`.
- **ROS 2 bridge = the bridge going forward (LAB-303, decided 2026-10-16/17).** Rig PCs run
`cd ros2-poc && pixi run rig` (its own pixi workspace: Zenoh router + FicTrac adapter + bridge node
`cd rig && pixi run rig` (its own pixi workspace: Zenoh router + FicTrac adapter + bridge node
+ optional `--bias` / `--scanimage` / `--foxglove`). Bridge 3.4 (`pixi run bridge` at the root) is
the legacy path for course rigs, BUT `fictrac-bridge/bridge.py` is STILL IMPORTED by
`ros2-poc/nodes/bridge_node.py` and `mcap_convert.py` (Hub, LogWriter, Pipeline, the pure
`rig/nodes/bridge_node.py` and `mcap_convert.py` (Hub, LogWriter, Pipeline, the pure
encoders): a change there changes ROS acquisition and the conversion of every historical bag.
**MCAP bag = the record**; the `behavior_v2` JSONL is DERIVED (converter), transitional and
FicTrac-only — new plugin streams go into the bag, never into a new `behavior_vN`. Bags are
archived on the group share (later a bucket), never pushed to GitHub, so `--max-bag-mb` (rosbag2
splitting) is an optional knob, off by default. `mcap_index.reindex` replaces the bag ATOMICALLY
(`os.replace`); never reintroduce an unlink-then-rename on the record of truth. Row encoders shared
by the live writer and the converter live in `ros2-poc/nodes/_rows.py` (NO ROS imports — the
by the live writer and the converter live in `rig/nodes/_rows.py` (NO ROS imports — the
converter must run without a ROS process; `converted == live` depends on ONE encoder).
- **Connector plugins (LAB-303):** a protocol plugin whose class is `BiasPlugin` → connector `bias`,
`ScanImagePlugin` → `scanimage` (`connectorPluginNames(exp)` in the Studio). The runner turns their
Expand Down
16 changes: 8 additions & 8 deletions docs/development/fictrac-columns-in-the-bag.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,15 +3,15 @@
**Status:** DECIDED (Michael, 2026-10-18): 10 columns, one record per frame, `float64`, one MCAP
message per frame (no batching; "simpler is better", 2× the JSONL accepted). Nothing in the recorder
changes with this PR; the implementation is its own PR (§4).
**Question:** the ROS 2 bridge (`ros2-poc/`, PR #254) records every FicTrac frame twice — the
**Question:** the ROS 2 bridge (`rig/`, PR #254) records every FicTrac frame twice — the
25-column record as the adapter received it (`/fictrac/frame`, with the verbatim line) and the
closed-loop evaluation (`/arena/frame`). Since the bags will be pushed to the GitHub data repo like
the `.jsonl.gz` is today (Michael, 2026-10-18), bag size matters: which FicTrac columns does the bag
have to keep so it stays near the size of today's `.jsonl.gz`?
**Answer in one line:** keep 10 of the 25 columns (the 5 the behavior row has, plus the error score,
the three lab-frame delta-rotation components and the sequence counter), store them in the ONE
per-frame `/arena/frame` record and stop recording the raw FicTrac topics by default. Measured on the
two bench bags with a chunk-only index and zstd 19, one message per frame (`ros2-poc/sizing/columns.py`,
two bench bags with a chunk-only index and zstd 19, one message per frame (`rig/sizing/columns.py`,
the reproducible script of the implementation PR #264): **2.41× and 2.68×** the `.jsonl.gz` (today's
bags: 10.2× and 30.4×; all 25 columns: 4.4× and 5.3×). For a 30 minute run at 50 Hz that is ≈ 5–5.5 MB
against ≈ 2.3 MB today. Two levers were measured and deliberately not pulled: batching the records
Expand Down Expand Up @@ -59,7 +59,7 @@ dashboard derivations), `js/runlog-replay.js` (`COL_X/Y/HD/TS` for `full`-level
forward / speed / turning / heading, `fc` gaps), `scripts/closed-loop-report.py` (`ms, fc, idx, hd`;
`f[0]`, `f[16]` on `full` logs), `scripts/runlog-check.py` (counts rows), `scripts/bench-check.py`
(`ms, idx`), `scripts/telemetry-report.py` / `wedge-scan.py` (skip FicTrac rows),
`fictrac-bridge/fictrac_sim.py` (writes all 25; col 22 in ns), and in `ros2-poc/`:
`fictrac-bridge/fictrac_sim.py` (writes all 25; col 22 in ns), and in `rig/`:
`nodes/bridge_node.py` (`arena_frame_msg`), `nodes/mcap_convert.py` (`frame_line`,
`fictrac_fields`: the 25 columns are read ONLY for `--level full`), `interfaces/…/FicTracFrame.msg`,
`interfaces/…/ArenaFrame.msg`. The MATLAB tools do not read these logs.
Expand Down Expand Up @@ -169,15 +169,15 @@ so the bag says "untracked" rather than "missing". The JSONL is unaffected (no r
## 3. Measured bag sizes

Two real bench runs (Mac, 2026-10-14, simulator-driven FicTrac at 50 Hz, G6 controller, BIAS node;
`ros2-poc/runlogs/arena-log-20261014-194449-517` = 181 s, typed topics; `…-204300-169` = 164 s,
`rig/runlogs/arena-log-20261014-194449-517` = 181 s, typed topics; `…-204300-169` = 164 s,
String fallback), each with its `.jsonl` as exported; `.jsonl.gz` = gzip −9 of that file, which is
what the Studio commits. Every MCAP variant is written by the pure-Python `mcap` writer the way
`mcap_index.reindex` writes the final layout: chunk index only (no per-message index), zstd level 19,
"batched" = one message per topic per wall-clock second holding that second's records. Column sets
are packed as little-endian structs (≈ typed CDR: `int64` for cols 1/23, `int64` ns for col 22,
`float64` otherwise); `/arena/frame` is re-packed from its decoded fields in BOTH bags so the typed
and the String run compare like for like; every other topic (`/arena/browser` echoes, `/bias/*`,
events, session) is kept byte for byte. Script: `ros2-poc/sizing/columns.py`
events, session) is kept byte for byte. Script: `rig/sizing/columns.py`
(`pixi run size-columns -- <bag> <jsonl> --all`), PR #264.

| Variant (chunk-only index, zstd 19) | bag 1 (9 055 frames) | × jsonl.gz | bag 2 (8 199 frames) | × jsonl.gz |
Expand Down Expand Up @@ -236,10 +236,10 @@ the pinned `ros-jazzy-rosbag2-storage-mcap` when implementing, or apply the tuni

## 4. Code change list (not implemented here)

All under `ros2-poc/` on PR #254's branch, its own PR after the pilot starts (plan §3 said the
All under `rig/` on PR #254's branch, its own PR after the pilot starts (plan §3 said the
archive publisher is its own PR; this is the recorder half of the same decision).

1. **`interfaces/arena_interfaces/msg/ArenaFrame.msg`** — add `err`, `dr_lab_x/y/z`, `ft_seq`,
1. **`interfaces/rig_interfaces/msg/ArenaFrame.msg`** — add `err`, `dr_lab_x/y/z`, `ft_seq`,
`ft_ns`, `flags` (above). One type for the live topic and the bag, one message per frame, as today;
the recorded copy carries NaN in `turning_dps` / `forward_mms` (live-only, §3 correction).
The String fallback carries the same fields in its JSON object.
Expand Down Expand Up @@ -278,7 +278,7 @@ archive publisher is its own PR; this is the recorder half of the same decision)
§2 becomes the `ArenaFrame` field reference; `telemetry-logging-reference.md` gets a one-paragraph
pointer (the JSONL section is unchanged).

Outside `ros2-poc/` nothing changes: `fictrac-bridge/bridge.py`, `js/kinematics.js`, the replay, the
Outside `rig/` nothing changes: `fictrac-bridge/bridge.py`, `js/kinematics.js`, the replay, the
dashboard and every script read the JSONL, whose rows are the same five columns as before.

## 5. Proposed Linear comment on LAB-303 (text only; not applied)
Expand Down
Loading
Loading