Skip to content

Review and correct occurrence tracks in the interface, behind the tracking flag - #1432

Open
mihow wants to merge 45 commits into
claude/revive-tracking-feature-OyMO3from
feat/tracking-ui
Open

mihow wants to merge 45 commits into
claude/revive-tracking-feature-OyMO3from
feat/tracking-ui

Conversation

@mihow

@mihow mihow commented Sep 22, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

This is the interface for occurrence tracking: the tools a person uses to review and correct what tracking did, split out of #1272 so each half can be reviewed on its own. It is based on #1272 and should merge after it.

The tools were shaped by a research partner who used them on a development deployment to build a fully human-checked set of tracks, which is what tracking will be measured against. You can follow a track across captures and see its path, with the earlier and later frames faded by distance. You can find the right occurrence to merge with, build or extend a track by clicking detections capture by capture, correct a wrong frame, and mark a track as complete and accurate.

Everything is behind the project's tracking feature flag, added in #1272. With the flag off, the interface looks like main. With it on, ML data managers and project managers (the people the server reports run_tracking for) can also start a tracking run from the jobs page or a session page.

List of Changes

Change (what a reviewer gets) Notes
Every tracking surface is hidden unless the project has opted in One helper, hasProjectFeature / useProjectFeature, gates the tracking cards, film strip and frame menus, extend mode, the path and faded frames, the capture-view panel, the grouping filter and the track columns. With the flag off, the capture view keeps main's tooltip
Start a tracking run from the interface "Run tracking" in the jobs dialog (a session or a capture set, an optional cost threshold, and whether feature vectors are required) and on the session page. Shown to the people the server reports run_tracking for (ML data managers and project managers). It sends the jobs API body documented in #1272
See a track's path over the capture, with the other frames drawn as faded crops, fainter the further away they are, always beneath the current capture's boxes, and the time offset on the nearest frame each side Onion-skinned path frames with a fallback when a crop is missing
Merge dialog lists the frames next to a track first, with time, distance, similarity, the tracking cost and the tracker's own verdict; merge several at once; compare crops side by side on hover; open a candidate in the session in a new tab Sortable Match and Cost columns
Build a track by clicking detections capture by capture Session view extend mode (?extend=): add, merge a whole track or move one frame, remove, replace, auto-advance, arrow keys, Escape
While extending, every box is coloured by how likely it is the same animal, and the box the tracker would link is marked Uses capture-matches
Link to a single detection in its original capture, which keeps working after the detection moves to another occurrence ?detection= with a copy-link button
A re-laid-out occurrence panel: identity, then frame position and status, then the actions "Mark track as complete and accurate" replaces "Mark grouping correct" (translation values only; API names unchanged)
Film-strip frames with each frame's own label and score, the names given across a track, frame position, and a badge on frames without a feature vector
Faster stepping between captures, and a timeline showing where each selected occurrence appears Neighbouring captures are prefetched
Filter occurrences by confirmed grouping; sort by frames, motion, size change and agreement; export confirmed tracks as CSV
"Extend track" is now "Edit track", marked with a pencil, and opens only for an occurrence the person may edit in this session Capture toolbar and ?extend= are gated on the same permission as the track edits
"Run tracking" appears for everyone the server says may run it, and the form says which sessions tracking will skip and why Reads run_tracking from the project's user_permissions (#1272)
Moving a frame shows the destinations ranked by how well that frame matches them, in a dialog wide enough for the ranked columns Uses merge-candidates?detection=; species names stay on one line
Merge candidates can be sorted by species and by frame count, and a failed or refused load says why instead of asking for a wider scope Refusals (403) are not retried
After confirming an occurrence from the list, the dialog moves to the next one, in the order the list was opened in A snapshot of the list's order; stays on the occurrence when it is not in the list
Tracking and other post-processing jobs are named in the jobs list, and the tracks export is offered only on projects that use tracking
Long tracks open quickly: the frame strip shows 20 frames at a time, earliest first, with "Frames i–j of N" and buttons for the first, earlier, later and last frames, and opens on the page holding the frame a ?detection= link points at, which is highlighted The first page comes from the occurrence detail, later pages from /occurrences/{id}/detections/ (#1272), cached per occurrence under the occurrence query key so track edits refresh them. The strip now runs earliest first (it ran newest first), so a split still moves the chosen frame and everything after it. Frame names and the first and last frame come from the server; extend mode in the session view reads every frame from the path payload instead of the detail

How to test

  1. Check out this branch, run yarn install, then yarn tsc --noEmit, yarn lint and yarn test.
  2. On a project with the tracking flag off, the occurrence page, the session view and the occurrences list look like main.
  3. Turn the flag on, then open a session, select an occurrence and use "Edit track". The other frames appear faded, clicking a box adds it, and the view steps to the next capture.
  4. Start "Run tracking" from the jobs page for one session, and watch the job finish.

Part of #1412. Based on #1272.

🤖 Generated with Claude Code

https://claude.ai/code/session_01C7Xf6VPbwWtTumhjjF15g8

mihow and others added 10 commits September 22, 2026 13:54
The review, correction and extend-mode interface for occurrence tracking,
split out of the server-side pull request so each can be reviewed alone.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C7Xf6VPbwWtTumhjjF15g8
The track panel, film strip captions and grouping actions on occurrence
details, the path, extend mode and track edits in the session capture view,
the grouping filter and the track statistic columns all read the project's
tracking feature flag through one hook, useProjectFeature. With the flag off,
an extend link opens the session normally.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C7Xf6VPbwWtTumhjjF15g8
… and a session

On projects with tracking switched on, the new job dialog offers a tracking job
next to image processing: pick one session or a capture set, optionally set a
cost threshold, and choose whether feature vectors are required. The session
page gets a Run tracking button that fixes the scope to that session. Both post
a post_processing job with the tracking task and start it with start_now, the
same way processing jobs start.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C7Xf6VPbwWtTumhjjF15g8
With the flag off, clicking a box on a capture shows the label and score
tooltip whose name opens the occurrence, as before tracking, instead of the
new track panel with its tracking sections hidden.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C7Xf6VPbwWtTumhjjF15g8
The session button and the tracking job type are shown when the project has
tracking on and the viewer may update the project. A member who can only
create jobs would otherwise get the job saved and then refused at start.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C7Xf6VPbwWtTumhjjF15g8
Brings in the per-project opt-in flag, the jobs API support, the track export and the evaluation command that the interface depends on.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C7Xf6VPbwWtTumhjjF15g8
…racking run

The jobs API rejects a job without delay, so a tracking run started from
the interface failed with a 400 before this.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C7Xf6VPbwWtTumhjjF15g8
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C7Xf6VPbwWtTumhjjF15g8
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C7Xf6VPbwWtTumhjjF15g8
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C7Xf6VPbwWtTumhjjF15g8
@coderabbitai

coderabbitai Bot commented Sep 22, 2026 •

Copy link
Copy Markdown
Contributor

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Advanced

Run ID: 85a5eced-5505-449b-96a6-6da38d64d33f

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
  • Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Autopilot is currently an internal CodeRabbit preview.


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@netlify

netlify Bot commented Sep 22, 2026 •

Copy link
Copy Markdown

✅ Deploy Preview for antenna-preview canceled.

Name Link
🔨 Latest commit d098429
🔍 Latest deploy log https://app.netlify.com/projects/antenna-preview/deploys/6abd5ddf005d0a000834cea8

mihow and others added 16 commits September 22, 2026 16:10
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C7Xf6VPbwWtTumhjjF15g8
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C7Xf6VPbwWtTumhjjF15g8
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C7Xf6VPbwWtTumhjjF15g8
Brings in the run_tracking permission flag, ranked single-detection moves,
count refreshes, frame-to-frame relinking after manual edits and the other
server fixes, so the interface commits that follow can rely on them.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C7Xf6VPbwWtTumhjjF15g8
…track"

The mode opened from an occurrence's track lets people add, move, remove and
replace frames, so "extend" undersold it. Only the visible copy changes; the
extend URL parameter and internal identifiers stay as they are.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C7Xf6VPbwWtTumhjjF15g8
Reviewers can now click the Species and Frames headers in the merge
candidate list to reorder it, alongside the existing when, distance,
similarity, cost and match columns. Species sorts alphabetically on the
determination name alone (without the occurrence id), ignoring case, with
undetermined rows last in either direction. Frames sorts by detection count
as a number and puts the most frames first on the first click. Ties keep the
server's ranking, which stays the initial order.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C7Xf6VPbwWtTumhjjF15g8
…ccurrences list

When the details dialog is opened from the occurrences list, the Confirm button
in its header now moves on to the next occurrence on the page, or closes the
dialog after the last one, so validating a list takes one click per occurrence.
The agree buttons on individual identifications, Suggest ID, the quick ID
buttons, the taxa list's dialog and the standalone views keep their behaviour.

The header Confirm button is keyed by occurrence so its confirmed state does not
carry over to the next occurrence, and the prev/next lookup is a pure helper
with its own tests.

Part of #1415.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C7Xf6VPbwWtTumhjjF15g8
…occurrence

The success callback of an identification request runs even after the Confirm
button that sent it has unmounted, so a slow request could reopen the dialog
after the user closed it, pull them back from an occurrence they had moved on
to, or close the dialog under them. The Confirm button now reports which
occurrence it confirmed, and the dialog advances only when it is still open on
that occurrence, looking up the next one at that moment rather than from the
render where the button was clicked.

The advance logic lives in a small hook with tests for moving on, closing after
the last item, and ignoring a confirm that finishes after the user moved away or
closed the dialog.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C7Xf6VPbwWtTumhjjF15g8
… confirm

Closing the occurrence dialog by hand clears ?tab= from the list URL, but
closing it automatically after confirming the last occurrence did not, so the
next occurrence opened from the list started on the leftover tab. Both paths now
go through one close handler.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C7Xf6VPbwWtTumhjjF15g8
The test covers only the prev/next lookup, so its name no longer claims dialog
behaviour; the advance-and-close behaviour is covered by the advance hook tests.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C7Xf6VPbwWtTumhjjF15g8
The Run tracking action now follows the run_tracking permission the project
API returns, so ML data managers see it alongside project managers while basic
members do not. The tracking flag is still checked so the action stays hidden
when tracking is off.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C7Xf6VPbwWtTumhjjF15g8
…ey match it

The "Move to another occurrence" dialog now asks the server for merge
candidates measured from the moved frame, so the best match by the tracking
method comes first, and it shows the same sortable columns, comparison preview
and search scope control as the merge picker (three captures either side by
default). Occurrences with a box on the same capture are left out by the
server, and its reason is shown when a frame cannot be scored. Removing a frame
into a new occurrence is unchanged.

The candidate query string is built by a tested helper that takes an optional
detection id. The unranked neighbourhood hook and the captures lookup only it
used are removed, and the picker no longer carries default texts that described
that neighbourhood.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C7Xf6VPbwWtTumhjjF15g8
… for a wider scope

The merge and move pickers showed the normal empty-scope message whenever the
candidates request failed for any reason other than an unusable frame, so a
server error or a missing occurrence told the operator to widen the search.
All three pickers now share one helper that shows the frame's reason, a
load-failure message, or the empty-scope message, in that order.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C7Xf6VPbwWtTumhjjF15g8
…edit the track

The panel under a selected box offered Merge, Split here, Edit this track and
Mark track as complete to everyone, including basic members and anonymous
visitors on a public project, who then met a permission error. The panel now
reads the occurrence's own permissions and applies the rule the occurrence
page and the server already use: restructuring needs the occurrence delete
right, confirming a grouping needs either the update or the delete right.
Showing and hiding the path stay available to everyone.

The rule lives in one helper so the occurrence page, the extend banner and the
capture panel cannot drift apart.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C7Xf6VPbwWtTumhjjF15g8
… session

A link carrying ?extend= opened edit-track mode for anyone, including viewers
who cannot edit the occurrence, and it also drew an occurrence from another
session over the one on screen. The mode now waits for the occurrence to load
and opens only when the viewer holds the right to restructure it and it
belongs to the session being viewed. Otherwise the parameter is ignored and
the capture view behaves as if it were absent.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C7Xf6VPbwWtTumhjjF15g8
The Move to another occurrence dialog rendered at the narrow default width, so
the ranked candidate columns were clipped. It now uses the wide layout the
merge dialogs already use. Closing the dialog also returns the scope to its
default, so the next frame starts from the same place, and the candidate list
stops refetching once the move has succeeded and only the result is shown.
The unused capture id on a pending frame action is removed.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C7Xf6VPbwWtTumhjjF15g8
mihow and others added 7 commits September 28, 2026 12:15
The export form listed the tracks CSV format on every project, although a
project with tracking turned off has no tracks to export. The format is now
listed only when the project's tracking feature is on. The help text above the
form claimed there were exactly two formats; it now describes the occurrence
formats and mentions that projects using tracking can also export tracks.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C7Xf6VPbwWtTumhjjF15g8
The button that opens the track in the session view for editing still carried
the plus icon from when it only added frames. It now uses the pencil icon the
other track edit controls use.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C7Xf6VPbwWtTumhjjF15g8
Species names in the occurrence picker stay on one line, cut with an
ellipsis past a sensible width and shown in full on hover. The wide track
edit dialogs, such as move and merge, are wider so every ranked column
fits beside the name, and they size to their content up to the viewport
height instead of always filling it.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C7Xf6VPbwWtTumhjjF15g8
…s opened in

Confirming an occurrence in the occurrences list dialog refetches the list,
and with the default sort by most recently updated the confirmed occurrence
jumps to the top. Previous then opened a different occurrence than the one
just confirmed. The dialog now pages and advances through a snapshot of the
list order taken when it opens, and takes a new one only when the page,
filters or sort change, not on a background refetch.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C7Xf6VPbwWtTumhjjF15g8
A tracking job leaves out any session that already has human identifications
or has already been tracked, because the form keeps the server's defaults for
skip_if_human_identifications and require_fresh_event. A reviewer who ran it
on such a session saw a job succeed with nothing linked and no reason why.
The form, shared by the session page and the new job dialog, now says so in
one line, and a test pins that the payload keeps those defaults so the line
stays true.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C7Xf6VPbwWtTumhjjF15g8
The Run tracking form said sessions that had already been tracked are
skipped. The server actually skips any session where an occurrence already
groups several detections, which includes a manual merge or a track reaching
in from a neighbouring session, and it can also skip sessions that are not
fully processed or lack feature vectors. The note now states that rule and
no longer reads as the only reasons.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C7Xf6VPbwWtTumhjjF15g8
… key

The occurrence dialog called the list snapshot hook with an empty key and a
full copy of the list even when the caller passed no key, then threw the
result away. The hook now takes an optional key and returns the live list
unchanged without one, so the dialog uses its result directly.

The taxa list keeps following the live list on purpose: verifying a row
changes that row's example occurrence, and the page's own effect moves the
dialog to the new id, which a frozen order would lose. A comment at that call
site and on the listKey prop records why.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C7Xf6VPbwWtTumhjjF15g8
mihow and others added 3 commits September 28, 2026 21:03
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C7Xf6VPbwWtTumhjjF15g8
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C7Xf6VPbwWtTumhjjF15g8
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C7Xf6VPbwWtTumhjjF15g8
mihow added a commit that referenced this pull request Sep 29, 2026
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C7Xf6VPbwWtTumhjjF15g8
mihow added a commit that referenced this pull request Sep 29, 2026
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C7Xf6VPbwWtTumhjjF15g8
mihow and others added 2 commits September 29, 2026 13:17
The occurrence page rendered every frame of a track, and read the frame list,
the per-label counts and the track ends from the detail response. A long track
now holds several hundred frames, so the detail response only carries the first
page of them.

The frame strip shows one page at a time, earliest first, with "Frames i–j of N",
buttons for the first, earlier, later and last frames, and opens on the page that
holds the frame a `detection` link points at, which is highlighted. The first
page comes from the detail response, so opening an occurrence costs no extra
request. Pages share the occurrence's query key, so every track edit that
refreshes the occurrence refreshes them too. Split and first-frame checks read
each frame's position in the whole track rather than on the page.

The names given across a track and the first and last frame now come from the
server, and track editing in the session view reads every frame from the
lightweight path payload instead of the occurrence detail.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C7Xf6VPbwWtTumhjjF15g8
Brings in the paginated occurrence detections endpoint that the frame strip reads.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C7Xf6VPbwWtTumhjjF15g8
mihow added a commit that referenced this pull request Oct 2, 2026
…e tracking rebase [skip ci]

Lists where every commit of the embeddings foundation and the results and
reviews branches now lives: in the post-processing results branch, in the
detection embeddings branch, or still to be carried by the tracking backend
(#1272) or the tracking UI (#1432). Also lists how #1272's migrations are
dropped and renumbered after the two new branches, the grouping review
writers the tracking backend must carry, and the conflicts to expect.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01L52AN9tabp76yjhjyCZkSJ
@mihow

mihow commented Oct 6, 2026

Copy link
Copy Markdown
Collaborator Author

Claude says: The merge order and plan for tracking, agreed with the owner today, are on #1412: #1412 (comment)

This PR's place: the review and correction interface comes back after #1469 as smaller PRs based on main (#1431, #1433), built from the work here and in #1441.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant