Skip to content

Keep a fixed list of occurrences as a set, and make one from the occurrences page - #1492

Draft
mohamedelabbas1996 wants to merge 5 commits into
mainfrom
feat/occurrence-sets
Draft

mohamedelabbas1996 wants to merge 5 commits into
mainfrom
feat/occurrence-sets

Conversation

@mohamedelabbas1996

@mohamedelabbas1996 mohamedelabbas1996 commented Oct 8, 2026 •

Copy link
Copy Markdown
Contributor

Summary

A project needs the same occurrences again later: comparing two classifiers only means
something if both saw the same rows, and a review pass wants the list it started from.
Describing that list as a filter answers differently as data arrives, so this stores the
membership instead.

An occurrence set is a fixed list of occurrences. It is created from the occurrences
page — select some rows, save them as a set — and the occurrence list can then be filtered
back down to it. Membership is decided when the set is created and nothing adds to or
removes from one afterwards.

This is split out of #1407, where sets are what a classifier is scored against. Nothing
here depends on that work, and scoring is not included.

List of Changes

# Change (effect) How (implementation)
1 A project can keep a fixed list of occurrences OccurrenceSet in ami/main/models.py: occurrences and projects, for_project() and visible_for_user() scoping; a set with no project is global, as TaxaList already does
2 Only the roles that curate a project's data can create one create_occurrenceset / update_ / delete_ on Project.Permissions, held by MLDataManager (and so ProjectManager); object-level backfill for existing projects in 0099
3 A set's contents cannot drift after it is made No add/remove endpoint; occurrence_ids is write-only and refused on update; no PUT, which would carry the occurrences
4 A global set cannot be edited through the API OccurrenceSet.get_project() returns the first project, and BaseModel.check_permission refuses every action without one
5 The occurrence list can be narrowed to one set occurrence_set on OccurrenceFilterSet, mapped to the evaluation_sets reverse relation
6 Sets are offered in pickers choices action on OccurrenceSetViewSet, following SourceImageCollectionViewSet.choices
7 Filter the occurrence list by set from the interface occurrence-set-filter.tsx + filter registry, dispatch and carry-over list, built like the capture-set filter
8 Save the selected occurrences as a set create-occurrence-set popover beside Export, built like suggest-id; useCreateOccurrenceSet hook

Screenshots

The occurrences page as it is today. Nothing is selected, so there is nothing to save.

Occurrences page with no rows selected

Selecting rows reveals Create set in the toolbar, beside Export. It is only there while
something is selected, because that is the only time it does anything.

Three rows selected and the Create set button in the toolbar

Naming the set. The line under the field says how many occurrences it will hold and that
the contents will not change afterwards.

The create-set popover with a name typed in

The set then appears in the occurrence filters, under More filters beside Capture set.

The occurrence set filter open in the filter panel

Choosing it narrows the list to exactly the occurrences the set holds.

The occurrence list filtered down to the three occurrences in the set

Related Issues

Split out of #1407 (retraining and scoring). The scoring side will consume these sets.

Detailed Description

  • Why membership is stored, not filtered. Anything recorded against a set was measured
    on exactly those occurrences. A set that grows quietly makes an old number mean something
    different, with nothing on screen to say so.
  • Permissions. Creating is checked against the project named in the payload, because no
    object exists yet; everything else goes through the object check. Saving a set changes no
    occurrence, so it is not behind update rights on them — which is why the button sits in
    the toolbar rather than in the selection bar, where the identification actions live.
  • Test data is built directly rather than through ami/tests/fixtures/. create_captures
    times its images from datetime.now(), and event grouping reads image dimensions from
    storage. Neither is deterministic in a test, so occurrences built that way made unrelated
    assertions fail at random, differently each run. make_occurrence in the test file builds
    the detection and determination the list endpoint needs. This is a deviation from
    canonical-patterns.md and worth a look.

Testing

  • ami.main.test_occurrence_sets — 16 tests, stable across repeated runs: creating,
    permissions, membership immutability, global sets, listing scope, and the filter.
  • Driven through the interface end to end against a seeded project: the button is absent
    until rows are selected, the popover refuses an empty name, the set saves with exactly the
    rows chosen, the filter narrows the list to them, and another project cannot see the set.
  • UI typecheck, prettier and eslint clean.

A project needs the same occurrences again later: comparing two classifiers only
means something if both saw the same rows, and a review pass wants the list it
started from. A filter answers differently as data arrives, so the membership is
stored rather than described.

An OccurrenceSet holds its occurrences and the projects it belongs to. A set with
no project is global and is offered everywhere, following how TaxaList already
treats a list with no project. Membership is decided when the set is created and
nothing adds to or removes from one afterwards, because anything recorded against
a set was measured on exactly those occurrences.

Creating, renaming and deleting are gated on new project permissions, held by the
roles that already curate a project's data. A global set has no single project to
check against, so it cannot be edited through the API at all.

The occurrence list takes an occurrence_set filter, and the sets are offered as
choices the same way capture sets are.
Adds the occurrence set to the occurrence filter panel, picked from the set
choices endpoint the same way a capture set is. A set is only useful if you can
look at what is in it, and this is where someone reviewing one starts.

The field is carried over from other views like the existing filters, so arriving
with a set already chosen shows it in the panel where it can be cleared.
Someone filtering the occurrence list to the rows they care about had no way to
keep that selection. Selecting occurrences now offers saving them as a set, next
to the identification actions already there.

The action does not change any occurrence, so it is not behind update rights on
them; the endpoint gates it on the project's own permission instead.
Registering a filter in the shared list is not enough for it to appear: the
occurrences page renders one FilterControl per field it offers, and the set was
missing from that list, so the filter existed everywhere except on screen.

It sits under More filters beside the capture set, and that section now opens on
arrival when a set is already applied, as it does for the other filters there.
The selection bar holds identification actions and is hidden from anyone without
update rights on the occurrences, so saving a set — which changes none of them —
was unavailable to a reader who could still create one. It also sat there as an
unlabelled icon among three others.

It now sits beside Export as a labelled button, and appears only while something
is selected, since that is the only time it does anything.
@netlify

netlify Bot commented Oct 8, 2026 •

Copy link
Copy Markdown

✅ Deploy Preview for antenna-preview ready!

Name Link
🔨 Latest commit c731d37
🔍 Latest deploy log https://app.netlify.com/projects/antenna-preview/deploys/6ac808f8f222af00086f3928
😎 Deploy Preview https://deploy-preview-1492--antenna-preview.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.
Lighthouse
Lighthouse
1 paths audited
Performance: 56 (🔴 down 9 from production)
Accessibility: 81 (🔴 down 8 from production)
Best Practices: 92 (🔴 down 8 from production)
SEO: 92 (no change from production)
PWA: 80 (no change from production)
View the detailed breakdown and full score reports
🤖 Make changes Run an agent on this branch

To edit notification comments on pull requests, go to your Netlify project configuration.

@coderabbitai

coderabbitai Bot commented Oct 8, 2026

Copy link
Copy Markdown
Contributor

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

  • Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true
  • Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@netlify

netlify Bot commented Oct 8, 2026 •

Copy link
Copy Markdown

✅ Deploy Preview for antenna-ssec canceled.

Name Link
🔨 Latest commit c731d37
🔍 Latest deploy log https://app.netlify.com/projects/antenna-ssec/deploys/6ac808f8fe888b0008aa3c58

@mihow

mihow commented Oct 8, 2026

Copy link
Copy Markdown
Collaborator

Great work!

  1. I like the simple method of adding a manual selection of occurrences to a set. Let's call these manually curated sets, like we do with the Capture Sets which are populated by a specific list of occurrences (rather than a snapshot from an interval sampler or dynamic capture set). I think favorites and flagged occurrences can also be manual occurrence sets.
  2. Does this interfere with the bulk ID tool? When you select multiple occurrences, a toolbar usually appears with identification tools
  3. Dynamic sets & sets populated from a query: I would like to implement creating occurrence sets by saving the currently selected filters in the occurrence list view. The "Create Set" button could always be visible, but it would be populated by serializing the current filter values. This is mostly already supported on the backend! And in our export framework! You can export Occurrences based on serialized filter params (thank you Mohamed). I am okay to make these dynamic by default, but then the user should have an option to make a static version where the query is run once and it becomes a static snapshot. Dyanmic sets could be a follow up PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants