Let taxa lists and processing services be shared with every project, with safe rules for who can edit them

[mihow]

Summary

Some things in Antenna are meant to be shared by every project, such as a reference species list, while others belong to one project. Until now there was no way to say which was which. A taxa list with no project was treated as shared by convention, but in practice nobody except a superuser could see it, so the shared lists that already exist in the database (imported reference lists, for example) never appeared anywhere in the app.

This PR gives taxa lists and processing services an explicit "public" setting. A public list or service shows up inside every project alongside the project's own, clearly marked, and only people with a specific platform-level permission can change it, so a member of one project cannot edit something every other project relies on. Nothing changes for project-owned lists and services.

It is the first step of #1424, which uses public taxa lists to show what each classifier can predict and lets a project copy such a list and curate it. It also lays the ground for platform-level roles such as a taxonomy curator, who would be granted the new permission without being a superuser.

List of Changes

#	What changes for users and operators	How
1	A taxa list or processing service can be marked public, in the admin	New is_public field, declared once on an abstract PublicScopedModel base that TaxaList and ProcessingService inherit; shown and filterable in the admin; read-only in the API
2	Public taxa lists and services appear inside every project, next to the project's own	New shared queryset method for_project(project, include_public=True); both viewsets use it
3	A client can ask the list endpoints for only the project's own rows	New include_public query parameter on the list actions only, default true; an invalid value returns 400; detail, update and delete routes always resolve public rows
4	Public rows are visible to everyone, including people who are not logged in; private rows in draft projects stay hidden, and a public row never reveals the ids of draft projects the viewer cannot see	visible_for_user() treats public rows as visible; the projects field on both serializers is filtered to projects visible to the requester, computed once per request
5	Only people holding a platform permission can edit or delete a public list, change its species, edit a public service, or register its pipelines. Being a member of a project, or staff, is not enough	New model permissions main.manage_public_taxalist and ml.manage_public_processingservice, checked without reference to any project; superusers pass
6	On a public row, user_permissions reports update and delete only to holders of the platform permission	add_m2m_object_permissions short-circuits on public rows. For project-owned rows nothing changes: user_permissions still comes from guardian role permissions while the write gate for taxa lists is plain membership, so a basic member can edit through the API but sees no buttons; that pre-existing gap is #1120's and one test now pins it so any change is deliberate
7	Existing shared taxa lists become public; lists of "taxa returned by" an algorithm stay hidden as they are today	Data backfill in the taxa list migration; no backfill for processing services
8	Reference lists created by the taxa import and update commands are public	get_or_create_for_project(..., is_public=True)
9	A person who is not a member of a draft project can no longer reach that project's taxa lists through the add/remove-species route	The nested route applies visible_for_user()
10	"Test connection" and "Register pipelines" keep working from the UI, and now respect visibility and the rules above	Both actions use get_object() instead of a raw id lookup; project_id stays optional for these two actions because the frontend calls them without it

Related Issues

Relates to #1424 (step A of its order of work). Overlaps with #1120 (permissions for things that belong to several projects) and #1081 (only administrators should be able to make a list shared by all projects).

Detailed Description

Why an explicit field. Treating "has no project" as "public" carries a silent risk: deleting a project removes its links, so a private list whose only project is deleted would become visible to everyone. With an explicit field, a row that is neither public nor linked to a project is an orphan that only superusers see.

Why the word "public". In this domain "global" already means geographic scope (the global moth classifier, global species lists). New code and comments say "public"; comments that said "global list" were reworded where touched.

Reading. BaseQuerySet.for_project() tests project membership with an Exists subquery on the many-to-many table rather than a join. A join combined with OR is_public would return a public row once per linked project and force a .distinct(). One test pins that a public list linked to several projects appears once.

Writing. The permission check happens at the object, not only at the request. An earlier shape of this change let a holder of the platform permission pass the request-level check and then edit a project-owned list in a project they do not belong to; the object-level check re-verifies membership for every non-public row, and two tests cover it. The add/remove-species route never calls DRF's object permission hook, so it checks explicitly. Creating through the API always produces a project-owned row.

Processing services. Writes on project-owned services stay staff-only, exactly as before; redesigning that belongs to #1120. The only new rule is that a public service needs the platform permission and staff status alone is refused.

Not in this PR. A public processing service is visible but not yet used to run jobs: job dispatch, the async worker heartbeat and count, the pipeline views' prefetch of services, and the check that gives a new project a default service all still filter services by project. These are listed in #1424 and come with the next step. No frontend changes are included; the API adds fields and a parameter and removes nothing.

Known and unchanged. The taxa list endpoint runs queries per row (measured: 34 queries for 5 rows, 58 for 10). That predates this PR and is fixed separately in #1428; the strict query-count test here pins today's number (35 for the 5-row fixture, one more than before for the once-per-request visible-projects lookup) and will be lowered when #1428 merges. The processing service list endpoint dropped from 34 to 21 queries for 5 rows in review, by not running guardian lookups that can never match (no *_processingservice permission exists).

Permission classes. One base class parameterised by the non-public fallback (project membership for taxa lists, active staff for processing services) backs both viewsets; the previous class names remain as thin subclasses.

Two questions this raises for the other shared models. is_public answers "who may see this row"; a second question is "who owns its contents", and the two are independent. Follow-up work on this feature adds a TaxaList.is_managed for the second: a list an algorithm points at mirrors that classifier's category map, so the API refuses hand edits to it and the user copies it instead. The same pair looks worth having elsewhere. Pipelines and algorithms are created by registration from what a processing service advertises, and are then edited by hand in the admin with nothing recording that the next registration may overwrite the edit. The default processing service is created from settings rather than by a person. Taxa created from classifier labels carry only a name and a rank, and nothing marks them as awaiting a curator. Declaring both flags on the shared base, with the same read-only serializer fields, would let a client label and lock those rows the same way everywhere instead of each page guessing. That is a direction to discuss rather than something to add here, and it belongs with #1120 and the platform roles it describes.

How to Test the Changes

Automated, run locally against the CI compose stack: the taxa list and processing service test classes pass on the final commit (86 across the matrices, re-run by the author after the structural review). The full ami.main (424 tests) and ami.ml (199 tests) modules passed during development, along with makemigrations --check --dry-run, black, isort and flake8; CI on this PR is the authoritative run. Coverage includes a permission matrix for each model (project member, authenticated non-member, anonymous, superuser, staff, and a plain user granted the platform permission) against a public and a project-owned row, for list, retrieve, update, delete, add and remove species, connection test and pipeline registration.

Manual:

1. In the admin, mark a taxa list public. Open any project's Taxa Lists page: the list appears. ?include_public=false hides it; ?include_public=abc returns 400.
2. As a member of that project who is not a superuser, try to rename the public list or add a species through the API: 403. Rename one of the project's own lists: allowed.
3. Grant a plain user "Can create, edit and delete public taxa lists" in the admin and repeat step 2 on the public list: allowed. The same user still cannot edit a list owned by a project they do not belong to.
4. On a processing service page, "Test connection" and "Register pipelines" still work.

Deployment Notes

Two migrations: main/0096 (adds the field and permission, makes a taxa list's projects optional, backfills) and ml/0029 (adds the field and permission, no backfill). Both are small and quick. After deploying, every taxa list that had no project becomes public and therefore visible in every project, except lists whose name starts with "Taxa returned by". It is worth looking at that set before deploying (TaxaList.objects.filter(projects__isnull=True)) in case any should be attached to a project instead. No processing service becomes public until someone marks it so. The reverse migration drops the field and does not restore anything else.

Checklist

• Tests added, including a permission matrix per model and strict query counts with multi-row fixtures
• Migrations included; makemigrations --check --dry-run is clean
• Query parameter validated; invalid input returns 400
• Frontend: label public rows and hide edit controls (follow-up step in Classifier species lists as taxa lists and support for public lists #1424)

🤖 Generated with Claude Code

https://claude.ai/code/session_01TngW6AshkUEhNp9D6z3U9Z

[netlify]

✅ Deploy Preview for antenna-preview canceled.

Name	Link
🔨 Latest commit	083913f
🔍 Latest deploy log	https://app.netlify.com/projects/antenna-preview/deploys/6ac582b8a59f2d0008f1b3c6

[netlify]

✅ Deploy Preview for antenna-ssec canceled.

Name	Link
🔨 Latest commit	083913f
🔍 Latest deploy log	https://app.netlify.com/projects/antenna-ssec/deploys/6ac582b8e775510008703a7c

[coderabbitai]

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

• Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true

• Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}