Stop every list endpoint from looking up permissions once per row #1430

Summary

Every list response in Antenna asks the database what the current user may do with each row, one row at a time, although the answer is the same for every row in the response. DefaultSerializer.get_permissions calls add_object_level_permissions, which calls BaseModel._get_object_perms, which resolves the row's project and then asks django-guardian for that user's permissions on it. Six serializers override that method; everything else inherits it.

#1428 removed this from the taxa lists endpoint, where it measured six queries per row: a ten-row page went from 37 queries to 8, and the page now costs the same whatever it holds. This ticket is about doing the same for the endpoints that still pay it, and about choosing a way to do it that does not need repeating by hand in every serializer.

Where it still happens

Measured since this ticket was opened, with a throwaway test as an ordinary project member (a superuser can short-circuit the permission path and hide the cost), and with the permission call patched out to separate its share from everything else on the page:

The effective default page size for both endpoints is 20, not the nominal 10: SourceImageUploadViewSet sets default_limit = 20 on the shared paginator class at import time, so every viewset that does not set its own inherits it. That is worth a separate look.

On the capture set list the query win is the cleanest of the three and the wall-clock win could not be demonstrated: that endpoint takes about seven seconds on a large project, dominated by count annotations over very large tables, and sixty indexed permission lookups disappear into that. Fix it for the query count, and treat the seven seconds as a separate, larger problem.

The rest of this section was counted from reading the code rather than measured:

Two serializers were checked and are not affected: the nested capture-taxon and classification serializers return an empty permission list without touching the database.

The part worth deciding first

#1428 fixed one endpoint by threading an already-resolved permission set into the many-to-many helper as an optional argument, and caching it on the serializer instance. That is correct there, but repeating it in four or five more serializers means the same six lines copied each time, and each copy is a chance to cache the wrong thing.

A direction worth considering instead: cache inside add_object_level_permissions / _get_object_perms themselves, keyed on the user and the project, on an object whose lifetime is one request. request.user is such an object — DRF builds it per request and the helper already receives it — so no call site would have to change at all, and every list endpoint would benefit at once. Points to settle before doing it that way:

• Whether anything reuses a User instance across what are logically separate requests (a management command or a Celery task looping over users would keep a stale cache), and whether to key the cache on the request instead when one is available.
• Whether permissions can change within a single request in a way that must be seen immediately, for example a view that grants a role and then serializes the affected rows.
• Whether the optional argument added in Speed up the Taxa Lists page & add a reusable way to look up permissions once per page #1428 should then be removed again, to leave one mechanism rather than two.

A trap to guard while doing this

#1428 also made the membership check in add_m2m_object_permissions read projects from the prefetch cache when the caller populated one. That is correct only if the prefetch is the unfiltered relation. A Prefetch("projects", queryset=Project.objects.filter(...)) anywhere would silently narrow the check and deny permissions the user really has. The constraint is currently implicit; it should be stated where the helper reads the cache, and any new prefetch of that relation checked against it.

What to verify

• Measure each endpoint before and after, as a project member who is not a superuser: superusers may short-circuit the permission path and hide the cost.
• None of these endpoints has a query-count test today, so a regression would not be noticed. Each fix should land with one, using a multi-row fixture and a strict count, in the shape of the taxa-list test added in Speed up the Taxa Lists page & add a reusable way to look up permissions once per page #1428. A fixture whose row count does not actually change between the two requests cannot see per-row growth, which is how an existing test on the taxa endpoint has been missing it.
• Check that permissions reported to the client are unchanged for: a project member, a member of another project, an anonymous user, and a superuser.

Depends on #1428.