Skip to content

Feat: paginate jobs endpoint - #2619

Open
nhoening wants to merge 17 commits into
mainfrom
feat/paginate-jobs-endpoint
Open

nhoening wants to merge 17 commits into
mainfrom
feat/paginate-jobs-endpoint

Conversation

@nhoening

@nhoening nhoening commented Sep 29, 2026 •

Copy link
Copy Markdown
Member

Description

Closes #2617

  • api/v3_0: paginate the /asset/ID/jobs endpoint (cache Redis results for a minute), status page uses pagination
  • tests: cover paged jobs and status cache behavior
  • docs: describe asset jobs pagination and cache freshness
  • Added changelog item in documentation/changelog.rst

The long loading time probably stems from JSON manufacturing, but we also don't want to re-read all jobs from Redis on each page iteration (the order is also not always the same)

Look & Feel

image

Context: Large job histories make the asset status page slow or time out.

Change: Add opt-in job pagination, batch Redis retrieval, and cache status-page job lists for one minute.
Context: Job histories now use Redis batching and page-scoped enrichment.

Change: Verify legacy and paged responses, cache reuse, expiry, and invalidation.
Context: API clients and status-page users need the pagination and freshness contract.

Change: Document optional paging, response counts, and the one-minute cache window.
@nhoening nhoening self-assigned this Sep 29, 2026
Signed-off-by: Nicolas Höning <nicolas@seita.nl>
Context: CI exposed a cached job whose Redis record had been cleared between tests.

Change: Read the status loaded with the cached job rather than refreshing it from Redis.
Context: A CI test failed when a cached RQ job refreshed its missing Redis status.

Change: Verify job enrichment uses the cached status throughout the cache window.
Context: The status page lost Created At and Queue ordering when it moved to server-side pagination.

Change: Add only the query fields the endpoint uses, sort before page selection, restore table controls, and clarify cache eviction.
Context: The status table now orders Created At and Queue across all pages.

Change: Cover both queue directions and ascending enqueue time in the asset jobs API.
Context: The status table can order jobs by Created At and Queue across pages.

Change: Document the sort-by and sort-dir API parameters.
Context: The status cache duration and capacity were embedded in get calls and eviction logic.

Change: Define both limits on JobCache and let the status page opt into caching with a boolean.
Context: The cache lifetime now belongs to JobCache.

Change: Exercise expiry and invalidation through the explicit cache opt-in.
Signed-off-by: Nicolas Höning <nicolas@seita.nl>
Signed-off-by: Nicolas Höning <nicolas@seita.nl>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Paginate /jobs endpoint

1 participant