Skip to content

PSv2: add Job relationship to a Classification and Detection objects #1156

Description

@mihow

Summary

Classifications & detection objects contain the history of results & which algorithms produced them, but not which job they came from (or which pipeline).

Implementation Details

Adding a simple job_id foreign key to Classification & Detection models would be very helpful when we need to inspect that relationship. For now we can just save the job value when processing results, and return it in the API, but it does not need to be utilized in any sort of rich way. However in the future this could be used for counting objects created by a job, or filtering.

Activity

  1. mihow commented on Jun 8, 2026

    @mihow
    CollaboratorAuthor

    Claude says: Adding a use case that makes this more valuable, plus two adjacent gaps it surfaced.

    Use case — surfacing post-processing provenance in the UI. The post-processing framework (the admin-triggered filters: small-size filter, occurrence tracking, class masking) creates and modifies Classifications. Operators want to see, in the UI for a given classification, which job produced or modified it and with what configuration — for example, the size threshold a filter ran with, or the cost threshold a tracking run used. The Job foreign key proposed here is the key enabler: once a classification links to its producing Job, we can expose job.params (the run config) and the job's stage metrics in the classification detail view. So beyond counting/filtering objects by job, this directly unlocks human-readable provenance in the UI.

    Adjacent gap 1 — Classification has no flexible/JSON field. Pipelines are currently limited to the fixed Classification columns for what they can persist alongside a result. A JSON field would let pipelines (and post-processing tasks) attach arbitrary structured detail without a schema migration each time.

    Adjacent gap 2 — Classification has no notes/comment field. Identifications have a comment field for humans, but Classifications have no equivalent. There is nowhere to record per-classification transformation notes — for example, "downgraded by the small-size filter," or "no taxon exists for genus X, rolled up to family." A free-text notes field (or the JSON field above) would give these somewhere to live.

    Suggested scoping: keep this ticket focused on the Job FK. The flexible JSON field and the notes field are related but distinct — happy to file a separate ticket for those, or fold them in here, whichever you prefer.

  2. added
    PSv2Async & distributed ML backend (PSv2): job state, NATS dispatch, result handling. Umbrella #515.
    post-processingPost-processing task framework and tasks: size filter, class masking, rank rollup, tracking
    Pipeline APIUpdates to the requests & responses to/from processing service workers for ML pipelines
    on Jun 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    PSv2Async & distributed ML backend (PSv2): job state, NATS dispatch, result handling. Umbrella #515.Pipeline APIUpdates to the requests & responses to/from processing service workers for ML pipelinesVISS-SSECdata-processingpost-processingPost-processing task framework and tasks: size filter, class masking, rank rollup, tracking

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions