Skip to content

Store a suggested confidence threshold for each model #1465

Description

@mihow

Summary

Each classification model has a confidence score below which its predictions should not be trusted, and that score differs from model to model. Today Antenna has one score threshold per project (the project's default filters), so a project that uses two models, or switches models, either hides good predictions from one or shows weak predictions from the other. This ticket stores a suggested confidence threshold on each model, so default filters, class masking, exports and the interface can use the right threshold for whichever model produced a prediction.

What is needed

  1. Storage. A suggested score threshold on each classification algorithm (model), with where it came from: declared by the processing service, set by an administrator, or produced by a calibration run.
  2. Source. The processing service can declare it in its algorithm information (/info), the same way it declares the category map. Administrators can override it. A calibration or evaluation run can propose a value.
  3. Use. Default filters fall back to the model's suggested threshold when a project has not set its own. The interface shows the threshold that applies next to a prediction's score. Exports record it.
  4. Provenance. When the suggested threshold changes, earlier runs still record the value they used, since a job's settings are stored with the job.

Open questions

  • Whether the project threshold overrides the model's, or the model's overrides the project's, or the stricter of the two applies.
  • Whether thresholds can differ by taxon rank, for example a lower bar for a genus-level prediction from rank roll-ups ([Draft] Taxon rank classifications ("rank roll-ups") #857).
  • How the threshold interacts with class masking, which changes the scores the threshold is compared against.

Related

#1360 (offline evaluation of post-processing filters), #1316 (which taxa need verification), #857 (rank roll-ups), #1457 (post-processing outputs), #1461.

Activity

  1. added
    mlrelated to machine learning models or pipeline services
    on Oct 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    mlrelated to machine learning models or pipeline services

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions