Summary
When an async job is started, it appears that the tasks get queued, and the worker sees the job, but the worker sees zero tasks to complete.
Detailed Description
In Antenna, I start a job and it appears that all tasks are queued successfully:
[2026-02-08 02:17:04] INFO Initialized task state tracking for 15 images
[2026-02-08 02:17:04] INFO Queuing 15 images to NATS stream for job '2225'
[2026-02-08 02:17:04] INFO Shuffling images
[2026-02-08 02:17:04] INFO Found 15 out of 31 images to process
In the worker, the job is seen in the API, but there are zero tasks to complete.
2026-02-08 07:19:28 [info ] Checking for jobs for pipeline quebec_vermont_moths_2023
2026-02-08 07:19:28 [info ] Processing job 2225 with pipeline quebec_vermont_moths_2023
2026-02-08 07:19:28 [info ] Worker 0/1 starting iteration for job 2225
2026-02-08 07:19:29 [info ] Worker 0: No more tasks for job 2225
2026-02-08 07:19:29 [info ] Worker 0: Iterator finished
2026-02-08 07:19:29 [info ] Done, detections: 0. Detecting time: 0.0, classification time: 0.0, dl time: 0.0, save time: 0.0
How to Reproduce the Bug
It would be helpful to expose the NATS dashboard for the live environment.
Environment
https://antenna.insectai.org/projects/18/jobs/2225
Steps to reproduce the behavior:
- Start a job, wait for the tasks to be queues
- View the logs for the worker service on one of the GPU worker VMs
Screenshots
Additional context
Add any other context about the problem here.
Summary
When an async job is started, it appears that the tasks get queued, and the worker sees the job, but the worker sees zero tasks to complete.
Detailed Description
In Antenna, I start a job and it appears that all tasks are queued successfully:
In the worker, the job is seen in the API, but there are zero tasks to complete.
How to Reproduce the Bug
It would be helpful to expose the NATS dashboard for the live environment.
Environment
https://antenna.insectai.org/projects/18/jobs/2225
Steps to reproduce the behavior:
Screenshots
Additional context
Add any other context about the problem here.