Skip to content

fix(qm): keep job ends when the shutdown flush fails - #1034

Closed
janbjorge wants to merge 1 commit into
mainfrom
fix/keep-unwritten-job-logs
Closed

janbjorge wants to merge 1 commit into
mainfrom
fix/keep-unwritten-job-logs

Conversation

@janbjorge

Copy link
Copy Markdown
Owner

Fixes #1027.

Summary

  • A shutdown flush that fails keeps the job ends on QueueManager.unwritten_job_logs instead of dropping them. Copy the list off before the next run, and write it with log_jobs on a live connection. A write after another worker owns the job is ignored.
  • pgq run --restart-on-failure writes those ends on the next cycle's connection before it dequeues. A new process does not have the list, so those jobs still run again, as ADR-0004 allows.

Testing

  • uv sync --frozen && uv run ruff check . && uv run ruff format . --check && uv run lint-imports && uv run mypy . && uv run pytest passed
  • test_run_keeps_job_end_when_shutdown_flush_fails keeps a successful end when the exit flush raises, and a later log_jobs removes the row
  • test_restart_writes_job_end_the_first_run_could_not writes that end on the restarted connection, and the handler does not run a second time
  • uv run ruff check on the touched files, uv run mypy on the touched files, and uv run lint-imports passed

Checklist

  • I have read the Contributing Guide
  • I have added or updated tests
  • I have updated documentation if necessary

A dead connection drops the status write, so a finished job stays
picked and runs again. The ends now stay on the manager, and an
in-process restart writes them on the next connection.

Fixes #1027.
@janbjorge janbjorge added the Agentic Issue or PR written mainly by an AI agent. Required on all agent-authored contributions. label Oct 24, 2026
@janbjorge

Copy link
Copy Markdown
Owner Author

Superseded by two stacked PRs:

@janbjorge janbjorge closed this Oct 24, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Agentic Issue or PR written mainly by an AI agent. Required on all agent-authored contributions.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Job ends that cannot be written when QueueManager.run exits are dropped, so a finished job runs again

1 participant