Skip to content

fix(domain): read a log traceback that psycopg returns as a dict - #1038

Merged
janbjorge merged 1 commit into
janbjorge:mainfrom
ericdelorefice:fix/log-traceback-parsed-dict
Oct 24, 2026
Merged

janbjorge merged 1 commit into
janbjorge:mainfrom
ericdelorefice:fix/log-traceback-parsed-dict

Conversation

@ericdelorefice

@ericdelorefice ericdelorefice commented Oct 24, 2026 •

Copy link
Copy Markdown
Contributor

Agentic

Summary

With PsycopgDriver, every read of a log row that carries a traceback raises TypeError: Expected bytes, bytearray or str. asyncpg returns the jsonb traceback column as JSON text. psycopg 3 returns it already parsed as a dict, and the BeforeValidator on Log.traceback calls from_json on it unconditionally.

I ran a real QueueManager on each driver against PostgreSQL 16.2, on main (9b8ab84) and on PyPI 1.6.0, nothing mocked. One job raises, one asks for a retry and then succeeds, one raises under on_failure="hold", so three log rows carry a traceback. Then:

                                     asyncpg    psycopg
Queries.queue_log()                  ok         TypeError: Expected bytes, bytearray or str
Queries.exception_logs()             ok         TypeError
Queries.job_log_history(id)          ok         TypeError
InsightsService.exception_logs()     ok         TypeError
InsightsService.job_history(id)      ok         TypeError
GET /failures                        200        500
GET /jobs/{id}                       200        500

The two dashboard rows are from main only: create_web_router mounted on a FastAPI app with app.state.pgq_queries = Queries(PsycopgDriver(conn)), the way the web dashboard docs describe mounting it in your own app, requested in-process through httpx's ASGI transport. create_web_app and the MCP server build their own asyncpg pool, so they are not affected. queue_log() fails as soon as the log holds one row with a traceback (a job that failed, was held or was retried); the other two fail when a row they return has one.

Job.headers is also jsonb and already takes both forms, with x if x is None or isinstance(x, dict) else from_json(x) from #793.

Change

The same check on Log.traceback in pgqueuer/domain/models.py, +2/-1 in one file (the line plus a comment saying why). Text and bytes still go through from_json, None stays None, and a dict goes straight to TracebackRecord validation. With the fix, every psycopg line above reads ok or 200.

Testing

  • uv sync --frozen && uv run ruff check . && uv run ruff format . --check && uv run lint-imports && uv run mypy . && uv run pytest passed (PostgreSQL 16.2 through EXTERNAL_POSTGRES_DSN, Python 3.12; mypy run as mypy . --no-incremental): 1528 passed, 10 skipped. On main: 1524 passed, 10 skipped (the 4 new cases are the difference).
  • Additional testing steps: test_drivers.py gets test_log_rows_with_a_traceback_read_back, parametrized over the three async drivers. It logs a job as exception with a traceback through log_jobs and reads it back through queue_log(), exception_logs() and job_log_history(). The psycopg case fails on main and passes with the fix; the asyncpg and pool cases pass on both. test_models.py gets a check next to the headers one that Log takes the traceback as a dict and as JSON text (fails on main).

Checklist

  • I have read the Contributing Guide
  • I have added or updated tests
  • I have updated documentation if necessary

asyncpg returns the jsonb traceback column as JSON text, psycopg 3 as
an already-parsed dict, and the Log.traceback validator passed it to
from_json unconditionally. Every log read through PsycopgDriver
(queue_log, exception_logs, job_log_history) raised TypeError once a
returned row had a traceback. Accept a dict as Job.headers already does.
@janbjorge
janbjorge merged commit 1ed405a into janbjorge:main Oct 24, 2026
21 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants