Skip to content

feat(lifecycle): separate runtime and history retention - #1036

Merged
hrygo merged 9 commits into
mainfrom
codex/data-lifecycle
Oct 9, 2026
Merged

hrygo merged 9 commits into
mainfrom
codex/data-lifecycle

Conversation

@hrygo

@hrygo hrygo commented Oct 8, 2026 •

Copy link
Copy Markdown
Owner

Closes #1037
Closes #1038

Why

Idle Worker cleanup and conversation history retention need independent clocks. A conversation that no longer needs a running Worker should remain visible and resumable, while each stored data category follows an explicit retention policy.

What changes

  • Add configurable lifecycle v2 defaults: Worker idle release 60m, absolute turn timeout 30m, archive after 168h, conversation and per-message content retention 4320h (180 days).
  • Give delivery payloads, execution facts, audit facts, media cache, and ACP traces independent retention and cleanup behavior.
  • Add paired additive SQLite/PostgreSQL migrations, retention-aware history queries, lifecycle GC and observability.
  • Make deletion immediately fence reads/resume, then use durable asynchronous cleanup with queryable status and retry handling.
  • Add an Admin legacy-data migration preview/apply flow. Apply requires exact confirmation, works in bounded batches, and only extends deadlines.
  • Keep new audit facts bodyless by default; document backups and external platform messages as outside HotPlex deletion scope.
  • Update WebChat lifecycle cues and user/admin documentation, including a generated lifecycle diagram.

Review fixes

  • Renew migrated sessions from their persisted policy, including cached legacy sessions and older migration revisions. Bounded migrations converge after legacy content retention increases.
  • Keep mutable turn aggregation owned by the event forwarder during timeout handling.
  • Report missing Worker deletion capability as unsupported and purge status as blocked; preserve lease fencing and stop recreating completed or unsupported cleanup tasks.
  • Snapshot retention windows and policy revisions when delivery/execution records are created. GC uses settlement time plus the stored window; shorter settings affect subsequent records, and old rows without snapshots remain protected.
  • Renew the session before acknowledging first queue acceptance. Renewal failure cancels the queued input, cannot fall back to volatile buffering, and remains a failure on identical retries. Dispatched/settled duplicates return their real status.
  • Rebind queue budget, claim, settlement and depth SQL for PostgreSQL. Real-database regression coverage includes cancellation, clearing, expiry and atomic queue/runtime transitions.
  • Explicitly type effect settlement timestamps as BIGINT in completion and operator decisions, including requeue clearing the settlement clock.
  • Serialize Repairer shutdown: wait for the background processor before a bounded final drain, and share one teardown across concurrent callers. Timeout/cancellation cannot start a competing processor.

Compatibility and safety

  • session.retention_period remains the legacy runtime expiry setting; it is not the v2 history-retention setting.
  • Existing rows remain on their current policy until an explicit migration. Migration never shortens deadlines or recreates deleted content.
  • HotPlex deletion does not remove messages already sent to Slack/Feishu or rewrite existing backups; restoring an older backup can reintroduce deleted data.

Verification

  • Current repair passed make quality (zero lint issues and full race/shuffle tests), make build, make docs-lint, and focused lifecycle regression tests.
  • Deterministic session failure-safety regressions cover refused/failed retirement, durable renewal errors, deleted-session fencing and legacy acceptance. Local full session race/shuffle tests report 72.6% coverage against the unchanged 70% gate.
  • Channel-controlled Repairer shutdown regressions reproduced duplicate processing and the race before the fix; active-loop, drain-only, timeout and cancellation scenarios passed 10 consecutive race/shuffle rounds after the fix.
  • PostgreSQL regression covers ordinary/queued acceptance, claim/cancel/clear/expiry, queue depth, 90-day integer snapshots, delivery/attempt GC, operator settlement/requeue clocks, duplicate stability, and protected legacy rows. It compiles locally; execution against a disposable PostgreSQL 18 service is covered by the PR's PostgreSQL CI gate.
  • Final commit and current-head CI results are recorded in fix(lifecycle): address six retention and cleanup review regressions #1038. Git hooks are retained.
  • Production configuration/data, real platform or backup deletion, browser lifecycle E2E, and representative-load cleanup SLO have not been tested.

Give runtime resources, conversations, message bodies, delivery evidence, audit facts, media, and traces independent configurable deadlines. Preserve existing data behind an explicit preview/apply migration and make deletion cleanup durable and observable.
@hrygo
hrygo merged commit 3424039 into main Oct 9, 2026
19 checks passed
@hrygo
hrygo deleted the codex/data-lifecycle branch October 9, 2026 02:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix(lifecycle): address six retention and cleanup review regressions feat(lifecycle): separate runtime and history retention

2 participants