Repository navigation
fix: recover imports, contacts, and app startup - #1297
Merged
Merged
Conversation
Complete imports when validated conversation data and counters are fully saved, then run opinion-group analysis in the background. This breaks the circular dependency where imports waited for analysis while the math worker rejected importing conversations. Scope active-import admission to CSV and URL imports so cached query data cannot disable ordinary conversation creation. Recover fully saved legacy imports before timeout cleanup, and publish new imports atomically with their completion notification before queueing analysis. Fix generated Pydantic datetime and UUID constraints at the schema-export boundary. Preserve required nullable event fields while omitting absent optional fields so the API accepts worker notifications. Apply a shared one-hour queueing and ingestion deadline, with owner-scoped heartbeats and API-side recovery when the worker is unavailable. Run bounded, non-blocking maintenance while ingestion slots are occupied; distinguish queued work from abandoned ingestion and protect saved imports across recovery batches. Guard terminal states, duplicate deliveries, cross-user updates, and cleanup after a lost commit acknowledgement. Keep normal polling read-only, use typed Drizzle and generated SQLAlchemy queries, derive request types from Zod, and remove obsolete analysis-gating and result plumbing. Add database-backed lifecycle, stale-replica, ownership, queue-privacy, query-count, saturation, and frontend cache regression coverage. Document diagnosis, recovery guarantees, and the worker-first rollout order for existing overdue imports. Deploy: import-worker, api, agora
Allow a contact to identify a person by first or last name, or use a selected organization without a personal name. Enforce identity and nonblank email-or-website requirements in the API and database, and generate the V0113 migration with synchronized schema fixtures. Display the organization as the primary contact name when no person is named, without repeating it below. Reuse organization logos, preserve image proportions and explicit contact image overrides, and retain the initials fallback across desktop and mobile. Consolidate contact form parsing and backend persistence. Normalize contact values on create/update using the existing email normalization helper, preserve stored values during reads, and always reset the save loading state. Refresh OpenAPI and both clients through normal generation. Cover the contact contract, form behavior, display logic, schema serialization, logo inheritance, and database insert/update constraints with focused tests. Deploy: agora, api
Quasar skips failed boot-module imports and can mount App.vue without Vue Query, causing useRealtimeSSE to throw before the application initializes. Coordinate required boot modules explicitly and render the application only after every required initializer succeeds. Load boot modules concurrently, preserve initialization order, and keep telemetry failures nonfatal. Statically import the coordinator from the root component so it cannot become another independently skipped boot chunk. Track completion per Vue app with weak references and route initialization failures through the existing chunk-recovery and privacy-aware error handler while allowing the recovery screen to mount. Add a standalone recovery screen with all eleven display languages, RTL support, theme styling, keyboard accessibility, and a native reload action. Keep recovery usable when browser storage is unavailable, and update the Vue Query boot file to the current Quasar wrapper. Cover shared QueryClient injection, per-app isolation, required module and initializer failures, optional telemetry failures, environment selection, localized recovery, focus management, and unavailable storage. Verify all 1,669 frontend tests, TypeScript, focused lint and formatting, a production-mode build, and browser-injected startup and navigation failures. Deploy: agora
✅ Deploy Preview for agoracitizen canceled.
|
Keep the detailed deployment and recovery runbook in the pull request description rather than a checked-in incident document. Remove the README link while retaining the essential worker-first rollout guidance. Deploy: none
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This combines three reviewed fixes:
92a516a1): stop active imports from blocking ordinary conversation creation; break the circular dependency between import publication and opinion-group analysis; repair notification code generation/serialization; add bounded, owner-scoped recovery and API-enforced deadlines.3243f341): support a person identified by first or last name, or a selected organization without a person name; enforce identity/contact-channel invariants; inherit organization logos; consolidate form/persistence handling and regenerate the API clients.e75b7eae): coordinate required boot modules before mounting the application, keep telemetry failures nonfatal, and provide a localized, accessible recovery screen instead of partially starting without required plugins.Import completion now means validated conversation data and counters are fully saved and published. Analysis is queued after that transaction commits. Fully saved legacy imports can recover without waiting for analysis; partial/abandoned imports fail rather than blocking the user indefinitely. Deployment and recovery guidance is included below.
Verification on the combined branch
pnpm test.git diff --checkpass.DROP TABLEinV0112__parallel_puppet_master.sql. V0113 in this PR changes contact nullability/check constraints, not application-row deletion.What must be deployed
project_contact.first_nameand enforces identity/nonblank contact-channel checksservices/import-worker/ Composeimport-workerservices/api/ Composeapiservices/agora/ ComposeappThe Svelte landing page (
services/app) and the other worker images do not need deployment for this PR. Keep the existing math updater running and verify it is healthy: it performs background analysis after publication. The email worker already coalesces absent person names and falls back to the project title, so nullable contact first names do not require an email-worker redeploy.There are no new runtime environment variables. The import deadline is the shared one-hour policy constant. Existing production secrets, database connections, Valkey configuration, and worker settings remain applicable.
Ordered rollout
This sequence assumes production is already on the current
mainbaseline. Check Flyway history first: if earlier migrations are still pending, their independent rollout requirements also apply. Do not treat an unrestricted migration from an older production baseline as only a V0113 upgrade.API_IMAGE_TAGandAGORA_IMAGE_TAG, but hardcodes the import worker tolatest; update the real worker image reference explicitly. Use the actual production Compose/environment rather than the checked-in placeholder settings.pnpm db:migratefrom the production-configured API release checkout). Migration is a separate operation: the API runtime image neither ships the migration directory nor automatically runs Flyway. PostgreSQL/Flyway's normal transactional execution keeps this four-statement change atomic; plan for its table lock/constraint validation.import-workerinitially. Let recovery drain fully saved legacy imports still marked processing before moving on: recovery is batched at 100 records, so one pass is not necessarily enough. Background analysis can then run on the published conversations.app), then test/reload the proxy after its container replacement. Preserve theagora-assetspersistent volume: the existing entrypoint merges release assets and retains old hashed chunks for open tabs. Avoid a volume-removing teardown. Fresh page loads receive the new startup gate; already-open tabs need a refresh to get the startup fix and the new contact rendering contract.For a Compose deployment, use service-scoped
pull/up -d --no-depssteps in the order above, with recovery/readiness checks between them. Do not rely onscripts/infra/update-agora.shto implement this ordered rollout: it pulls only nginx and runs a bulkup -d. Explicitly pull the new API/frontend/import-worker images, stage their replacement, and use the existingnginx -t/nginx -s reloadprocedure after container changes.Post-rollout checks
Rollback considerations
first_name = NULL. A pre-PR API/frontend expects a first name, and restoringNOT NULLcan fail. Keep the expanded schema and a contact-compatible API/frontend, or reconcile the affected data before a pre-PR rollback; do not blindly undo the migration.Deploy: import-worker, api, agora