Repository navigation
Conversation
Document the live Amplify report investigation and the publication/delivery refactor proposal. The observed bottleneck is transferring a 12.1 MB uncompressed HTML artifact directly from S3, while the viewer can report a misleading timeout before the transfer completes. Record controlled identity, gzip, and Brotli measurements using the unchanged JavaScript report, along with the limits of those observations and a standalone-report browser compatibility risk. Propose precompressed HTTP representations, explicit storage and publication invariants, and rollout verification while preserving the existing CSP hashes, iframe sandbox, audience isolation, and authorization on every open. Keep CDN adoption, additional security hardening, and content-format changes conditional on further evidence. Include authoritative references, migration and rollback considerations, and outstanding HTTPS negotiation and offline-download verification. No application or infrastructure implementation is included. Deploy: none
👷 Deploy request for agoracitizen pending review.Visit the deploys page to approve it
|
Precompress normalized project documents once at publication into Brotli quality 6 and gzip representations, retaining the original object and logical file metadata. Negotiate published representations from Accept-Encoding and keep browser transfers directly on S3 without API proxying or per-request compression. Add durable representation reservations, checksums, encoding metadata, explicit failure states, a nullable marker for incompressible files, and database constraints. Coordinate cleanup on the writer, tombstone objects before deletion, prevent interrupted publishers from reviving deleted representations, bound upload deadlines and cleanup work, and await all deletion attempts before returning. Include the generated Drizzle and Flyway migration, snapshot, journal entry, synchronized schema copies, and regenerated database test fixtures. Verify the migration in disposable PostgreSQL against existing document data, preserving file metadata and script permissions while checking foreign keys, active uniqueness, positive sizes, and deletion/failure invariants. No development or production database migration has been applied. Provide a dry-run-by-default documents:compress command for existing published files, with explicit --apply and scoped project/document selection. Preserve existing CSP bytes, iframe sandboxing, network restrictions, facilitator authorization, restricted download naming, and owner no-store behavior. Exempt public participant grants from the shared-IP quota so workshop attendees can open the same report simultaneously; retain the facilitator quota and authorize every grant. Replace the misleading 15-second viewer failure state with a non-blocking loading indicator and keep manual reload available during slow transfers. Remove the earlier investigation/design documents so the PR contains the implementation rather than a plan. Cover compression byte preservation, encoding negotiation, concurrent grants/publishers, cancellation, cleanup retry, legacy backfill, permission revocation, S3 metadata, and slow viewer behavior. Validation: 128 focused backend tests and 13 viewer tests pass; API/frontend typechecks, production TypeScript compilation, targeted lint/format checks, and the soft-delete policy pass. Local Chromium checks retain report interactions and byte-identical offline downloads; Safari/iOS and deployed HTTPS behavior remain rollout verification steps. Deploy: agora, api
Store exactly gzip and Brotli copies for new textual project documents without uploading an identity object. Prepare both encodings even for tiny or incompressible text so gzip is always available as the compatibility fallback. Keep PDFs as one unencoded object and preserve logical sizes, normalized HTML/CSP bytes, script permissions, authorization, and restricted-download behavior. Track the atomic compression cutover with compressed_at. Legacy documents remain readable through identity until both encoded copies are available; new grants then use encoded copies only. Retire legacy originals through the existing bounded API cleanup after the maximum seven-day signed-URL lifetime, without tombstoning the live document. Preserve original-deletion acknowledgements on failed document publication and use privacy-aware error reporting for deletion retries. Centralize upload, stale-reservation, and URL-expiry policy constants. Capture signing timestamps under a file share lock and pass them to the S3 presigner so grants issued before conversion cannot outlive the original-retention window. Prefer gzip when Accept-Encoding is absent, honor explicit encoding exclusions, and do not fall back to identity for converted files. Recover a successful publication when its database COMMIT acknowledgement is lost instead of invalidating the only live copies. Extend database constraints and cleanup indexing, synchronize canonical schema copies, and regenerate test fixtures. Undo the earlier unapplied migration generation and replace it with one final Drizzle/Flyway migration, snapshot, and journal entry. Do not introduce an additional follow-up migration. Verify the actual generated SQL against existing document data in disposable PostgreSQL, including unchanged file metadata and script permissions and the new storage-state constraints. No development or production database migration has been applied. Validation: 135 focused backend tests and 13 viewer tests pass, including encoded-only uploads, both upload-failure paths, gzip fallback, PDF handling, legacy original retirement and retry, concurrency, permission revocation, migration upgrade behavior, and lost-COMMIT recovery. API/frontend typechecks, production TypeScript compilation, targeted lint and formatting checks, git diff checks, and the soft-delete policy pass. Deploy: api
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Implement faster delivery of JavaScript-powered project documents without changing their rendering or security policy.
zlibdirectly in the API upload path, not a separate recurring process. The standalone command is only for converting existing files.Accept-Encoding; browsers download directly from S3, without API proxying or per-open recompression.documents:compresscommand for existing published files.The previous investigation/design documents are removed from the PR's final diff.
Database
Generated through
pnpm db:generate:services/api/database/flyway/V0115__equal_golden_guardian.sqlproject_document_representation, encoding/status enums, nullableproject_document_file.compressed_at, and an index for retiring legacy originals.The actual generated SQL was applied and tested only in disposable PostgreSQL. Existing file metadata and script permissions were preserved. No development or production database has been migrated.
Validation
make check-soft-deletes.Local measurements are not production guarantees. Safari/iOS and deployed HTTPS/S3/proxy behavior remain rollout checks. Public-grant IP-quota exemption is intentional; infrastructure-level abuse protection remains important.
Rollout
apiandagora.Deploy: agora, api