google drive connection and bug fix

.github/workflows/backend-tests.yml

@@ -0,0 +1,36 @@
+name: Backend tests
+
+on:
+  push:
+    branches: [main, master]
+  pull_request:
+
+permissions:
+  contents: read
+
+jobs:
+  test:
+    runs-on: ubuntu-latest
+    defaults:
+      run:
+        working-directory: backend
+
+    steps:
+      - name: Check out repository
+        uses: actions/checkout@v4
+
+      - name: Set up Node.js
+        uses: actions/setup-node@v4
+        with:
+          node-version: 22
+          cache: npm
+          cache-dependency-path: backend/package-lock.json
+
+      - name: Pin npm version
+        run: npm install --global npm@10
+
+      - name: Install dependencies
+        run: npm ci
+
+      - name: Run backend integration tests
+        run: npm test

backend/.env.example

@@ -27,9 +27,14 @@ ADMIN_PASSWORD=
 ADMIN_FIRST_NAME=Admin
 ADMIN_SURNAME=Signpak
 
-# Where uploaded files go. Only "local" exists today; a Google Drive driver plugs in here later.
+# Where uploaded files go. For Google Drive, share the target folder with the service account email.
 STORAGE_DRIVER=local
+# Submission recordings are append-only archives. Set this to gdrive for Drive storage.
+ARCHIVE_STORAGE_DRIVER=local
 UPLOAD_DIR=uploads
+# Required when STORAGE_DRIVER=gdrive. Store the whole service-account JSON as one environment value.
+# GOOGLE_SERVICE_ACCOUNT_JSON={"type":"service_account",...}
+# GOOGLE_DRIVE_FOLDER_ID=your-shared-drive-folder-id
 MAX_VIDEO_UPLOAD_MB=300
 MAX_RECORDING_UPLOAD_MB=100
 SUBMISSION_COOLDOWN_MS=30000

backend/README.md

@@ -1,6 +1,6 @@
 # SignPakCommons: backend
 
-Express + Node.js + MongoDB (Mongoose). Cookie-based auth, local file storage (Google Drive ready), layered architecture.
+Express + Node.js + MongoDB (Mongoose). Cookie-based auth, local base-video storage, optional append-only Google Drive recording archives, layered architecture.
 
 ## Run it
 
@@ -67,7 +67,7 @@ Codes: `BAD_REQUEST` 400, `UNAUTHORIZED` 401, `FORBIDDEN` 403, `NOT_FOUND` 404,
 | `PATCH`/`DELETE /videos/:id` | admin | Moving to another category appends to the end of that path |
 | `GET /submissions` | logged in | Contributors: their own. Admin: all |
 | `POST /submissions` | logged in | `multipart/form-data`: `recording`, `videoId`, `trimStart`, `trimEnd`, `mirrored`, `duration` |
-| `GET /submissions/:id/recording` | admin | Contributors can never play a submission back |
+| `GET /submissions/:id/recording` | none | Deliberately unavailable; submissions are append-only archives |
 | `POST /contact` | anyone | Rate limited: 5 per hour per IP |
 | `GET /admin/users`, `/admin/stats?days=14`, `/admin/messages` | admin | |
 
@@ -79,11 +79,11 @@ Codes: `BAD_REQUEST` 400, `UNAUTHORIZED` 401, `FORBIDDEN` 403, `NOT_FOUND` 404,
 
 **Admin.** Created or promoted from `ADMIN_EMAIL` / `ADMIN_PASSWORD` on every start. Signup ignores any `role` in the body. Changing `ADMIN_PASSWORD` later does not reset an existing admin's password.
 
-**The cooldown rule.** This is a data-collection tool, not a one-shot quiz: a contributor can submit the same video more than once (more varied takes make better training data), but not back to back. `SUBMISSION_COOLDOWN_MS` (default 30s, in `.env`) is the minimum gap between two submissions for the same `(user, video)`. It is enforced atomically, not by a simple read-then-write: `repositories/submissionRepo.js#claimCooldown` does a conditional `findOneAndUpdate` with `upsert: true` against a `SubmissionCooldown` collection that has a unique index on `(user, video)`. If a live cooldown already exists, the upsert's insert path collides with that unique index (`E11000`), which is treated as "still cooling down" — so three simultaneous submissions for the same video still produce exactly one success (verified: `tests/api.test.js`'s concurrent-submit test). A submission that fails after claiming the cooldown (a storage or DB error) releases it, so a failed attempt never costs the contributor their next try. Regardless of timing, no submission can ever be edited or deleted once made, and only an admin can watch one back.
+**The cooldown rule.** This is a data-collection tool, not a one-shot quiz: a contributor can submit the same video more than once (more varied takes make better training data), but not back to back. `SUBMISSION_COOLDOWN_MS` (default 30s, in `.env`) is the minimum gap between two submissions for the same `(user, video)`. It is enforced atomically, not by a simple read-then-write: `repositories/submissionRepo.js#claimCooldown` does a conditional `findOneAndUpdate` with `upsert: true` against a `SubmissionCooldown` collection that has a unique index on `(user, video)`. If a live cooldown already exists, the upsert's insert path collides with that unique index (`E11000`), which is treated as "still cooling down" — so three simultaneous submissions for the same video still produce exactly one success (verified: `tests/api.test.js`'s concurrent-submit test). A submission that fails after claiming the cooldown (a storage or DB error) releases it, so a failed attempt never costs the contributor their next try. Regardless of timing, no submission can ever be edited, deleted, or read back once archived. Admins receive archive metadata only.
 
 **Uploads.** Multer writes to `uploads/tmp`, the service checks the file's real type from its first bytes (MP4, WebM or MOV; JPEG, PNG or WebP for posters; the browser-supplied type is not trusted), then hands it to storage. Failures at any step leave no files behind.
 
-**Storage and Google Drive.** MongoDB stores only `{ driver, key, mimeType, size }`. `services/storage/localDriver.js` documents a four-method contract (`save`, `stat`, `createReadStream`, `remove`). To add Google Drive: write `googleDriveDriver.js` with those four methods (Drive's `files.get` with `alt=media` accepts Range headers), register it in `services/storage/index.js`, and allow `gdrive` in `STORAGE_DRIVER`. Files already stored keep working, because each one remembers which driver holds it.
+**Storage and Google Drive.** Base videos use the readable storage driver selected by `STORAGE_DRIVER`. Submission recordings use the separate append-only driver selected by `ARCHIVE_STORAGE_DRIVER`; set it to `gdrive` with `GOOGLE_SERVICE_ACCOUNT_JSON` and `GOOGLE_DRIVE_FOLDER_ID`. MongoDB stores the archive pointer and metadata, while the API never reads, updates, or deletes the archived recording. Existing file records keep working because each one remembers its driver.
 
 **Stats.** Per-category counts only include submissions whose video still exists. Totals and daily activity always count everything. Days are UTC. Per-day bucketing is done in code from a timestamps-only query; if submissions reach hundreds of thousands, move it to a MongoDB aggregation.
 

backend/docs/API.md

@@ -55,7 +55,7 @@ Errors use a structured object suitable for form-level and field-level messages:
 | GET | `/videos/:id/poster` | Public when published | Stream a published poster. |
 | POST/PATCH/DELETE | `/videos`, `/videos/:id` | Admin | Upload and manage base videos. |
 | GET/POST | `/submissions` | User | List or create contributor recordings. |
-| GET | `/submissions/:id/recording` | Admin | Stream a contributor recording. |
+| GET | `/submissions/:id/recording` | None | Deliberately unavailable; submission archives are append-only. |
 | POST | `/contact` | Public | Send a contact message. |
 | GET | `/admin/users` | Admin | List users. |
 | GET | `/admin/stats` | Admin | Read dashboard statistics. |
@@ -72,5 +72,6 @@ The server validates file signatures, size limits, metadata, and route IDs. Clie
 - Anonymous users see only published videos assigned to a category.
 - Admins can list and manage draft, published, and unassigned videos.
 - Contributor submissions are private to their owner in `/submissions`.
-- Only admins can stream submission recordings.
+- Submission recordings are never readable through the API, including by admins. Admins can access the MongoDB submission record and its archive metadata.
+- Archive paths follow `commons/{userId}_{sequence}/{category_slug}/{video_slug}/{video_slug}.{ext}`. A repeated submission for the same user and video increments `sequence` instead of overwriting the previous file.
 - Video reference files are public only when the corresponding video passes the same published visibility check.

backend/docs/ARCHITECTURE.md

@@ -45,7 +45,9 @@ The authentication token is stored in the `httpOnly` `signpak_token` cookie. The
 
 MongoDB stores users, categories, videos, submissions, cooldown records, and contact messages. Video and recording bytes are stored through the storage driver; MongoDB stores a driver reference, key, MIME type, and size rather than file contents.
 
-Published videos assigned to a category are visible to anonymous visitors. Their metadata, posters, and reference files can be read by the public demo and library. Draft or unassigned videos remain restricted to admins. Contributor recordings are never publicly playable; only admins can access a submission recording.
+Published videos assigned to a category are visible to anonymous visitors. Their metadata, posters, and reference files can be read by the public demo and library. Draft or unassigned videos remain restricted to admins. Contributor recordings are never publicly playable.
+
+Submission recordings use a separate append-only archive path when `ARCHIVE_STORAGE_DRIVER=gdrive`: `commons/{userId}_{sequence}/{category_slug}/{video_slug}/{video_slug}.{ext}`. The API creates the archive object and stores its metadata in MongoDB, but exposes no read, update, or delete operation for the recording. Admin submission responses include the archive path for operational access; contributor responses do not.
 
 Uploads are first written to a temporary directory, checked by file signatures, and moved into storage only after validation. Failure cleanup removes temporary and already-stored files where necessary.
 
@@ -56,7 +58,7 @@ Uploads are first written to a temporary directory, checked by file signatures,
 - SameSite HTTP-only JWT cookies plus origin verification for state-changing requests.
 - Global and operation-specific rate limits.
 - Zod validation for JSON, multipart fields, route IDs, and query parameters.
-- Role checks for admin routes and submission recording playback.
+- Role checks for admin routes and archive metadata access.
 - File type sniffing instead of trusting browser-provided MIME types.
 
 ## Testing and extension points

backend/docs/setup.md

@@ -0,0 +1,239 @@
+# Backend Setup
+
+This guide explains how to install, configure, run, test, and deploy the SignPak Commons backend.
+
+## 1. Prerequisites
+
+Install:
+
+- Node.js 20 or newer
+- npm
+- MongoDB 7+ locally, or a MongoDB Atlas cluster
+- Git
+
+The backend does not require Google Drive for local development. Local storage is the default.
+
+## 2. Install the backend
+
+From the repository root:
+
+```bash
+cd backend
+npm install
+```
+
+Windows PowerShell uses the same commands:
+
+```powershell
+Set-Location backend
+npm install
+```
+
+## 3. Create the environment file
+
+Copy the committed template. Never commit the resulting `.env` file.
+
+macOS/Linux:
+
+```bash
+cp .env.example .env
+```
+
+Windows PowerShell:
+
+```powershell
+Copy-Item .env.example .env
+```
+
+The application loads `.env` automatically through `dotenv`. `src/config/env.js` validates every value at startup and stops the process with a readable error when a required value is missing or invalid.
+
+## 4. Configure MongoDB
+
+### Local MongoDB
+
+Start MongoDB and use:
+
+```env
+MONGODB_URI=mongodb://127.0.0.1:27017/signpakcommons
+```
+
+### MongoDB Atlas
+
+1. Create a cluster at [MongoDB Atlas](https://www.mongodb.com/atlas).
+2. Create a database user with access to the application database.
+3. Add the backend server IP to Atlas Network Access. For local development, use your current IP rather than opening access to everyone.
+4. Select **Connect > Drivers** and copy the Node.js connection string.
+5. Replace the username, password, cluster host, and database name in `.env`:
+
+```env
+MONGODB_URI=mongodb+srv://USERNAME:PASSWORD@CLUSTER.mongodb.net/signpakcommons?retryWrites=true&w=majority
+```
+
+URL-encode special characters in the database username or password.
+
+## 5. Environment variables
+
+### Runtime and networking
+
+| Variable | Where to get it | Purpose |
+| --- | --- | --- |
+| `NODE_ENV` | Set manually: `development`, `test`, or `production` | Selects runtime behavior and production cookie defaults. |
+| `PORT` | Set manually; default `5000` | Port used by the Express server. |
+| `LOG_LEVEL` | Set manually: `fatal`, `error`, `warn`, `info`, `debug`, `trace`, or `silent` | Controls structured log verbosity. |
+| `TRUST_PROXY` | Set `true` behind Render, Railway, nginx, or another reverse proxy | Makes rate limiting use the original client IP. |
+| `CLIENT_ORIGIN` | Frontend URL, for example `http://localhost:5173` | CORS allowlist and state-changing request origin check. Separate multiple origins with commas. |
+
+### Authentication and cookies
+
+| Variable | Where to get it | Purpose |
+| --- | --- | --- |
+| `JWT_SECRET` | Generate locally with the command below, or use a secret manager | Signs authentication tokens. Use a unique value of at least 32 characters. |
+| `JWT_EXPIRES_DAYS` | Set manually; default `7` | Token lifetime in days. |
+| `COOKIE_SECURE` | Usually leave unset; production defaults to `true` | Sends cookies only over HTTPS when true. Use false only for local HTTP development. |
+| `COOKIE_SAMESITE` | Usually `lax`; use `none` only for cross-site HTTPS deployment | Controls cross-site cookie behavior. `none` requires `COOKIE_SECURE=true`. |
+| `BCRYPT_ROUNDS` | Set manually from `4` to `15`; default `12` | Password hashing cost. Increase only when deployment CPU allows it. |
+
+Generate a JWT secret:
+
+```bash
+node -e "console.log(require('crypto').randomBytes(48).toString('hex'))"
+```
+
+### Admin bootstrap
+
+| Variable | Where to get it | Purpose |
+| --- | --- | --- |
+| `ADMIN_EMAIL` | Choose the initial administrator email | Creates or promotes the admin account at startup. |
+| `ADMIN_PASSWORD` | Choose a strong password, at least 8 characters and at most 72 | Sets the initial administrator password. |
+| `ADMIN_FIRST_NAME` | Set manually; default `Admin` | Initial admin profile first name. |
+| `ADMIN_SURNAME` | Set manually; default `Signpak` | Initial admin profile surname. |
+
+The admin is ensured on every startup. Changing `ADMIN_PASSWORD` later does not reset an existing admin password.
+
+### Storage
+
+| Variable | Where to get it | Purpose |
+| --- | --- | --- |
+| `STORAGE_DRIVER` | `local` or `gdrive` | Storage for readable base videos and posters. Use `local` for development. |
+| `ARCHIVE_STORAGE_DRIVER` | `local` or `gdrive` | Append-only storage for contributor recordings. Use `gdrive` for production archives. |
+| `UPLOAD_DIR` | Local filesystem path; default `uploads` | Temporary uploads and local storage root. |
+| `GOOGLE_SERVICE_ACCOUNT_JSON` | Download from Google Cloud, described below | Service-account credentials used when either storage driver is `gdrive`. |
+| `GOOGLE_DRIVE_FOLDER_ID` | The ID of the shared Google Drive root folder | Parent folder for application-created Drive folders. |
+| `MAX_VIDEO_UPLOAD_MB` | Set manually; default `300` | Maximum base-video upload size. |
+| `MAX_RECORDING_UPLOAD_MB` | Set manually; default `100` | Maximum contributor-recording upload size. |
+
+### Application controls
+
+| Variable | Where to get it | Purpose |
+| --- | --- | --- |
+| `RATE_LIMIT_ENABLED` | `true` or `false`; default `true` | Enables API and upload rate limits. Keep enabled outside local debugging. |
+| `SUBMISSION_COOLDOWN_MS` | Set manually; default `30000` | Minimum delay between submissions for the same user and reference video. |
+
+## 6. Google Drive append-only archives
+
+Google Drive is optional for base videos and submission archives. When `ARCHIVE_STORAGE_DRIVER=gdrive`, contributor recordings are created under this structure:
+
+```text
+{GOOGLE_DRIVE_FOLDER_ID}/
+  commons/
+    {userId}_{sequence}/
+      {category_slug}/
+        {video_slug}/
+          {video_slug}.webm
+```
+
+A repeated submission receives the next sequence number instead of replacing an existing file. The API writes the file and stores its pointer and metadata in MongoDB. It does not expose a read, update, or delete route for archived recordings.
+
+### Create Google credentials
+
+1. Open [Google Cloud Console](https://console.cloud.google.com/).
+2. Create or select a project.
+3. Open **APIs & Services > Library** and enable **Google Drive API**.
+4. Open **IAM & Admin > Service Accounts**.
+5. Create a service account for the backend.
+6. Create a JSON key under **Keys > Add key > Create new key > JSON**.
+7. Store the downloaded JSON in a secret manager. Do not commit the JSON file.
+8. Create a folder in Google Drive and copy its ID from the folder URL:
+
+```text
+https://drive.google.com/drive/folders/FOLDER_ID
+```
+
+9. Share that folder with the service account email, usually shaped like `name@project.iam.gserviceaccount.com`.
+10. Give the service account permission to create files and folders.
+11. Put the complete JSON document into the deployment environment as one value:
+
+```env
+ARCHIVE_STORAGE_DRIVER=gdrive
+GOOGLE_DRIVE_FOLDER_ID=FOLDER_ID
+GOOGLE_SERVICE_ACCOUNT_JSON={"type":"service_account",...}
+```
+
+For hosting platforms that provide a secret editor, paste the JSON there. For local development, keep it in `.env` only. Do not put the JSON in `public/`, source control, or a frontend environment variable.
+
+`STORAGE_DRIVER=gdrive` can also be used for base videos, but base videos need to remain readable by the public demo or authenticated library. Use `ARCHIVE_STORAGE_DRIVER=gdrive` when the intended behavior is write-only contributor archives.
+
+## 7. Start the backend
+
+Development mode restarts when source files change:
+
+```bash
+npm run dev
+```
+
+Production-style start:
+
+```bash
+npm start
+```
+
+The startup sequence is:
+
+1. Validate environment variables.
+2. Connect to MongoDB.
+3. Initialize the selected storage drivers.
+4. Create or promote the configured admin account.
+5. Start listening on `PORT`.
+
+Check the service:
+
+- API root: `http://localhost:5000/`
+- Health: `http://localhost:5000/api/v1/health`
+- Swagger UI: `http://localhost:5000/docs`
+- OpenAPI JSON: `http://localhost:5000/docs.json`
+
+## 8. Run tests
+
+The backend test suite uses Node's built-in test runner and starts `mongodb-memory-server` when `MONGODB_URI_TEST` is not provided:
+
+```bash
+npm test
+```
+
+To use a disposable MongoDB database instead, set `MONGODB_URI_TEST` before running tests. The test suite drops that database, so never point it at production data.
+
+The GitHub Actions workflow at `.github/workflows/backend-tests.yml` runs `npm ci` and `npm test` on pushes to `main` or `master` and on pull requests.
+
+## 9. Frontend connection
+
+The frontend uses the API base configured by `VITE_API_ORIGIN`. For local development, the Vite proxy can forward `/api` to the backend. Make sure:
+
+```env
+CLIENT_ORIGIN=http://localhost:5173
+```
+
+The browser must send credentials for the HTTP-only authentication cookie. In production, set `CLIENT_ORIGIN` to the exact deployed frontend origin and configure HTTPS cookie settings.
+
+## 10. Production checklist
+
+- Use MongoDB Atlas or a managed MongoDB deployment.
+- Set `NODE_ENV=production`.
+- Generate a unique `JWT_SECRET` and store it in the host secret manager.
+- Set a strong admin password and never expose it to the frontend.
+- Set the exact production frontend URL in `CLIENT_ORIGIN`.
+- Use HTTPS with `COOKIE_SECURE=true`.
+- Keep `COOKIE_SAMESITE=lax` when frontend and API share a site; use `none` only when required by cross-site deployment.
+- Use `ARCHIVE_STORAGE_DRIVER=gdrive` with a restricted service account for append-only recordings.
+- Keep `RATE_LIMIT_ENABLED=true`.
+- Confirm the upload limits match available storage and request limits.
+- Never commit `.env`, Google service-account JSON, JWT secrets, or database credentials.