October 2026 integration: import check (#656), quorum reduction (#617), final-review PRs, sentry/timeout/export/import-lock fixes

Procfile

@@ -1 +1 @@
-web: gunicorn --bind=0.0.0.0:8000 --workers=4 --forwarded-allow-ips=* app:app
+web: gunicorn --bind=0.0.0.0:8000 --workers=4 --timeout=120 --forwarded-allow-ips=* app:app

README.md

@@ -11,7 +11,14 @@ series of photography competitions. Montage provides a configurable
 workflow that adapts to the conventions of all groups.
 
 - [montage on Wikimedia Commons](https://commons.wikimedia.org/wiki/Commons:Montage)
-- [montage on Phabricator](https://phabricator.wikimedia.org/project/view/2287/)
+- [montage on GitHub Issues](https://github.com/hatnote/montage/issues)
+- [montage on Phabricator](https://phabricator.wikimedia.org/project/view/2287/) (archived)
+
+## Running locally
+
+See [dev.md](dev.md) for full setup instructions.
+
+---
 
 ## Development
 

config.default.yaml

@@ -6,6 +6,12 @@ db_url: "sqlite:///tmp_montage.db"
 cookie_secret: ReplaceThisWithSomethingSomewhatSecret
 superuser: Slaporte
 
+# Optional: set to true to bypass OAuth and log in automatically (dev only)
+debug: false
+# Optional: set these to log in as yourself in debug mode (defaults to __montage_debug__ / 0)
+# debug_username: Slaporte
+# debug_userid: 6024474
+
 dev_local_cookie_value: "contact maintainers for details"
 dev_remote_cookie_value: "contact maintainers for details"
 oauth_client_id: "register an OAuth 2.0 consumer at https://meta.wikimedia.org/wiki/Special:OAuthConsumerRegistration/propose"

deployment.md

@@ -41,6 +41,26 @@ toolforge envvars create MONTAGE_DB_URL         # mysql+pymysql://<user>:<pass>@
 toolforge envvars create MONTAGE_SUPERUSERS     # OPTIONAL — enables su-impersonation only, NOT admin access (see "Maintainer vs superuser" below)
 toolforge envvars create MONTAGE_API_LOG_PATH   # e.g. /data/project/montage-beta/montage_api.log
 toolforge envvars create MONTAGE_REPLAY_LOG_PATH
+toolforge envvars create MONTAGE_IMPORT_CHECK_PATH  # the tool's own folder, e.g. /data/project/montage-beta/import_checks (see below)
+```
+
+`MONTAGE_IMPORT_CHECK_PATH` is required everywhere except local development: Montage refuses to
+start without it. It is the folder for import checks (hatnote/montage#510): each check of a
+first-round import source is written there as a JSON file (mode 0600). A check can be used to
+save a round for 1 hour. The file is deleted after at most 7 days: every check, import, download
+and app start deletes older files, and at most 5,000 are kept. The files hold the source link,
+the rows and Commons' data per file, including Commons uploader names.
+
+- It must be a folder **of its own**: Montage refuses to start if it holds other files, because
+  the cleanup deletes old check files there.
+- It must be absolute and on the tool's NFS share (`/data/project/<tool>/...`, mounted with
+  `--mount all`), so that every pod and a restarted pod see the same files.
+
+Create the folder first, in the tool's home (`$HOME` is `/data/project/<tool>`), and print the
+value to enter at the `envvars create` prompt:
+
+```bash
+mkdir -m 700 "$HOME/import_checks" && ls -ld "$HOME/import_checks" && echo "$HOME/import_checks"
 ```
 
 Optional env vars (all have sensible defaults):
@@ -191,7 +211,25 @@ git -C ~/www/python/src checkout master
 git -C ~/www/python/src pull --ff-only
 ```
 
-#### 4. Run the deploy script
+#### 4. Check new required settings
+
+A release can add a required environment variable; the app then refuses to start until it is
+set. Since hatnote/montage#510 that is `MONTAGE_IMPORT_CHECK_PATH` (see Fresh install, step 3).
+`deploy.sh` checks the required variables before it builds, and stops if one is missing.
+Before the first deploy that includes it, create the folder and set the variable on that tool
+(as the tool, after `become`):
+
+```bash
+mkdir -m 700 "$HOME/import_checks" && ls -ld "$HOME/import_checks" && echo "$HOME/import_checks"
+```
+
+```bash
+toolforge envvars create MONTAGE_IMPORT_CHECK_PATH
+```
+
+Enter the path the first command printed (e.g. `/data/project/montage-beta/import_checks`).
+
+#### 5. Run the deploy script
 
 Which ref to deploy depends on the tool (see [Multi-environment setup](#multi-environment-setup)):
 
@@ -233,6 +271,24 @@ from `toolforge envvars`. A tool that worked on the legacy service can therefore
 buildservice if the `MONTAGE_*` env vars (Fresh install step 3) are not set — and its database may
 need migrating (see "Startup crash: missing column" below).
 
+#### Import checks: troubleshooting and rollback
+
+- **The pod does not start and its log says `MONTAGE_IMPORT_CHECK_PATH ...`:** create the folder
+  and the variable (step 4), then `toolforge webservice buildservice restart --mount all`. No
+  rebuild and no rollback are needed.
+- **A check of a large source fails with a server error after about 30 s:** the request hit gunicorn's
+  worker timeout. Nothing was saved; try a smaller source (split a list, or a sub-category).
+- **Old check files:** every use of the folder (and every app start) deletes files older than
+  7 days. To delete them by hand:
+
+```bash
+find "$HOME/import_checks" -name '*.json' -mtime +7 -delete
+```
+
+- **Rolling back to a version before #510:** the variable and the folder can stay (older
+  versions ignore them). Delete the check files (`rm "$HOME"/import_checks/*.json`): they hold
+  Commons uploader names that nothing reads any more. Entries already imported stay.
+
 ---
 
 ## Debugging

dev.md

@@ -96,6 +96,8 @@ Other frontend development commands:
 ### 6. Use the Makefile to start the backend
 * Open a new terminal tab and change directory to root of repo
 * Copy and edit `config.dev.yaml` based on `config.default.yaml`
+* (Optional) Set `debug: true` in `config.dev.yaml` to bypass OAuth entirely. This is the easiest way to get started locally without setting up OAuth. To log in, navigate to http://localhost:5001/complete_login directly — the "Login with Wikimedia" button still requires OAuth and won't work locally.
+* (Optional) Set `debug_userid: <your numeric Wikipedia CentralAuth ID>` and `debug_username: <your Wikipedia username>` to be auto-logged in as yourself in debug mode. Your CentralAuth ID can be found at https://en.wikipedia.org/wiki/Special:CentralAuth/YOUR_USERNAME.
 * (Optional) In `config.dev.yaml` there is a line for `dev_local_cookie_value`. To get it,
 log in to the local app in your browser, and then copy the value from the
 `clastic_cookie` in the apps' cookies. This is your login cookie.
@@ -106,11 +108,17 @@ juror.)
 This will give your user top-level permissions in the full app, so you can view
 some logs (audit logs, active users), add/remove organizers, and get a
 coordinator view into all campaigns.
+* Build the frontend for the backend (run from the `frontend/` directory):
+```bash
+npm run toolforge:build
+```
 * Start the montage backend
 ```bash
 make start
 ```
-This will build the docker image for the montage backend and start the container. Apart from `make start`, these are other commands:
+The backend is ready when you see `Debugger PIN` in the logs. It is available at http://localhost:5001.
+
+Apart from `make start`, these are other commands:
 * `make start-detached` : Start the backend container in detached mode
 * `make stop` : Stop the backend container
 * `make logs` : Stream the backend container logs in real-time.
@@ -195,8 +203,8 @@ npm run build
 
 ## Project structure
 ```bash
-├── DEV.md
-├── Dockerfile
+├── dev.md
+├── dockerfile
 ├── LICENSE
 ├── MANIFEST.in
 ├── Makefile
@@ -240,7 +248,6 @@ npm run build
 │   ├── templates
 │   ├── tests
 │   └── utils.py
-├── report.html
 ├── requirements-dev.txt
 ├── requirements.in
 ├── requirements.txt
@@ -304,5 +311,5 @@ These provides a detailed explanation of the main components in the **Montage Pr
 
 
 #### Docker Files
-- **`Dockerfile`**: Builds the backend container.
+- **`dockerfile`**: Builds the backend container.
 - **`docker-compose.yml`**: Orchestrates service for backend.

frontend/cypress/e2e/campaign.cy.js

@@ -1,3 +1,39 @@
+// #510: a clean check of one file, valid from now on
+const stubCheck = () => {
+  const now = Date.now()
+  cy.intercept('POST', '/v1/admin/campaign/*/import/check', {
+    body: {
+      status: 'success',
+      data: {
+        token: 'cypress-check-token-000000000000',
+        checked_at: new Date(now).toISOString().replace(/\.\d+Z$/, 'Z'),
+        expires_at: new Date(now + 3600 * 1000).toISOString().replace(/\.\d+Z$/, 'Z'),
+        import_method: 'selected',
+        source: { file_names: ['Example.jpg'] },
+        columns: { name: null, file_id: null, ignored: [] },
+        counts: {
+          ok: 1, renamed: 0, duplicate: 0, unknown_name: 0,
+          unknown_file_id: 0, malformed_file_id: 0, same_name: 0
+        },
+        blocking: false,
+        total_rows: 1,
+        importable_count: 1,
+        issues: [],
+        issues_total: 0,
+        issues_truncated: false,
+        same_name_groups: []
+      }
+    }
+  }).as('checkImport')
+}
+
+const savedRound = () => ({
+  id: 999,
+  name: 'My Test Round',
+  import: { new_round_entry_count: 1, warnings: [], disqualified: [] }
+})
+
+
 describe('Campaign Details Page', () => {
   beforeEach(() => {
     cy.setCookie('clastic_cookie', '<cookie-validation-string>');
@@ -84,8 +120,53 @@ describe('Campaign Details Page', () => {
     .find('li')
     .first()
     .click();
-    cy.get('.button-group button').contains('Add Round').click().click();
-    cy.log(' Round created successfully');
+  // #510 / #447: the source is checked first; Save sends the round together
+  // with its checked import, once
+  stubCheck()
+  cy.intercept('POST', '/v1/admin/campaign/*/add_round', (req) => {
+    req.reply({ delay: 500, body: { status: 'success', data: savedRound() } })
+  }).as('addRound')
+  cy.get('.form-container').contains('label', 'File List').click()
+  cy.get('.form-container textarea').first().type('Example.jpg')
+  cy.get('[data-testid="add-round-button"]').should('be.disabled')
+  cy.get('[data-testid="check-source-button"]').click()
+  cy.wait('@checkImport')
+  cy.get('[data-testid="import-check-result"]').should('be.visible')
+  // a double click must not save twice
+  cy.get('[data-testid="add-round-button"]').click().click({ force: true })
+  cy.wait('@addRound').its('request.body').should((body) => {
+    expect(body.import).to.deep.equal({
+      import_method: 'selected',
+      check_token: 'cypress-check-token-000000000000'
+    })
+    expect(body).not.to.have.property('file_names')
+  })
+  cy.get('@addRound.all').should('have.length', 1)
+  cy.get('.juror-campaign-round-card').should('not.exist')
+})
+
+it('keeps the form open when the server refuses the import', () => {
+  cy.get('.add-round-button').click()
+  cy.get('.form-container input[type="text"]').first().clear().type('My Test Round')
+  cy.get('.form-container').within(() => {
+    cy.get('input[placeholder="YYYY-MM-DD"]').first().clear().type('2025-08-15')
+  })
+  cy.get('[data-testid="userlist-search"] input').type('AadarshM07')
+  cy.get('[data-testid="userlist-search"]').find('li').first().click()
+  stubCheck()
+  cy.intercept('POST', '/v1/admin/campaign/*/add_round', {
+    statusCode: 400,
+    body: { status: 'failure', error_type: 'import_check_expired', detail: 'check expired' }
+  }).as('addRound')
+  cy.get('.form-container').contains('label', 'File List').click()
+  cy.get('.form-container textarea').first().type('Example.jpg')
+  cy.get('[data-testid="check-source-button"]').click()
+  cy.wait('@checkImport')
+  cy.get('[data-testid="add-round-button"]').click()
+  cy.wait('@addRound')
+  cy.get('.juror-campaign-round-card').should('exist')
+  cy.get('[data-testid="import-check-result"]').should('contain', 'expired')
+  cy.get('[data-testid="add-round-button"]').should('be.disabled')
 })