Active learning facial recognition for Wikimedia Commons. Train an ML model to recognize specific people and automatically add P180 (depicts) Structured Data to matching images.

export-1775234516672.mp4

🔗 Quick links

• 📖 Local dev guide: test-local.md
• 🧪 Face service guide: TESTING.md
• 🚀 Toolforge deploy guide: how-to-run-it.md
• 🤝 Contributing: CONTRIBUTING.md
• 🔒 Security policy: SECURITY.md
• 📜 Code of conduct: CODE_OF_CONDUCT.md

✨ Highlights

• 🧠 Active learning UI: fast yes/no classification with keyboard shortcuts, undo, skip, and manual face drawing
• 🧵 Background worker: crawls Commons categories, downloads images, and stores the 512D face embeddings the face service returns
• 🔀 Multi-instance workers: distributed locking lets multiple workers process projects concurrently without conflicts
• 🧩 Separate face service: SCRFD face detection and ArcFace embeddings run on ONNX Runtime in their own KServe service, deployed as a second Toolforge tool — the web and worker processes ship with zero ML dependencies
• 🛡️ Secret isolation: the face service parses untrusted image bytes in a tool that holds no OAuth, database, or token-encryption secret, behind a bearer token and a request-size cap
• 📦 Batched detection: images are sent in size-aware batches, and a service outage leaves work pending for the next cycle instead of failing it permanently
• 🧷 Bootstrap from existing tags: seeds the model via SPARQL when P180 depicts claims already exist on Commons
• 🤖 Autonomous inference: centroid-distance classification once you have enough confirmed examples, with a live status card on the project page
• 🧪 Experimental metadata pre-filter: optional LiftWing verdicts on whether a file could show a person, with recall/skip statistics on the project settings page — shadow mode only, never enforced
• ✍️ User-triggered Commons edits: click "Send Edits to Wikimedia Commons" to write depicts claims via the Wikibase API (interruptible mid-batch; idempotent on already-removed claims)
• 🔐 Hardened OAuth flow: PKCE (S256) plus Redis-backed single-flight token refresh to survive concurrent gunicorn workers
• 🌍 i18n-ready: translations included (en, nb, es, fr)

🧭 How it works

1. 🆕 Create a project — pick a Wikidata entity (e.g., Q42) and a Commons category
2. 🔎 Discover images — the worker traverses the category and detects faces
3. 🧷 Bootstrap (optional) — if Commons already has depicts claims, seed the model from them
4. ✅❌ Classify — review faces one-by-one with Yes/No (keyboard shortcuts: Y / N)
5. 🤖 Infer — after enough confirmed faces (default 5), classify remaining faces automatically
6. ✍️ Write to Commons — send approved matches as depicts claims (OAuth)

🏗️ Architecture

+---------------------------+      +---------------------------+
|       Flask Web App       |      |  Background Worker(s)     |
|          (app.py)         |      |       (worker.py)         |
|      no ML deps           |      |      no ML deps           |
|---------------------------|      |---------------------------|
| OAuth 2.0 login           |      | Category traversal        |
| Project CRUD              |      | Image download            |
| Active learning UI        |      | SPARQL bootstrapping      |
| Classification UI         |      | Autonomous inference      |
| Queue SDC writes          |      | Write SDC claims          |
| Approve/reject/edit bbox  |      | Distributed claim locking |
+------------+--------------+      +------------+--------------+
             |                                  |
             |   face_client.py (HTTP, base64 image bytes)
             +----------------+-----------------+
                              v
                 +----------------------------+
                 |       Face Service         |
                 |      (model-server/)       |
                 |----------------------------|
                 | KServe V1 predictor        |
                 | SCRFD-500M face detection  |
                 | ArcFace MBF 512D embedding |
                 | (ONNX Runtime, CPU)        |
                 +----------------------------+
             |                                  |
             +----------------+-----------------+
                              v
                        +------------+
                        |   MariaDB  |
                        |  (ToolsDB) |
                        +------------+

• 🧰 Stack: Python 3.11+, Flask, gunicorn, PyMySQL, requests-oauthlib — plus KServe + ONNX Runtime + OpenCV, isolated in the face service
• ☁️ Hosted on: Wikimedia Toolforge — wikivisage (web + workers) and wikivisage-face (face service)
• 🔀 Workers: Multiple instances run concurrently — each claims projects via SELECT … FOR UPDATE with automatic stale-claim expiry (15 min)

🗂️ Project layout

WikiVisage/
├── app.py               # Flask app: OAuth, routes, classification API
├── worker.py            # Background pipeline: crawl, remote detect, infer, write (multi-instance)
├── face_client.py       # HTTP client for the face service (zero ML deps)
├── model-server/        # Face detection service — the only place ML runs
│   ├── model.py         # KServe V1 predictor
│   ├── backend.py       # SCRFD + ArcFace on ONNX Runtime, weight verification
│   ├── weights/         # ONNX models (non-commercial licence, see LICENSES.md)
│   ├── guard.py         # Bearer-token + body-size middleware
│   ├── Procfile         # Toolforge buildpack entrypoint (production)
│   ├── Dockerfile       # Local development and CI
│   └── requirements.txt # kserve, onnxruntime, opencv-python-headless
├── token_crypto.py      # Fernet encrypt/decrypt helpers for OAuth tokens at rest
├── database.py          # MariaDB connection pool with retry logic
├── schema.sql           # Database schema (10 tables + indices)
├── migrate.py           # Idempotent migration script with --reset flag
├── jobs.yaml            # Toolforge jobs definition (2 worker instances)
├── templates/           # Jinja2 templates (10 files, all extend base.html)
├── static/              # Logos + screenshots
├── translations/        # i18n: en, nb, es, fr
├── requirements.txt     # Runtime dependencies (no ML libraries)
├── requirements-dev.txt # Dev/test deps (pytest, ruff)
└── tests/               # Hybrid suite: 1059 tests (993 unit, 38 integration, 28 face-service/contract)

🧑‍💻 Setup

✅ Prerequisites

• A Toolforge tool account
• An OAuth 2.0 consumer registered on Meta with grants: - Basic rights - Edit existing pages - Callback URL: https://<toolname>.toolforge.org/auth/callback

1) 🔐 Environment variables (Toolforge)

# Database credentials (find yours in ~/replica.my.cnf on Toolforge)
toolforge envvars create TOOL_TOOLSDB_USER      "s<NNNNN>"
toolforge envvars create TOOL_TOOLSDB_PASSWORD  "<password>"
toolforge envvars create WIKIVISAGE_DB_NAME     "s<NNNNN>__wikiface"

# OAuth 2.0
toolforge envvars create OAUTH_CLIENT_ID        "<client-id>"
toolforge envvars create OAUTH_CLIENT_SECRET    "<client-secret>"
toolforge envvars create OAUTH_REDIRECT_URI     "https://<toolname>.toolforge.org/auth/callback"

# Flask
toolforge envvars create FLASK_SECRET_KEY "$(python3 -c 'import secrets; print(secrets.token_hex(32))')"

# Face service (required — it runs as a second tool, see how-to-run-it.md)
toolforge envvars create WIKIVISAGE_FACE_SERVICE_URL   "https://wikivisage-face.toolforge.org"
toolforge envvars create WIKIVISAGE_FACE_SERVICE_TOKEN "<shared token, same value on both tools>"

# Token encryption (required in production — the app refuses to start on Toolforge without it)
toolforge envvars create WIKIVISAGE_TOKEN_KEY "$(python3 -c 'from cryptography.fernet import Fernet; print(Fernet.generate_key().decode())')"

2) 🗄️ Create database

mariadb --defaults-file=$HOME/replica.my.cnf -h tools.db.svc.wikimedia.cloud

CREATE DATABASE s<NNNNN>__wikiface;

3) 🧱 Run migration

python3 migrate.py

4) 🚀 Build & deploy

# Build container image
toolforge build start https://github.com/DiFronzo/WikiVisage.git

# Start web service
toolforge webservice buildservice start

# Start background workers (uses jobs.yaml — 2 worker instances)
toolforge jobs load jobs.yaml

The app will be live at https://<toolname>.toolforge.org.

The face service is a second tool (wikivisage-face) and must be running before the workers start — otherwise every image stays pending. Setup order, the shared token and rollback are covered in how-to-run-it.md.

🧪 Local development

Start the face service first — the app and worker call it over HTTP:

cd model-server && docker compose up --build   # http://localhost:8080

Then, in another terminal:

pip install -r requirements-dev.txt

export WIKIVISAGE_FACE_SERVICE_URL=http://localhost:8080
export TOOL_TOOLSDB_USER=root
export TOOL_TOOLSDB_PASSWORD=yourpassword
export TOOL_TOOLSDB_HOST=127.0.0.1
export WIKIVISAGE_DB_NAME=wikiface_dev
export OAUTH_CLIENT_ID=<client-id>
export OAUTH_CLIENT_SECRET=<client-secret>
export OAUTH_REDIRECT_URI=http://localhost:8000/auth/callback
export FLASK_SECRET_KEY=dev-secret-key
export OAUTHLIB_INSECURE_TRANSPORT=1

mysql -u root -p -e "CREATE DATABASE wikiface_dev"
python migrate.py
python app.py                                   # Web app on http://localhost:8000
python worker.py --worker-id local-1            # Background worker (separate terminal)

Confirm everything is wired up:

curl -s http://localhost:8000/health   # expects "face_service": "reachable"

For local OAuth you'll need a separate consumer with http://localhost:8000/auth/callback as the callback URL. Set OAUTHLIB_INSECURE_TRANSPORT=1 to allow OAuth over HTTP.

⚙️ Configuration

Each project has a couple of tunables:

Parameter	Default	Description
distance_threshold	0.5	Face-distance cutoff for autonomous classification (lower = stricter).
min_confirmed	5	Minimum confirmed matches before autonomous inference starts.

🔧 Worker environment variables (optional)

Variable	Default	Description
WIKIVISAGE_WORKER_POLL_INTERVAL	60	Seconds between poll cycles
WIKIVISAGE_WORKER_MAX_PROJECTS	4	Max projects processed concurrently per worker
WIKIVISAGE_WORKER_IMAGE_THREADS	4	Parallel image download/detection threads per project
WIKIVISAGE_WORKER_BATCH_SIZE	50	Images per processing batch
WIKIVISAGE_DB_POOL_SIZE	auto	DB connection pool size (auto = min(max_projects × image_threads + 3, 6))
COMMONS_DOWNLOAD_THROTTLE_SECONDS	0	Delay between image downloads (seconds)

🧩 Face service variables

Read by both the web app and the worker (see TESTING.md for the full list):

Variable	Default	Description
WIKIVISAGE_FACE_SERVICE_URL	http://localhost:8080	Base URL of the face service
WIKIVISAGE_FACE_SERVICE_TOKEN	(unset)	Bearer token; must match the face tool's (required in production)
WIKIVISAGE_FACE_SERVICE_BATCH_SIZE	8	Max images per detection request
WIKIVISAGE_DETECTION_WIDTH	960	Commons thumbnail width used for detection and manual drawing (a standard Commons step)

🧪 Metadata Pre-Filter (experimental, optional)

Projects can switch on Metadata Pre-Filter (experimental) in project settings. In shadow mode the worker asks Wikimedia LiftWing's llm-qwen36-27b whether a file could show a person, using only public Commons metadata — never image bytes. The verdicts are stored for evaluation, never block processing, and the settings page shows the shadow results (files checked, would-skip rate, and any human-confirmed matches it would have dropped). A background sweep records verdicts for any project with the mode on — including completed ones the main pipeline no longer touches — up to 250 files per project per pass. Calls are serialised and spaced by a jittered minimum interval (default 2 s) so the worker never bursts at the shared service; anonymous access works (rate-limited per IP) and WIKIVISAGE_LIFTWING_API_TOKEN (a Wikimedia personal API token) moves calls to the authenticated quota. See how-to-run-it.md for the remaining variables.

✅ Testing

# Unit tests (CI mode — integration tests auto-skipped)
pytest tests/ -v

# Unit + integration tests (requires local MariaDB)
WIKIVISAGE_TEST_DB=1 pytest tests/ -v

# With coverage
WIKIVISAGE_TEST_DB=1 pytest tests/ --cov=. --cov-report=term-missing

📝 License

MIT for the WikiVisage code.

⚠️ The face models in model-server/weights/ are third-party InsightFace weights released for non-commercial research use only; they are not covered by the MIT licence. See LICENSES.md and model-server/weights/LICENSE-NOTICE.