Live: https://talk-snapshots.toolforge.org/ — overview in English, wikis ordered by volume, plus a coverage table of other Wikipedias (app/coverage.py);
/wiki/<dbname> — per-wiki page in that wiki's language (app/i18n.py);
/wiki/<dbname>/mismatches — case-by-case list where the discussion and the deletion log disagree (page deleted but discussion open, kept but deleted, delete but exists, redirect but article, no page recognised, page found nowhere), each row linking to the discussion, the page and its log (app/core/mismatch.py);
/api/report.json — the same aggregates as JSON (CORS * is deliberate: the API is meant
for on-wiki gadgets and user scripts); /healthz (liveness), /readyz (showcase cache built, database reachable and, on Toolforge, Toolsdb rather than SQLite).
Who argues with whom, about which pages, in deletion discussions — and what happens to those pages afterwards — across Wikipedias with one codebase.
Seven wikis are wired in: ruwiki, enwiki, dewiki, ukwiki, zhwiki,
itwiki, jawiki. Adding one more is a single declarative file, see below.
Universal. The participation layer (who spoke, when, replying to whom)
comes from the DiscussionTools API and works on any wiki. The fate of a page
(exists / redirect / deleted, when, by whom, on which grounds / moved /
recreated) comes from the page and logging tables of the database
replicas — the same schema on every wiki, one query for all of them.
Per wiki. Where the discussions live, how nominations are separated on a
page, which pages a nomination is about, where the closing decision is written
and how a comment expresses a stance. None of this is hard-coded: each axis is
a small strategy in app/core/, and a wiki is a composition of strategies in
app/wikis/<dbname>.py.
| Axis | Strategies | Used by |
|---|---|---|
| listing | DayPage, DailyLog |
ru/de/uk/zh · en/it/ja |
| pages (1:N) | Subsections, HeadingLinks, TitleFromHeading, TitleAfterPrefix |
group nominations, headings, per-nomination pages |
| outcome | OutcomeSection, HeadingSuffix, ClosingTemplate, CommentPattern, NoOutcome |
ru/uk · de · en/zh · ja · it |
| stance | VoteWords, SectionStance, NoStance |
en/de/uk/zh/ja · it · — |
| reason | ReasonClass regexes over the deletion-log comment |
all |
Discussion outcome and page state are stored separately
(discussion_outcome vs page_state): a page can be kept by the discussion
and speedily deleted a month later, or deleted and recreated — those are
different facts.
A nomination may concern several pages (nomination_pages): Russian
Wikipedia groups articles under one heading with sub-headings per article,
and a fifth of nominations there are such groups or non-article pages.
| Source | What it provides | Module |
|---|---|---|
| DiscussionTools API | thread structure as MediaWiki itself sees it: author, timestamp, nesting, sub-headings, transcludedfrom |
core/threads.py |
| Database replicas (Toolforge; API elsewhere) | revision history of the discussion page, page state and deletion log, categories and Wikidata item snapshot | core/revisions.py, core/state.py, core/topics.py |
Wikidata API (wbgetentities, no maxlag) |
P31/P106 of the nominated page's item, taken at nomination time |
core/topics.py |
| Wikitext parsing | fallback for dump processing, where an API call per page is not affordable; also the closing templates | core/parse.py |
Topics are snapshotted at nomination time: after deletion the categories
are gone from every source (replicas, Wikidata sitelinks, even the enwiki
DELSORT categories vanish when the AfD is closed).
The Wikidata item itself is deleted soon after the last sitelink goes, so
P31 (instance of) and P106 (occupation) are stored at the same moment as
class QIDs — those survive (Q5, Q4830453 «business»…).
The same pass stores the page's authors (page_authors: edits, bytes added, first/last
edit per user) — after deletion the history is admin-only, and these are the people to
invite to the discussion (core/authors.py).
pip install -r requirements.txt
export TS_DB=ts.sqlite # or toolsdb:<db> on Toolforge (set by envvars; unset there = refuse to start, no SQLite fallback)
export TS_WIKIS=ruwiki,enwiki,dewiki
python -m app.cli ingest --from 2026-07-01 --to 2026-07-03 # all TS_WIKIS
python -m app.cli --wiki dewiki ingest --recent 3 --refresh-changed
python -m app.cli state # page fate from logs, whole volume
python -m app.cli quality # metrics vs thresholds, non-zero exit on breach
python -m app.cli logs --from 2026-07-01 # log counters: deletions by channel, restores, moves, protections
python -m app.cli daily # what the cron runs: the four above
python -m app.stats --report top # participation reports--refresh-changed re-reads a day page only if its latest revision on the
wiki differs from the stored one: fixing a discussion on-wiki is the trigger,
there is no management UI. Page state is recomputed over the whole volume on
every run — it has no window.
- Create
app/wikis/<dbname>.pywith aWIKI = WikiSpec(...): locale (months, user-namespace aliases, namespace prefixes, bots, signature format), and one strategy per axis.dewiki.pyis the shortest example. - Save one day of DiscussionTools output as
tests/fixtures/<dbname>/dt_<day>.jsonand pin counts inexpected_<day>.json. The registry test refuses wikis without a fixture. - Add the dbname to
TS_WIKIS. Nothing inapp/core/changes — a test asserts the core never mentions a wiki name.
Procfile declares web (gunicorn serving the dashboard straight from Toolsdb with an
hourly in-memory cache — no files shared between the cron and the web pod), migrate,
daily, stream and smoke; toolforge.yaml schedules daily and keeps stream running.
stream is a continuous job: it listens to EventStreams recentchange, and when a
deletion venue's day page (the event's day, the day before or after in UTC) is edited it
re-reads the nominations there and snapshots categories, the Wikidata item with P31/P106
and the page authors for pages it has not seen in the last three days. Articles closed within
hours are thus captured before deletion. Snapshots are keyed by page (page_topics,
page_authors) with first/last seen, not by nomination id, which changes on every refresh.
A cheap parse&prop=sections check skips passes when only replies changed; the stream
position is kept in tool_state, never moves past an event whose page is still queued, and is
replayed with a two-minute overlap after a restart (at most 24 hours back). migrate copies
the pre-2026-10-25 topics table (keyed by nomination id) into page_topics once.
Licensed under Apache-2.0.