Repository navigation
feat: banded timeline, browse by domain, seven new sources - #6
Merged
Merged
Conversation
…ries A sparse source inside a dense decade (160 IMDB events among 500k music releases) walked events_by_start and rejected almost every row: 16 s, past the component's 20 s limit on the next decade. When event_density says the requested sources are sparse in the window, the page is now taken from events_by_source_year first and only those rows are joined (56 ms). Source membership is checked on the events row in every query rather than on sources.slug after the join (3 s to 27 ms for NBA 1960s). Domain views ask the LOD tables for up to fifty categories at once. The primary keys lead with the bucket, so that scanned every category in range (0.7-1.2 s per tile). New category-first indexes and a rank column on event_lod/span_lod let the overview cut the top events per bucket across categories before touching events; `make rerank` fills the rank. event_density gets a category index for the domain counts. Also new: /timeline/sources (per-source totals, year range, top categories, last sync) and density?by=source for the minimap. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The site is no longer an IMDB browser with a timeline API on the side. Timeline (web/, a small TypeScript app bundled by Vite): one horizontal band per data source, scrolled sideways over a world range of years; a sticky year axis, a density minimap of the whole range, and zoom with ctrl+wheel or +/-. Zoomed out, each band shows the top events per century, decade or year from the LOD tables; zoomed in it reads the full event stream in ten-year tiles. Name, category, kind and source filters, domain chips, a detail panel with participants, and "Compare from here", which adds a second axis of years before and after the chosen event. The view state lives in the URL so any view can be shared. Pure layout maths (tiers, tiles, ticks, lane packing) is unit tested under web/__tests__. Domains (wasm/src/domains.ts, /timeline/domains): Film, TV & games, Music, Sports, Books, Awards, Science, Politics, Wars, Religion, Business, Arts, Exploration, Lives and Wikipedia lists, each a set of event categories and kinds with live counts, so the menu stays short while categories keep growing; categories without data show as stubs. Pages: browse.html (domain grid and per-domain view with highlights by century), sources.html (every dataset with counts, years, licence, last sync and what is planned), entity.html (an entity's events across sources), search.html now covers timeline entities from every source through /search (trigram FTS, ranked by the entity's top event), and the TV page explains an empty sample import instead of showing nothing. The IMDB tables stay reachable from the Film domain. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…dated stubs Five declarative sources from the files already under data/: Book- Crossing books (271k publications with their authors as participants), Steam games (89k releases keyed by app id, which Wikidata carries as P1733), famous painters (lives), international cricket fixtures and MLS matches. The CSV reader lifts the field limit for Steam's descriptions. Wikipedia lists: scripts/wikipedia_lists_fetch.py pulls bibliographies and "List of ..." pages through the MediaWiki API (seeded from "Lists of books", or any page given), and the wikipedia_lists adapter turns every dated entry into an event: cite templates and italic-title lines become books and articles with authors, every other dated line a stub "listed" event, and each list page an entity that takes part in its entries. Wikidata: `wikidata_extract.py --all-dated` keeps any item with a calendar date (inception, publication, point in time) as a "dated" stub under its first class, so everything Wikipedia dates can be represented before it has a parser of its own. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The rerank's big write transactions hit 'database is locked' after the default five seconds while TrailBase held readers on the database; a 300 s busy timeout lets make rerank and make sync-events run against a live server. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…gories are stubs Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Thyme becomes a general timeline site rather than an IMDB browser with a timeline API on the side.
/timeline.html,web/timeline/): one horizontal band per data source, scrolled sideways across a world range of years, with a sticky year axis, a density minimap and zoom (ctrl+wheel, +/−). Zoomed out, each band shows its top events per century, decade or year from the LOD tables; zoomed in it reads the full event stream in ten-year tiles. Name, category, kind, source and domain filters, a detail panel with participants, and Compare from here for a second axis of years before and after an event. View state lives in the URL./browse.html,wasm/src/domains.ts,/timeline/domains): Film/TV/games, Music, Sports, Books, Awards, Science, Politics, Wars, Religion, Business, Arts, Exploration, Lives and Wikipedia lists, each a set of event categories and kinds with live counts; categories without data show as stubs. Nav is Timeline · Browse · Sources · Search · About; the IMDB tables stay reachable from the Film domain.data/, plus Wikipedia bibliographies and "List of …" pages through a new fetcher (make wikipedia-lists) and parser.wikidata_extract.py --all-datedkeeps any Wikidata item with a calendar date as a stub.U1791270000; runmake rerankon existing databases.Test plan
yarn lint,yarn type-check(wasm/ and web/),yarn test(51 Jest tests)make test-ingest(ingest, Wikipedia list parser and Wikidata extract suites)make rerank🤖 Generated with Claude Code