Analytics

Overview

Stay up to date with hiero organisation activity and contributor diversity

This repository provides analytics for the Hiero repositories.

Latest dashboard: hiero-hackers.github.io/analytics

Contributing? Start with the contributor guide — setup, workflow, skill levels, AI policy, and testing conventions.

Quickstart

Prerequisites: Git, Python 3.11+, and uv. uv provisions the right Python version and all dependencies for you.

git clone https://github.com/hiero-hackers/analytics.git
cd analytics
uv sync          # create the environment and install everything
uv run pytest    # verify — no credentials needed

Most work, and the whole test suite, runs without credentials. For pipelines that fetch live GitHub data, add a .env file in the project root with a token that has public-repo read access (this only raises your API rate limit):

GITHUB_TOKEN=<your token>

Contributing? The contributor guide has the full setup, the fork workflow, linting and pre-commit, skill levels, and testing conventions.

Running the Analytics

With your GITHUB_TOKEN configured (see Quickstart), run every analytics pipeline with a single command:

uv run hiero-analytics

What this does:

• Runs all analytics pipelines in one process (one Python start-up instead of one per pipeline), reusing the on-disk fetch cache between pipelines
• Runs them for every organisation in the org registry (src/hiero_analytics/config/orgs.py), each getting the pipelines its capabilities allow (governance pipelines only for orgs that publish a governance config, for example)
• Writes data tables to outputs/data/ and the JSON data API to outputs/data/api/
• Isolates failures — if one pipeline errors it is logged and the rest still run. The command exits non-zero if anything failed for the primary organisation or the data API; failures confined to another organisation are reported but do not fail the run (add --strict to all to fail on those too)

For faster local debugging, run the explicit all command with --fail-fast:

uv run hiero-analytics all --fail-fast

This stops after the first pipeline failure and exits non-zero immediately, so the original traceback is not buried beneath output from later pipelines. Without --fail-fast, the default behaviour is unchanged: failures are logged, the remaining pipelines continue, and all failures are reported at the end.

To try an organisation that is not registered yet, add it to one run (it is treated as ungoverned; see docs/adding-an-organisation.md):

HIERO_ANALYTICS_TRIAL_ORGS=<github-org> uv run hiero-analytics

Everything under outputs/ is generated and gitignored. The scheduled workflow publishes the dashboard to GitHub Pages instead of committing generated data and reports.

This is the same command the scheduled Refresh Analytics Data workflow runs.

⏱️ The first run is slow. It fetches org-wide activity from the GitHub API (subject to rate limits), so a cold first run can take the better part of half an hour (and can fail if GitHub is flaky). Later runs are incremental and much faster (see Incremental data fetching).

Skip the wait: after creating your .env, run uv run hiero-analytics seed-datasets first. It downloads the datasets the nightly refresh already built, so your first real run is an incremental one (see Seeding from CI).

Viewing the dashboard

Just want to look? The latest refresh is published to GitHub Pages — open https://hiero-hackers.github.io/analytics/ to view it in your browser, no clone or setup required. The scheduled Refresh Analytics Data workflow rebuilds and republishes it automatically.

The dashboard is a static Vite + React app in web/ that renders the versioned JSON data API (outputs/data/api/v1/) — it is built from the generated data, so generate the data first or the dashboard will be empty. uv run hiero-analytics already emits the API as its last step, so on a fresh checkout that one command gives you data and a populated dashboard. To develop locally:

uv run hiero-analytics data_api        # re-emit the API from existing outputs
python3 -m http.server 8642 -d outputs # serve the data API (dev proxy target)
npm run dev --prefix web               # the app, on http://localhost:5173

New sections, charts, and orgs appear in the app automatically — it renders whatever the API manifest lists (one tab per organization that has data), so adding analytics rarely requires frontend changes.

Tracing a chart or table back to its data

Nothing generated is committed, and each Pages deploy replaces the last, so every artifact carries its own provenance instead:

• The dashboard shows the data watermark in its header and the code revision in its page footer (both from the API manifest's provenance block), plus a per-section data as of badge. A -dirty suffix on the revision means the data was built from uncommitted code and cannot be reproduced from any commit.
• The dataset snapshot archived by each scheduled run carries the same manifest (see docs/snapshots.md).
• CSVs on disk (outputs/data/) keep their provenance in a <name>.csv.meta.json sidecar — generated_at, git_sha, record_count. The CSV body is left clean so pd.read_csv works unchanged.
• Each scheduled run archives its dataset snapshot as a dataset-snapshot-<run>-<sha> workflow artifact, including a SNAPSHOT.json manifest of per-dataset watermarks and SHA-256s.

Downloading a table as CSV from the dashboard writes # comment lines above the header, naming the view, the data watermark, and the revision. The export takes the rows currently visible, so filtering before you download gives you a subset — the preamble says so (# 2 of 7 rows (filtered view)).

Those comment lines mean a downloaded file needs the comment flag when read programmatically:

python -c "import pandas as pd; print(pd.read_csv('maintainers.csv', comment='#'))"

Without it pandas raises ParserError rather than mis-reading the header. Spreadsheet apps open the file fine, showing the preamble as four leading text rows. This applies only to browser downloads — the CSVs under outputs/data/ have no preamble.

Pull request dashboard previews

Pull requests that change analytics code build the full site (data API + web app) and upload it as a dashboard-preview workflow artifact. Download and unzip it, then serve the folder:

python3 -m http.server -d .

Open the URL it prints (http://localhost:8000/). A server is needed because the app fetches its JSON over HTTP, so opening index.html from the filesystem won't load data. The artifact is built from the pull request's own code — treat it as you would any other contributor-authored code.

Most previews restore the latest base-branch datasets and set HIERO_ANALYTICS_OFFLINE=1. This keeps the input data fixed so the artifact isolates code changes. Offline mode never falls back to a network fetch: it fails clearly when a required dataset or governance snapshot is missing. Pipelines backed only by live repo or third-party APIs are skipped, so Scorecard, CODEOWNERS/runner, repo-only, and Hiero Hackers sections may be absent from an offline preview.

Changes to the field-bearing ingestion layer (data_sources/models.py, github_ingest/, the queries, the governance snapshot) or to the org registry (config/orgs.py) automatically use a live fetch, because a cached dataset cannot contain newly introduced fields or newly registered orgs; transport-only changes (client, rate limiting) keep the fast offline path. Those fields are populated only for records fetched during the preview; the scheduled refresh remains responsible for a complete backfill. Preview workflows restore caches but never save them.

Running a single pipeline

The same hiero-analytics CLI runs each pipeline as a subcommand:

uv run hiero-analytics scorecard

Repo-scoped pipelines accept --org and --repo (defaulting to the configured GITHUB_ORG / GITHUB_REPO); run uv run hiero-analytics <command> --help to see a subcommand's options. Each pipeline lives in src/hiero_analytics/pipelines/<command>.py and is declared in the registry in src/hiero_analytics/pipelines/__init__.py.

Available pipelines:

Command	What it produces
difficulty	Issue difficulty distribution
difficulty_over_time	Difficulty trend over time
onboarding	Onboarding signal (issues vs. contributors) — writes onboarding_signal.csv and onboarding_efficiency.csv
contributor_profiles	Per-contributor profiles
maintainer_pipeline	Maintainer pipeline by governance role
contributor_activity	Org-wide contributor activity tables
entity_activity	Per-repository and per-contributor activity (by period and month) behind the dashboard's detail views
contributor_heatmap	Build contributor activity heatmap tables
role_coverage	Governance roles vs. real activity per repo
affiliation	Contributor affiliation mapping
scorecard	OpenSSF Scorecard results
codeowner_and_runner	CODEOWNERS presence and CI runner usage
releases	GitHub Releases cadence and per-repo staleness (latest_release, days_since_last_release)
ci_health	Audit CI/CD workflows and repository configuration for security, reliability, and compliance
org_overview	Org-wide composition (contributor counts, languages, pushes)
hip_implementation	Maps HIPs to the PRs that reference them across the org — feeds the HIPs dashboard tab
repo_growth	Generate repository-growth timeline tables
data_api	Emits the versioned JSON data API (outputs/data/api/v1/) the web dashboard renders — the full run does this last
discord_analytics	Discord analytics — needs manual CSV inputs, so not part of the full run
contributor_churn	Contributor churn analysis (funnel, retention, transitions and tenure CSVs) — on-demand, not part of the full run
build_affiliations	Regenerates affiliations.yaml from public signals (hand corrections go in affiliation_overrides.yaml) — maintenance tool, needs gpg

Fetched GitHub data is cached under outputs/cache/ for 24 hours, so repeated runs within a day reuse it instead of re-querying the API.

Incremental data fetching

To avoid re-downloading all of GitHub history on every run, fetching is incremental:

• The first run does a full fetch and stores a dataset under outputs/data/datasets/ (this run is the slow one).
• Later runs fetch only what changed since the last run and merge it in — much faster.
• Every 30 days (or with refresh=True) it does a full re-fetch to self-heal, so missed updates or deleted items can't accumulate.

The datasets are not committed to git — they're gitignored. Persistence is handled differently per environment:

• Locally: the dataset lives on your disk under outputs/data/datasets/. Nothing to set up — just run the pipeline. To force a clean rebuild, delete that folder.
• In CI: the scheduled workflow persists the dataset between runs via actions/cache (see .github/workflows/update-analytics.yml). If the cache is ever evicted, the next run simply does one full fetch and then resumes incrementally.

Local and CI datasets are independent — each maintains its own and stays correct on its own; you never need to sync them.

Seeding from CI

To avoid the slow first fetch, copy the nightly refresh's datasets instead of rebuilding them:

uv run hiero-analytics seed-datasets

This needs GITHUB_TOKEN in your .env (GitHub only serves Actions artifact downloads to authenticated requests). It finds the most recent successful run of the Refresh Analytics Data workflow (failed runs also archive a partial snapshot, which is skipped), downloads its dataset-snapshot-… artifact, checks it, and unpacks it into outputs/data/datasets/. It then prints the run, code revision and "data as of" time of what you now have; your next uv run hiero-analytics only fetches what changed since.

• It only ever downloads from the configured repository's own Actions artifacts, and validates the archive before writing anything: flat .json file names only, every file must parse, dataset files must carry the store's version key, and the total size is capped.
• It refuses to write into a datasets directory that already has files; pass --force to overwrite files of the same name.
• Forks: the default repository is hiero-hackers/analytics; point it at your own with --repo OWNER/NAME (your fork needs a completed run of the refresh workflow). --run-id N picks a specific run instead of the latest.

---

Documentation

• Architecture — the layer map, the extensibility registries, the fetch/persistence model, multi-org scheduling, and where new features go. Start here to understand the codebase.
• Adding an organisation — the one registry entry that adds an org to the dashboard, what each capability gives it, and how to trial-run it first.
• Role-holder affiliations — how each maintainer and committer is mapped to an organisation, how to make manual corrections in affiliation_overrides.yaml, and how to resolve the unknowns.
• Snapshot archive — every refresh's data API is archived to the data/snapshots branch, so the dashboard's history is queryable and diffable. How to read a past snapshot, and how to build on it.

---

License

• Available under the *Apache License, Version 2.0 (Apache-2.0)