UK wave 1: SLC student loan forecasts, student support and repayments (#132) - #155
UK wave 1: SLC student loan forecasts, student support and repayments (#132)#155juaristi22 wants to merge 2 commits into
Conversation
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…PolicyEngine#132) Six packages, 199 facts - the wave-1 remainder unblocked by PolicyEngine#136, and the first consumer of the academic_year period type: - slc-student-loan-borrower-forecasts-england-2025 (116): EES Table 6a borrowers liable to repay / earning above threshold by product group, plan and status, AY2024/25-2029/30 on the ees_permalink_table_html lane. Fact periods store the OPENING year while the EES value columns are END-year named (value_2025 = AY 2024/25) - the exact leak the ALLOWED_PERIOD_TYPES caution documents. All values are forecasts: model_output + assertion source_projection, so profiles opt in via assertion_policy (PolicyEngine#154). - slc-student-support-england-2025 (72): Table 3A maintenance loans (Grand-total recipients and GBP-million amounts) and Table 4C(i) Adult Dependents Grant + Parents Learning Allowance, AY2013/14-2024/25. uk-data's whole-pound roundings of the published GBP-million precision are enumerated; cohort rows and the other 4C products are named backfill exclusions. - slc-student-loan-repayments-england-2025 (8): the CORRECTED slcsp01 workbook's Table 1A net-repayments row, FY2024-25 block, all seven plan columns + total at publisher precision. uk-data's plan_2 and postgraduate targets are FT+PT / Masters+Doctoral sums of these columns - populace-side derivations. - slc-student-loan-repayments-{scotland,wales,northern-ireland}-2025 (1 each): the HTML-only country statistics pages, archived and parsed on the document-numbers lane (203.3m / 229.1m / 181.7m GBP, FY2024-25). Every uk-data snapshot value reconciles exactly (counts and England repayments byte-equal; GBP amounts equal at publisher precision). Bundle expectations re-measured under the disclosed-and-reverted sandbox: 39,418 facts / 66 packages on this branch, merged-bundle test passes. Manifests are upload-truthful. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…olicyEngine#132) Six artifacts uploaded to ledger-raw under content-addressed keys via the post-PolicyEngine#148/PolicyEngine#149 publish path. Flips the SLC bundle contract from designed-red to green. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
078d911 to
c20fa07
Compare
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
vahid-ahmadi
left a comment
There was a problem hiding this comment.
Six SLC packages covering all four nations with a registry-only code change — clean. Splitting England (which has the full forecast + support tables) from the three smaller repayment packages matches how SLC actually publishes.
1. academic_year looks like a new period type — is the vocabulary declared anywhere?
This PR introduces academic_year:2013 through academic_year:2029, which I don't see elsewhere in the existing by_period counts. If period types are a closed vocabulary validated at load, that needs a registration; if they're open, then the consumer side needs to know how academic_year:2024 aligns to a tax_year or calendar_year base period, or these facts will be unresolvable by any profile that asks for a normal year.
Related: academic_year:2025 through 2029 are borrower forecasts. #154 (open) adds the assertion: source_projection axis precisely for this, with observed_only as the default resolution policy. Without that field these forecast years look identical to the observed 2013-2024 history, and the first consumer to ask for "latest" quietly gets a projection. Worth sequencing against #154 — the SLC forecast table is a textbook case for it.
2. The golden-count test will collide with the other three open UK waves.
#137, #139, #141 and #155 all branch from the same base and rewrite the absolute totals in tests/test_chronicle_bundle.py to different values:
| PR | fact_count |
source_package_count |
period_count |
|---|---|---|---|
| #137 | 42895 | 64 | 19 |
| #139 | 40131 | 69 | 61 |
| #155 | 39418 | 66 | 33 |
Each is correct in isolation and stale the moment a sibling lands. That's four rebases with hand-recomputed constants, and resolving the conflict by picking a side gives you a green test that has stopped checking anything. Worth filing as its own issue — per-package delta assertions would compose where one whole-bundle snapshot can't.
3. Minor
- The diff includes a couple of unrelated re-orderings in the
by_sourcedict (hmrc/iciswapped,onem_rva_unemployment/onsswapped) that aren't caused by this PR's data. Harmless in a dict literal, but they add conflict surface against the three sibling branches that touch the same lines. Reverting them would make the rebase cheaper. db/data/slc/student_loan_borrower_forecasts_england_2025/ees-table-6a-permalink.htmlis committed as a single line (+1). If that's a minified permalink response it's fine — just checking it isn't a truncated download.- Three of the four nations land as
.htmlscrapes while England is.xlsx; if SLC publishes equivalent workbooks for Scotland/Wales/NI, those would be more stable inputs than parsed HTML.
The wave-1 remainder, unblocked by #136's
academic_yearmerge — with #137 (region×age population; ISC landed via #141 batch 4), this completes #132's package inventory.Closes #132.
Lane:
ledger-source-ingestor— judgesledger-source-fidelity+ledger-boundary.What
Six packages, 199 facts, new
slcsource directory — and the first real consumers of two fresh mechanisms:slc-student-loan-borrower-forecasts-england-2025ees_permalink_table_htmllane: borrowers liable to repay / earning above threshold, by product group × plan × status, AY2024/25–2029/30. Firstacademic_yearconsumer: fact periods store the opening year while the EES value columns are END-year named (value_2025= AY 2024/25) — the exact leak theALLOWED_PERIOD_TYPEScaution documents. All values are forecasts →model_output+assertion: source_projection, so profiles opt in viaassertion_policy(#154). Null publisher cells skipped per year, not zero-filled.slc-student-support-england-2025'Grand total 'carries a trailing space in the workbook — guards pin the exact text. Cohort rows and the other 4C products are named backfill exclusions.slc-student-loan-repayments-england-2025slc-student-loan-repayments-{scotland,wales,northern-ireland}-2025Fidelity vs policyengine-uk-data (every delta enumerated)
TESTING=1snapshots; the archived permalink page is now the pinned artifact.Gates
validate-package+build-suite+agent_acceptancegreen for all six; spot checks byte-exact against artifact cells and the uk-data snapshot table.raw_r2_uri.Draft because the consumer export awaitsPublished: credentialedpublish-rawchronicle publish-rawran on this branch (post-Pin Cloudflare account for R2 publishing and fix publish-raw failures #148/Fill missing source_id/package_id so publish-raw covers every manifest #149) — all six SLC artifacts uploaded toledger-rawunder content-addressed keys and the manifests commit their realstorage:blocks. No designed-red remainder. Known conflict-by-design with UK wave 1: ONS population source packages — mid-year estimates 2023+2024 and 2024-based projections (#132) #137/UK wave 2: national fiscal/benefit source packages (#133) #139/UK wave 2 continuation: DWP, VOA, scotgov, DfT and ONS national packages (#133) #141 onSOURCE_PACKAGE_ALIASES+ the bundle constants stands; whichever merges second re-measures.🤖 Generated with Claude Code