Make the sector TFP residual unit-invariant in K and L

[vahid-ahmadi]

Fixes #71.

Why

_sector_tfp backs out sector TFP as a Solow residual, dividing GVA by a production aggregator with K in £m (ONS CAPSTK) and L in thousands of jobs (ONS JOBS02). Neither branch of that calculation is scale-free:

• CES (epsilon != 1): K and L are summed inside the aggregator, so the units they happen to be measured in set their relative weight.
• Cobb-Douglas (epsilon == 1, which _EPSILON[2] hits): K^gamma_m carries a sector-specific exponent, so a common rescaling of K does not cancel in the GVA-weighted normalisation either.

The consequence is that the cross-sector TFP dispersion — the entire economic content of the calibration — was an artefact of a measurement choice:

same data, different units	Z spread
as shipped (K in £m, L in thousands)	25.2x
L expressed in jobs	3.79x
K expressed in £	2.17e6x

What changes after merging

Multi-sector (M=8) TFP is calibrated from the data rather than from the units, and Z becomes invariant to rescaling either input:

	Z	spread
before	[2.611 0.538 0.416 0.220 4.820 0.191 0.343 0.420]	25.22x
after	[1.302 0.935 1.062 1.045 2.268 0.604 0.517 0.630]	4.39x

Real Estate remains the outlier but falls 4.82 → 2.27, and the six non-outlier sectors collapse into ~0.52–1.30. The residual 4.4x is plausible genuine dispersion — Real Estate's imputed-rent GVA against a dwelling-dominated capital stock is a real feature of the data.

This changes M=8 model output. That is the point of the fix, not a side effect, but any saved multi-sector baselines are now stale. multi_sector=False (the default) is untouched — this dict only loads under M=8.

Change

Each input is converted to a share of its own aggregate before entering the aggregator, applied before the loop so it covers both the CES and Cobb-Douglas branches. Output normalisation is unchanged, so Z still has GVA-weighted mean 1 (verified: exactly 1.0). Epsilon and gamma values are untouched.

Optional capital= / labour= arguments were added purely so the invariance is testable — the module globals are otherwise unreachable from a test.

Also fixes two contradictory comments in the same file: industry_params.py:157-158 claimed epsilon shrinkage was applied "in get_industry_params()" while the get_industry_params docstring claimed it was "applied in _EPSILON values". Neither was true — line 354 is a plain epsilon = list(_EPSILON). Both now state what the code does. The gamma shrinkage is real and documented as before.

Evidence

oguk/tests/test_industry_params.py, 8 tests in 0.15s. The key one is parametrised over rescaling K alone, L alone, and both, using both the real unit-conversion factors (1e6, 1e3) and arbitrary non-round ones (137.0, 0.017), asserting rtol=1e-12. Measured after the fix:

K x1e6   max reldiff 0.00e+00
L x1e3   max reldiff 0.00e+00
odd      max reldiff 2.22e-16

Verified to fail on the pre-fix code. Also tested: GVA-weighted mean is 1, and get_industry_params()["Z"][0] matches _sector_tfp with the shipped arguments.

ruff format --check and ruff check clean. Two failures in test_get_micro_data.py are pre-existing on main — DatasetMaterializationError from a missing HUGGING_FACE_TOKEN, unrelated to this change (confirmed against main).

[vahid-ahmadi]

Correcting an overclaim in the description above, flagged by an independent review round.

The PR says the fix means multi-sector TFP is "calibrated from the data rather than from the units", and calls the residual 4.39x spread "plausible genuine dispersion". The first half is right; the second overstates what this establishes.

Unit-invariance is not the same as convention-independence. Under CES with epsilon != 1 the dispersion depends on the relative level of k to l, and "each input as a share of its own aggregate" pins that at exactly 1.0 — which is a convention, not a measurement. Holding the ONS data fixed and varying only that relative level:

relative k/l level	spread
0.1	3.64x
1.0 (this PR's shares convention)	4.39x
4.98 (the model's own aggregate K/L)	6.95x
93.1 (the raw ONS aggregate ratio)	25.23x

The last row essentially reproduces main's shipped Z. So main's 25x and this PR's 4.4x are the same family of numbers at two points on one dial.

What the fix does establish, and what still matters: on main that dial was set by an accident of measurement units — express K in £ instead of £m and the spread moves to 2.2e6x, with no change in the underlying data. After this change the dial is set explicitly and Z is invariant to how either input is measured. That is the defect being fixed, and it stands.

What it does not establish is that 4.39x is the economically correct dispersion. Note in particular that the shares convention pins the effective k/l at 1.0 while the model solves at an aggregate K/L of about 4.98 (from out_b_pic/SS/SS_vars.pkl: K = 2.0594, L = 0.4134), so the calibrated Z does not reproduce the target GVA shares at the model's own factor levels. A model-consistent normalisation would be a fixed point, since Z feeds back into K/L — worth doing, but a larger piece of work than this PR.

So the accurate framing is: this replaces an indefensible implicit normalisation with an explicit, unit-invariant one. The remaining dispersion is still convention-dependent and not yet consistent with the model's own K/L. I'd rather state that than leave the stronger claim standing.

Two smaller points from the same review, both fair:

• The test's _GAMMA_SHRUNK duplicates the production shrinkage formula rather than importing it, so it would drift silently if the shrinkage changed.
• "Verified to fail on the pre-fix code" holds only trivially — main's _sector_tfp has no capital=/labour= kwargs, so the parametrised test raises TypeError there rather than failing numerically. The before/after Z table is the real evidence that the pre-fix numbers were wrong.

[vahid-ahmadi]

@jdebacker Would you have time to look at this one? It is ready and has been sitting since 31 August.

Short version: _sector_tfp divides GVA by a production aggregator with K in £m and L in thousands of jobs, and neither branch is scale-free — under CES the two are summed, and under Cobb-Douglas K^gamma_m carries a sector-specific exponent. So the calibrated cross-sector TFP dispersion was set by a units choice: 25.2x as shipped, 3.79x with L in jobs, 2.2e6x with K in £, all on identical data.

This changes M=8 model output, which is the reason it deserves a look rather than a rubber stamp. multi_sector=False (the default) is untouched. Z goes from [2.611 0.538 0.416 0.220 4.820 0.191 0.343 0.420] to [1.302 0.935 1.062 1.045 2.268 0.604 0.517 0.630].

One honest caveat I added in a comment below: unit-invariance is not convention-independence. The shares normalisation pins the effective k/l at 1.0, while the model itself solves at an aggregate K/L of about 4.98 — and varying only that relative level moves the spread across 3.64x / 4.39x / 6.95x / 25.23x. So this replaces an indefensible implicit normalisation with an explicit, unit-invariant one; it does not establish that 4.39x is the economically right answer. A model-consistent normalisation would be a fixed point and is a larger piece of work.

The Test check is red for an unrelated reason — see my note on #75, the HUGGING_FACE_TOKEN secret appears to have expired and is failing every PR in the repo.