Repository navigation
perf(cache): report publication phases and archive creation - #4988
SamuelSchlesinger wants to merge 2 commits into
Conversation
Part of #4972. Record revision/toolchain/thread context, root archive counts, and phase wall/self/child CPU times without changing staged bytes or publisher credentials. Keep optional JSON reports outside the publication inventory. Real-Lake controls verify missing archive packing without OLean changes and reuse when archives are retained. Validation: publication tests (19), full-check parity (16), Operator manual tooling (72), manual source/link checker, and diff checks passed. Operator troubleshooting and cache guide updated; Agent Operator manual roles and workflow are unchanged. Hosted integration validation and independent exact-head/rendered review remain.
|
Agent review claim: coordinator |
SamuelSchlesinger
left a comment
There was a problem hiding this comment.
Independent agent review (coordinator agy-634bba4c, session 634bba4c-9d3c-4f80-8d5b-6097f495b913) at e58c8d9041a7804a3abe2481863b310d8b48b692: APPROVE (holding queue until ci-checks and Operator's Manual PDFs finish).
- Staging instrumentation (
scripts/cache/publish.py).stage_phaserecords wall, Python CPU, and optionalresource.RUSAGE_CHILDRENuser/system CPU deltas inside afinallyblock, preserving the original exception on command failure.--reportrejects paths insidedestinationbeforedestinationis created, andpublish.validateconfirms diagnostic JSON never enters the staged inventory. None of the CI callers in.github/workflows/lean_action_ci.ymlparsepublish.py stagestdout, so the phase lines andCACHE_STAGING_METRICSJSON line are safe onmainpushes. - Tests (
scripts/test_cache_publish.py). Independently rantest_cache_publish.py(includingRealLakePublicationTestsunderLEAN_NUM_THREADS=2) andtest_full_check.py(35 tests, all passing): deleting.ltararchives andLAKE_CACHE_DIRcauseslake build ... -oto pack the two root archives while leaving OLean digests and mtimes unchanged, and the second run seesarchives_before.count == 2. - Manuals & docs. Checked the pinned
v4.36.0-rc2Lake/Build/Module.lean#L1336(trackOutputsIfEnabled/packLtar) reference and confirmed inlean_action_ci.yml(L246 vs L370) thatSave .lake to the Actions cacheruns beforeStage cache publication.manual/workflows/troubleshooting.mdadds a concise section linkingscripts/cache/README.md#profile-cache-publication(disjoint from #4982's hunk), andAgent Operator's Manualneeds no edit because roles and workflow are unchanged.test_operator_manual.py(72 tests),check_operator_manual.py(5,656 links), andcheck_hygiene.pypass.
Hold: waiting for hosted ci-checks and the rendered Operator's Manual PDFs artifact at e58c8d9041a7804a3abe2481863b310d8b48b692 before queueing. This is agent review, not human review.
Run each warm archive control with an empty private artifact cache, and assert retained archive hashes/mtimes as well as OLean outputs stay unchanged. Clarify that aggregate metrics JSON is emitted after successful staging; failed phases retain textual diagnostics and command failures. All 107 publication/parity/manual fixture tests passed.
SamuelSchlesinger
left a comment
There was a problem hiding this comment.
Independent agent delta review (coordinator agy-634bba4c, session 634bba4c-9d3c-4f80-8d5b-6097f495b913) at f39b476a6e17142c90b4cabd5c434712acc2bbaa (over e58c8d9041a7804a3abe2481863b310d8b48b692): APPROVE (holding queue until ci-checks and Operator's Manual PDFs finish).
scripts/cache/README.md: accurately clarifies thatCACHE_STAGING_METRICSis emitted only on a successful stage while a failed phase keeps its textual diagnostic and command failure.scripts/test_cache_publish.py: clearsLAKE_CACHE_DIRbefore theretained-archivesiteration as well (matching GitHub Actions retention) and asserts that retained.ltarmtimes and digests are unchanged. Re-ranLEAN_NUM_THREADS=2 python3 -B -m unittest scripts/test_cache_publish.py(19 tests OK) andpython3 -B scripts/check_hygiene.py.
This is agent review, not human review.
|
Strengthened the native control in The real 244-module probe and its limits are recorded on #4972; it confirms archive-creation work with unchanged compiled outputs. Full-sized producer profiling remains to be observed after integration. |
Cache publication can spend minutes inside its mapping-producing build even when strict build and tests are warm. Report separate wall/self/child CPU timings for that build, archive staging and inventory hashing, plus source/toolchain/thread context and root archive counts/bytes. An optional JSON report stays outside the published inventory; commands, staged bytes and publisher authorization are unchanged.
Part of #4972. A controlled real-module probe produced 244 mapping entries without recompilation: 12.31 seconds with archives absent, versus 5.94 seconds with archives retained. Each arm used an empty private artifact cache and unchanged compiled outputs. This is one subset probe, not a full-CI speedup claim. The native regression also checks missing versus retained archives with fixed tiny sources, fresh private caches in both arms, and unchanged archive/OLean hashes and mtimes. Full-sized producer measurements become visible in normal main publication logs after integration.
Validation
107 direct tests passed: publication (19), full-check parity (16), Operator manual tooling (72). The source/link checker and diff checks passed. Native controls run under
LEAN_NUM_THREADS=2; timing values are diagnostic, not performance thresholds. Hosted integration checks and independent exact-head/rendered review remain. External coordinatoragy-634bba4chas review ownership; the delta atf39b476a6strengthens the retained-archive control and clarifies success-only aggregate JSON.Manual impact
The cache guide explains metric meanings, the clean-checkout staging command, comparison controls and limits; Operator troubleshooting links it. The Agent Operator's Manual needs no edit because roles, authorization and workflow are unchanged. No Lean examples or mathematical claims changed.
AI assistance: Codex
/rootauthored and checked this change for Samuel Schlesinger. Independent review is required before integration.