Improve CI build times and Rust cache reuse

[itsyaasir]

Problem

The Notarization Toolkit CI spends increasingly long compiling Rust dependencies and rebuilding artifacts. Comparing two successful main-branch runs, elapsed time grew from 42.1 to 68.3 minutes (62%), while combined job time grew from 148.8 to 234.6 minutes. Queue time was negligible in both runs.

Measurement	20 August run	8 September run
Overall elapsed time	42.1 min	68.3 min
Linux release build	15.5 min	30.4 min
macOS release build	14.9 min	42.5 min
Windows release build	32.9 min	53.4 min

These are representative comparisons, not a claim that every run is slower than the previous one. Cache reuse causes substantial variation. The investigation sampled 59 runs and inspected five job logs.

Findings

• The PoI test dependency graph adds substantial build work. test-cluster pulls in IOTA node/core and RocksDB dependencies. Windows builds workspace tests even though it skips their execution.
• Linux compiler caching is ineffective. The setup pins sccache v0.2.15. The September Linux job reported 2,896 non-cacheable requests with “multiple input files” and zero Rust cache hits. Upgrade sccache and verify actual Rust cache reuse.
• CI invalidates reusable build state. Linux runs cargo clean after feature checks, removing restored release artifacts too. Setup also runs cargo update and rustup update each time, introducing dependency and compiler churn.
• The test step includes another substantial compilation. On September 8, compilation consumed 8m18s of the 14-minute Linux test step and 12 minutes of the 18.1-minute macOS test step. Move packages are published after the initial Rust build; build scripts watch Move.lock, making publication a suspected invalidation trigger. Confirm this with Cargo fingerprint diagnostics.
• The previous PoI example runner rebuilt almost every invocation. Approximately 7.2 of its 7.3 minutes were compilation. The unified example runner now executes prebuilt binaries; measure the resulting improvement before making further changes here.

WASM builds were roughly 2–4 minutes each in the September run and were not the critical path. Prioritize native compilation and repeated builds first.

Proposed work

• Upgrade and consistently configure sccache across runners; record Rust cache hits, misses, and non-cacheable reasons.
• Preserve release artifacts during feature checks by removing the blanket clean or isolating feature-check output.
• Diagnose rebuild triggers and arrange Move publication and Rust compilation so tests and examples reuse the final artifacts.
• Make dependency and Rust toolchain selection reproducible for PR runs; consider separate scheduled dependency-update coverage.
• Review whether heavy PoI integration-test dependencies need compilation on every OS while preserving intended platform coverage.
• Measure the unified example runner and ensure example execution does not trigger compilation.

Validation

• Compare several successful runs before and after changes, separating cold-cache and warm-cache results.
• Report overall elapsed time, combined job time, native build time, test-step compilation time, and Rust cache statistics.
• Preserve required feature checks, tests, examples, and platform coverage; document any deliberate coverage changes.
• Keep sandbox tests single-threaded and verify the full workflow remains green.

This issue tracks the investigation and implementation; no CI optimization changes were made as part of the timing analysis.