Dated reference imagery: dft_stac_composite, RGB map layers, NDWI/MNDWI (#79)

[NewGraphEnvironment]

Fixes #79

Stacked on #84 (#80, gdalcubes from GitHub). Merge #84 first; its four commits disappear from this diff once it lands.

Why

floodplains#93 needs dated reference imagery to run a stratified accuracy assessment of IO LULC at sample points across whole floodplains. The Esri and Google basemaps have no capture date. drift could fetch Sentinel-2, but only as a single-index monthly cube.

What

• dft_stac_composite(), new. For each year's run of calendar months it returns a cloud-masked median of any band roles, as surface reflectance. It shares the read path with dft_stac_cube(), which was refactored into shared helpers; a live untiled cube from main and from the refactor match cell for cell (max diff 0). The cache is written as a COG.
• Per-point chips. To review sample points, call it once per buffered point, so each chip is small and cached on its own. The planned alternative, all the points in one AOI with tiling, turned out in plan review to give a floodplain-sized raster under a single cache key.
• dft_map_interactive(rgb =): composites as rasters (leafem) or COG URLs (titiler), drawn beneath the land cover with one per-band stretch shared across years. x may be NULL.
• NDWI and MNDWI in dft_index_table(), with the green and blue roles.
• HLS is split to HLS (Harmonized Landsat Sentinel-2) as a cube source: Earthdata auth, CMR-STAC query, bit-mask Fmask #82. A live spike found:
  • CMR-STAC returns 500 on intersects and ignores the CQL2 cloud filter;
  • reading needs a real Earthdata Login, because earthdatalogin's default credentials now return 401;
  • S30 and L30 name their bands differently;
  • Fmask is a bit mask.

Defects found and fixed here

found by	defect	fix
live e2e	terra reads gdalcubes' multi-variable NetCDF alphabetically (blue, green, red); renaming by position would silently swap red and blue	select layers by name
live e2e	first call returned the double stack while the cache holds Float32, so it differed from later cache hits	return the re-read cache
probe	a date-only STAC end bound drops that day's scenes (22 of 23)	explicit times in the composite; the cube's copy is #83
plan review	classified layers went to leaflet in 4326 with project = FALSE, so they were misregistered against anything beneath them	project to 3857
code-check R1+R2	a time on the stack made the COG writer emit a .aux.json that cache_write_atomic() stranded. The R1 fix was incomplete: the time comes from the NetCDF reader, and the fixture had none	strip time before the write; the atomic publisher moves .aux.json too
code-check R1	transition legend dropped with x = NULL	split the legend gate
BULK scale test	Planetary Computer returns HTTP 413 past about 1 MiB of POST body (accepted at 860 KB, rejected at 1.07 MB). BULK's floodplain is 104,584 vertices (4.2 MB), so neither the cube nor the composite could query it	above 20,000 vertices, query by convex hull; queries under that are unchanged
BULK scale test	Signed URLs expire in about 45 min. Features were signed once, and the 54-min tiled read lost 15 of 30 tiles. gdalcubes reports failed chunks only on stderr, so the partial result would have been cached as complete	re-sign features before each read extent (live: corrupted tokens read 102,364 cells after re-signing, 0 before). Detection is #87: stderr capture was tried and caught the report in 1 of 4 configurations, so it was not shipped

Also fixed: five opt-in network assertions left on the pre-#48 cache path (they are on #84 too).

Verification

• Offline devtools::test(): FAIL 0, SKIP 15, PASS 1113. lintr: 0 lints on changed lines. pkgdown::check_pkgdown(): clean.
• DRIFT_TEST_NETWORK=true, cube (incl. tiled vs untiled): PASS 110.
• DRIFT_TEST_NETWORK=true, composite: PASS 64. It checks 3 bands in order, reflectance values, LAYOUT=COG, a declared NoData, identical values on a cache hit, and that false-colour NIR is more than 2× red.
• Mutation checks each turn a test red: band order, 4326 projection, key call site, offset-straddle refusal, per-image stretch, empty-year skip, time strip, .aux.json move, layer reorder call, transition legend, transition centring.
• Live map: 2017 Jun–Jul and 2023 Aug–Sep on the packaged AOI in Chrome. Both layers register against the Esri basemap, and 2023 reads darker and hazier under the shared stretch.
• /code-check: three rounds, and the loop was ended by an enumeration.
  • R1: 2 findings.
  • R2: 1 finding, a defect inside the R1 fix.
  • R3: enumerated every raster write in R/ (none emits a sidecar on real output) and every test stub against its real upstream. It found 2 test gaps, now fixed.

BULK scale test (data-raw/benchmark_composite_bulk.R, RSS sampled every 2 s)

stage	result
chips: 100 buffered points (300 m), 2023 Jul–Aug	84.1 min; per chip median 40.2 s, mean 50.5 s, max 140.9 s; 3,660 cells/chip; cache 3.8 MB; peak RSS 0.48 GiB
floodplain-wide, tile_size = 20000	run 1: HTTP 413 at the query. Run 2: 72 items, 30 tiles, 54 min, peak RSS 5.09 GiB; 15 tiles lost to expired tokens, then the final COG write failed (a synthetic floodplain-size COG writes fine: 82 s, 459 MB). Run 3, with re-signing: killed at a 2-h cap after 16 tiles (5.4–11.9 min each), no chunk failures. Run 4: all 30 tiles in 3 h 16 min, 0 chunk failures, peak RSS 3.90 GiB, then terra::mask could not read the merged mosaic's temp file. Open as #88: not reproduced offline (a synthetic 30-tile merge and mask with the real polygon passes in 0.7 min), and the benchmark now inventories temp files on failure. Run 2's COG-write failure was probably the same thing

Chip cost is COG reads, not the query (1.4–2.5 s of about 50), so concurrency is #85.

Follow-ups filed

• HLS (Harmonized Landsat Sentinel-2) as a cube source: Earthdata auth, CMR-STAC query, bit-mask Fmask #82: HLS source.
• Cube read path: date-only end bound drops the last day, no #51 paging guards, Jan 2022 cover bias, miss/hit drift, temp NetCDFs #83: pre-existing cube read-path defects (date-only end bound, missing dft_stac_fetch() reads one STAC page — a wide AOI can build a raster from a truncated item set #51 paging guards, Jan 2022 cover, miss/hit drift, temp NetCDFs).
• Composite chips run serially at ~50 s each: run sample-point chips concurrently #85: concurrent chips.
• Failed gdalcubes chunk reads are silent: a partial cube passes the empty check and is cached #87: failed gdalcubes chunk reads are silent, and a partial cube is cached.
• Floodplain-wide tiled composite: terra::mask cannot read the merged mosaic after a 3-hour read #88: the floodplain-wide tiled composite fails at the clip after a full read.
• flooded#60: the same gdalcubes resolve failure in flooded.

Known limitation at merge: a floodplain-wide composite over an AOI the size of BULK does not yet complete (#88). Per-point chips, the path floodplains#93 needs first, and reach-scale composites do.

The last three commits are NEWS, the PWF archive (planning/archive/2026-09-issue-79-dated-reference-imagery/, whose README has the measurement record) and Release v0.18.0.

Relates to NewGraphEnvironment/sred-2025-2026#16

🤖 Generated with Claude Code

https://claude.ai/code/session_01PRhUJsuKABLfpBGktPoiBN