You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The cube read path (stac_cube_items(), extracted from dft_stac_cube() in #79 and now shared with dft_stac_composite()) has three defects that predate #79. Each one changes which scenes reach a cube, silently.
A date-only datetime end bound drops the last day's scenes. Planetary Computer reads "…/2021-08-27" as 00:00Z, and Sentinel-2 over BC is acquired around 19:00Z. Measured 2026-09-28 on the packaged AOI:
2021-07-01/2021-08-31 gave 23 items, the last on 2021-08-27;
2021-07-01/2021-08-27 gave 22;
2021-07-01T00:00:00Z/2021-08-27T23:59:59Z gave 23.
dft_stac_composite() already queries with explicit times. dft_stac_cube() passes the user's string through.
The January 2022 layer of a monthly cube is not a median. Items are split at the 2022-01-25 offset boundary, and the two sides are coalesced with terra::cover(pre, post). The pre-boundary median therefore wins wherever it has data, and post-boundary scenes only fill gaps. dft_stac_composite() refuses such a window. The cube accepts it without comment.
Not verified, but worth checking at the same time: whether gdalcubes reads the SCL mask band with the view's resampling = "bilinear". If it does, mixed-class pixels at cloud edges stop matching a mask value and leak through.
Two more, measured in drift#79's code review (terra 1.9.50, gdalcubes 0.7.5):
A cube's first call and its cache hits disagree. On a miss, dft_stac_cube() returns the in-memory double stack, while the cache holds Float32. A later hit therefore differs by up to 2.98e-08. dft_stac_composite() returns the re-read cache file for exactly this reason.
Assembly leaves large temp files behind. Each build_stack() call writes an uncompressed float64 NetCDF to tempfile() and never unlinks it before the session ends. A floodplain-wide read over a BULK-sized bbox is on the order of 4 GB of tempdir per year.
Why a separate issue
Fixes 1 and 2 change what an existing cache key holds. The key hashes the datetime string, not the item set, so a fix alone leaves stale entries served under unchanged keys. Each fix needs a deliberate cache decision, such as a scheme bump or a key change (code-check.md, "Written data outlives the fix"). That belongs in its own PR, not inside a feature.
Acceptance
stac_cube_items() normalizes date-only bounds to full-day times, and cube keys change deliberately with it.
stac_cube_items() uses the same paging guards as stac_items_paged(): one implementation, not a second copy.
The monthly cube either warns on a straddling month or computes that month exactly. Whichever is chosen is documented in inst/notes/gdalcubes-pc-gotchas.md.
Problem
The cube read path (
stac_cube_items(), extracted fromdft_stac_cube()in #79 and now shared withdft_stac_composite()) has three defects that predate #79. Each one changes which scenes reach a cube, silently.A date-only
datetimeend bound drops the last day's scenes. Planetary Computer reads"…/2021-08-27"as 00:00Z, and Sentinel-2 over BC is acquired around 19:00Z. Measured 2026-09-28 on the packaged AOI:2021-07-01/2021-08-31gave 23 items, the last on 2021-08-27;2021-07-01/2021-08-27gave 22;2021-07-01T00:00:00Z/2021-08-27T23:59:59Zgave 23.dft_stac_composite()already queries with explicit times.dft_stac_cube()passes the user's string through.The cube paging has none of dft_stac_fetch() reads one STAC page — a wide AOI can build a raster from a truncated item set #51's safeguards.
stac_items_paged()indft_stac_fetch.RchecksnumberMatched, aborts on duplicate or missing ids, and strips a stalenextlink.stac_cube_items()callsrstac::items_fetch()bare, so a truncated or duplicated page set reaches the cube unnoticed.The January 2022 layer of a monthly cube is not a median. Items are split at the 2022-01-25 offset boundary, and the two sides are coalesced with
terra::cover(pre, post). The pre-boundary median therefore wins wherever it has data, and post-boundary scenes only fill gaps.dft_stac_composite()refuses such a window. The cube accepts it without comment.Not verified, but worth checking at the same time: whether gdalcubes reads the SCL mask band with the view's
resampling = "bilinear". If it does, mixed-class pixels at cloud edges stop matching a mask value and leak through.Two more, measured in drift#79's code review (terra 1.9.50, gdalcubes 0.7.5):
dft_stac_cube()returns the in-memory double stack, while the cache holds Float32. A later hit therefore differs by up to 2.98e-08.dft_stac_composite()returns the re-read cache file for exactly this reason.build_stack()call writes an uncompressed float64 NetCDF totempfile()and never unlinks it before the session ends. A floodplain-wide read over a BULK-sized bbox is on the order of 4 GB of tempdir per year.Why a separate issue
Fixes 1 and 2 change what an existing cache key holds. The key hashes the
datetimestring, not the item set, so a fix alone leaves stale entries served under unchanged keys. Each fix needs a deliberate cache decision, such as a scheme bump or a key change (code-check.md, "Written data outlives the fix"). That belongs in its own PR, not inside a feature.Acceptance
stac_cube_items()normalizes date-only bounds to full-day times, and cube keys change deliberately with it.stac_cube_items()uses the same paging guards asstac_items_paged(): one implementation, not a second copy.inst/notes/gdalcubes-pc-gotchas.md.Relates: drift#79, drift#51