Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
c87de4e
Install gdalcubes from GitHub now that CRAN archived it (#80)
NewGraphEnvironment Sep 28, 2026
98df81b
Point the network e2e cache assertions at the #48 scheme dir (#80)
NewGraphEnvironment Sep 28, 2026
431577f
Name the GitHub install when gdalcubes is missing (#80)
NewGraphEnvironment Sep 28, 2026
ed8c7e0
Correct the test count in the 0.17.1 NEWS entry (#80)
NewGraphEnvironment Sep 28, 2026
d1ee766
Release v0.17.1 (#80)
NewGraphEnvironment Sep 28, 2026
0d6a43a
Initialize PWF baseline for #79
NewGraphEnvironment Sep 28, 2026
c29f4f3
Add NDWI and MNDWI to the index registry (#79)
NewGraphEnvironment Sep 28, 2026
ec70c9b
Extract shared cube internals from dft_stac_cube (#79)
NewGraphEnvironment Sep 28, 2026
efd8421
Add dft_stac_composite for dated reflectance composites (#79)
NewGraphEnvironment Sep 28, 2026
515c9ef
Add dated RGB layers to dft_map_interactive (#79)
NewGraphEnvironment Sep 28, 2026
5d215c7
Regenerate dft_stac_config docs for the green and blue roles (#79)
NewGraphEnvironment Sep 28, 2026
2c911e6
Order composite layers by name, not position (#79)
NewGraphEnvironment Sep 28, 2026
92d0ec8
Record the live map check and the HLS spike (#79)
NewGraphEnvironment Sep 28, 2026
4d333a0
Return the cached composite so every call gives the same values (#79)
NewGraphEnvironment Sep 28, 2026
f4e3477
Route the composite's gdalcubes check through check_gdalcubes (#79)
NewGraphEnvironment Sep 28, 2026
e100d63
Fix a stranded COG sidecar and the transition legend with x = NULL (#79)
NewGraphEnvironment Sep 28, 2026
7266fe7
Strip time before the COG write, and move .aux.json sidecars (#79)
NewGraphEnvironment Sep 28, 2026
467ad69
Pin the composite's layer reorder and sidecar-free cache in tests (#79)
NewGraphEnvironment Sep 28, 2026
2c30c3c
Document the composite and per-point chip pattern in CLAUDE.md (#79)
NewGraphEnvironment Sep 28, 2026
fecdae9
Query a floodplain-sized AOI by its convex hull (#79)
NewGraphEnvironment Sep 28, 2026
e9e46a7
BULK scale test for dft_stac_composite: 100 per-point chips (#79)
NewGraphEnvironment Sep 28, 2026
5d8769d
Re-sign STAC features before each read extent (#79)
NewGraphEnvironment Sep 28, 2026
27f6b55
Record the BULK floodplain runs 2 and 3 (#79)
NewGraphEnvironment Sep 28, 2026
df2fd14
Record the BULK floodplain-wide run; keep temp-file evidence on failu…
NewGraphEnvironment Sep 28, 2026
c528f00
NEWS for dft_stac_composite and the floodplain-scale findings (#79)
NewGraphEnvironment Sep 28, 2026
505218d
gotchas: floodplain-scale and multi-band read findings (#79)
NewGraphEnvironment Sep 28, 2026
481ad57
Fix two lints in the new cube tests (#79)
NewGraphEnvironment Sep 28, 2026
6bb3f3a
Archive planning files for issue #79
NewGraphEnvironment Sep 28, 2026
0c3cdce
Release v0.18.0 (#79)
NewGraphEnvironment Sep 28, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -74,6 +74,17 @@ breaks <- dft_rast_break(cube, start = c(2022, 1)) # abrupt change, dated (BFA
trend <- dft_rast_trend(cube) # gradual change (Theil-Sen slope + Mann-Kendall)
```

Dated reference imagery (#79): per-year median reflectance composites on the same read path as
the cube. They are cached as COGs and drawn beneath the land cover with one stretch shared across years:

```r
tc <- dft_stac_composite(aoi, years = c(2017, 2023), months = 6:7) # true colour; bands = roles
dft_map_interactive(classified, aoi = aoi, rgb = tc) # or rgb = COG URLs via titiler
# sample points: ONE CALL PER BUFFERED POINT (small, separately cached chips).
# All points in one aoi spans the floodplain bbox under one cache key.
chips <- lapply(seq_len(nrow(buf)), \(i) dft_stac_composite(buf[i, ], years = 2023))
```

## Key Patterns

- **Dual-mode maps:** `dft_map_interactive()` uses `addRasterImage()` for local SpatRasters, `addTiles()` via titiler for remote COGs
Expand Down
3 changes: 2 additions & 1 deletion DESCRIPTION
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
Package: drift
Title: Detecting Riparian and Inland Floodplain Transitions
Version: 0.17.0
Version: 0.18.0
Date: 2026-09-06
Authors@R: c(
person("Allan", "Irvine", , "al@newgraphenvironment.com", role = c("aut", "cre"),
Expand Down Expand Up @@ -53,6 +53,7 @@ Suggests:
tidyterra,
tmap
Remotes:
appelmar/gdalcubes,
bhaskarvk/leaflet.extras,
NewGraphEnvironment/flooded,
NewGraphEnvironment/gq
Expand Down
1 change: 1 addition & 0 deletions NAMESPACE
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ export(dft_rast_summarize)
export(dft_rast_transition)
export(dft_rast_trend)
export(dft_stac_classes)
export(dft_stac_composite)
export(dft_stac_config)
export(dft_stac_cube)
export(dft_stac_fetch)
Expand Down
24 changes: 24 additions & 0 deletions NEWS.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,27 @@
# drift 0.18.0

- **Dated reference imagery (#79).** The Esri and Google basemaps say what a place is but not when it changed. New `dft_stac_composite()` returns a cloud-masked median composite of any Sentinel-2 band roles for each year's run of calendar months, for example true colour for June–July in 2017 and in 2023. Values are surface reflectance, not display bytes, so a composite is data and can later be classifier input. It shares its read path with `dft_stac_cube()`. The cache entry is written as a Cloud Optimized GeoTIFF, so it can be served through titiler unchanged. floodplains#93 is the first consumer: a stratified accuracy assessment of IO LULC, reviewed at sample points across whole floodplains.
- **Chips, one per point.** To review sample points, call `dft_stac_composite()` once per buffered point. Each chip is small and cached on its own, so adding a point fetches one chip. The obvious alternative, all the points as one AOI with read tiling, was the plan until review: it returns a raster the size of the floodplain under one cache key. On the BULK floodplain, 100 chips (300 m, 2023 Jul–Aug) took 84.1 min: median 40.2 s per chip, peak RSS 0.48 GiB. Almost all of that is remote COG reads (the STAC query is about 2 s), so running chips concurrently is #85.
- **`dft_map_interactive(rgb =)`.** Composites go on the map as switchable layers beneath the land cover, as rasters (leafem) or as COG URLs (titiler). Every composite gets the same per-band stretch, pooled across years, so a brightness difference between two years is in the data, not an artefact of stretching each image to its own histogram. `x` may be `NULL` for an imagery-only map.
- **NDWI and MNDWI** join `dft_index_table()`, with the `green` and `blue` Sentinel-2 roles they need. The tests evaluate the resolved expressions on known reflectance and on post-2022 DN through the offset, not only their strings.
- **Floodplain scale, two defects fixed and one open.** Both are in the read path the cube shares, so `dft_stac_cube()` had them too.
- Planetary Computer returns HTTP 413 once a search body passes about 1 MiB. BULK's floodplain is 104,584 vertices (4.2 MB of GeoJSON), so it could not be queried at all. Above 20,000 vertices the query now uses the AOI's convex hull; below that it is unchanged.
- Signed URLs expire in about 45 minutes and were signed once, so a 54-minute tiled read lost 15 of 30 tiles, silently. Features are now re-signed before each tile. A full BULK read then got all 30 tiles in 3 h 16 min with no failures. gdalcubes reports a failed chunk only on stderr, which R cannot reliably capture, so detection is #87.
- **Open (#88):** after that full read, clipping the merged mosaic failed ("cannot read from" terra's temp file). It did not reproduce offline. A floodplain-wide composite over an AOI the size of BULK does not complete yet; chips, and reach-scale composites, do.
- **Four defects found on the way, three of them silent.**
- terra reads a multi-variable gdalcubes NetCDF alphabetically. A true-colour request came back blue, green, red, and renaming layers by position would have swapped red and blue. The live end-to-end caught it through the layer-order guard.
- Classified and transition layers were projected to EPSG:4326 and handed to leaflet with `project = FALSE`, which assumes Web Mercator pixels. At reach scale the misregistration was invisible; beneath imagery at floodplain scale it is not.
- A date-only STAC end bound excludes that day's scenes, 22 of 23 in one measured window. The composite queries with explicit times; `dft_stac_cube()` has the same defect (#83).
- Five network end-to-end assertions had listed a pre-#48 cache path since that change, so they failed on every opt-in run.
- **What a composite refuses.** A window with Sentinel-2 scenes on both sides of the 2022-01-25 offset change has no single median, because gdalcubes aggregates before the offset is applied. It is refused rather than returned as a pre-side cover. The monthly cube's January 2022 layer has that bias today (#83). A year with no usable scenes is dropped with a warning, and the other years are kept.
- **`cache_write_atomic()` moves a `.aux.json` sidecar as well as `.aux.xml`.** A COG written from a raster carrying a time (or units, varnames, longnames, metags or scoff) gets one from terra. Before this, it was stranded under the temp name on every write. The composite also strips the time the gdalcubes NetCDF gives it before writing, so the cached COG stands alone. The first fix for this was incomplete, and code review caught it: the fixture had no time and could not reach the failure.
- **Refactor, measured.** The STAC query, offset split and tiled assembly moved out of `dft_stac_cube()` into helpers shared with the composite. A live untiled NDVI cube from main and from the refactor are identical cell for cell (max difference 0), and a new test pins how `dft_stac_cube()` calls its cache key. Both matter because the frozen-key test cannot see the call site.
- **HLS is #82.** A live spike found that CMR-STAC rejects `intersects` and ignores the CQL2 cloud filter, that reading needs a real Earthdata Login (`earthdatalogin`'s default credentials now return 401), and that S30 and L30 name NIR and SWIR differently. #82 records these and the source shape they imply.

# drift 0.17.1

- **gdalcubes installs from GitHub (#80).** gdalcubes was archived on CRAN on 2026-09-16. That broke the pkgdown workflow's dependency resolve, which had failed on every push since 2026-09-26, and it left new users unable to install what `dft_stac_fetch()` and `dft_stac_cube()` need. `appelmar/gdalcubes` is now in `Remotes:` with no suffix, and the README says to install it with `pak::pak("appelmar/gdalcubes")`. So does the prompt `dft_stac_fetch()`, `dft_stac_cube()` and `dft_index_expr()` raise when it is missing. rlang's own install offer means CRAN and would fail. Upstream master (0.7.5) has merged the `filter_geom` segfault fix from `NewGraphEnvironment/gdalcubes@newgraph`, so the fork is no longer needed. On 0.7.5 the offline suite passes (1007 tests), and so does the opt-in network end-to-end for the cube and fetch paths (109 and 165). That run also surfaced five network assertions that had listed the pre-#48 cache path since that change, so they failed on every opt-in run. They now use `cache_scheme_dir()`.

# drift 0.17.0

- **The article gains patch shape and location, and the corridor question comes back negative (#73).** [What a Land-Cover Change Figure Is Made Of](https://newgraphenvironment.github.io/drift/articles/temporal-composition.html) reported *how much* of a two-epoch comparison is temporally unstable and never raised the spatial axis. It now reports both grains. Body prose 731 -> 1469 words against a cap raised 1000 -> 1500; the cap is a variable now, because the literal appeared twice on one line and changing one would have left the message lying.
Expand Down
2 changes: 1 addition & 1 deletion R/dft_index_expr.R
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,7 @@ dft_index_expr <- function(cube,
roles = NULL,
scale = NULL,
offset = NULL) {
rlang::check_installed("gdalcubes", reason = "to apply an index to a cube")
check_gdalcubes("to apply an index to a cube")
cfg <- dft_stac_config(source)
roles <- roles %||% cfg$roles
scale <- scale %||% cfg$scale %||% 1
Expand Down
Loading
Loading