Skip to content

feat(python/sedonadb): allow 1-D rasters in Raster.from_numpy/lazy - #1097

Open
james-willis wants to merge 3 commits into
apache:mainfrom
james-willis:jw/raster-1d-support
Open

feat(python/sedonadb): allow 1-D rasters in Raster.from_numpy/lazy#1097
james-willis wants to merge 3 commits into
apache:mainfrom
james-willis:jw/raster-1d-support

Conversation

@james-willis

Copy link
Copy Markdown
Contributor

Summary

Raster.from_numpy() and Raster.lazy() rejected any array/shape with fewer than 2 dimensions, on the assumption the trailing two axes are always a spatial (y, x) pair. That's a Python-layer constraint only — the underlying RasterSchema already allows empty spatial_dims/spatial_shape, and Rust's own RasterMetadata::width()/height() are already Option-returning (rust/sedona-raster/src/builder.rs even has a test asserting a single-dim spatial_dims=["x"] case).

This blocks representing any purely non-spatial N-D chunk (a plain time series, an embedding vector, a reduction result that collapses below 2-D) as a raster at all — hit directly while building an xarray ChunkManagerEntrypoint backend on top of sedonadb (.mean(["y", "x"]) on a (time, y, x) array produces a (time,) result with nowhere to go).

  • Relax the from_numpy()/lazy() guard from ">= 2 dims" to ">= 1 dim".
  • _build_raster: for a 1-D array, spatial_dims/spatial_shape are empty rather than assuming a spatial pair; crs/transform are rejected explicitly in that case (ambiguous without a spatial pair) rather than silently ignored.
  • Raster.width/height become Optional[int], None for a 1-D raster, matching the Rust side's existing Option. __repr__ falls back to showing the band's source_shape when there's no width/height.
  • _resolve_dim_names needed no changes: it already requires explicit dim_names for any ndim != 2, which correctly covers ndim == 1 too.

Test plan

  • pytest python/sedonadb/tests/test_raster.py — 33 passed (updated the two existing invalid-shape tests, which asserted the old "at least two" rejection for a 1-D shape that's now valid input; added test_raster_lazy_1d / test_raster_from_numpy_1d plus cases for the zero-dim and crs/transform-on-1D rejections)
  • pytest --doctest-modules python/sedonadb/python/sedonadb/raster.py — 2 passed (new 1-D examples in lazy/from_numpy docstrings)
  • ruff check / ruff format --diff clean
  • Confirmed unrelated to two pre-existing test_rs_value_point failures (reproduce identically on origin/main)

Raster.from_numpy() and Raster.lazy() rejected any array/shape with
fewer than 2 dimensions, on the assumption the trailing two axes are
always a spatial (y, x) pair. That's a Python-layer constraint only --
the underlying RasterSchema already allows empty spatial_dims/
spatial_shape, and Rust's own RasterMetadata::width()/height() are
already Option-returning (rust/sedona-raster/src/builder.rs even has a
test asserting a single-dim spatial_dims=["x"] case).

This blocks representing any purely non-spatial N-D chunk (a plain
time series, an embedding vector, a reduction result that collapses
below 2-D) as a raster at all.

- Relax the from_numpy()/lazy() guard from ">= 2 dims" to ">= 1 dim".
- _build_raster: for a 1-D array, spatial_dims/spatial_shape are empty
  rather than assuming a spatial pair; crs/transform are rejected
  explicitly in that case (ambiguous without a spatial pair) rather
  than silently ignored.
- Raster.width/height become Optional[int], None for a 1-D raster,
  matching the Rust side's existing Option. __repr__ falls back to
  showing the band's source_shape when there's no width/height.
- _resolve_dim_names needed no changes: it already requires explicit
  dim_names for any ndim != 2, which correctly covers ndim == 1 too.

Updated the two existing invalid-shape tests (they asserted the old
"at least two" rejection for a 1-D shape, which is now valid input)
and added test_raster_lazy_1d / test_raster_from_numpy_1d plus cases
for the zero-dim and crs/transform-on-1D rejections.
@james-willis
james-willis marked this pull request as draft July 30, 2026 02:35
@github-actions
github-actions Bot requested a review from prantogg July 30, 2026 02:36
@james-willis
james-willis marked this pull request as ready for review July 30, 2026 03:47

@paleolimbot paleolimbot left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok by me, but are these just "Band"s without the spatial wrapping? Or do we need to update some documentation somewhere that a missing height/width can happen?

@james-willis

Copy link
Copy Markdown
Contributor Author

@paleolimbot Theres some argument to be made here about layers a tensor type and putting spatial raster on top of it but im avoiding that for now.

I will update this PR to update documentation for those UDFs. Ill also check to see if the UDFs need any kind of handling

…er with no spatial (y, x) pair instead of panicking

RasterRef::metadata() now returns Result instead of panicking when the
raster has no spatial (y, x) pair (empty spatial_shape) or a malformed
transform. Per-row UDF closures guard on the error and emit NULL for that
row; helpers, dataset builders, and infrastructure propagate the error
with `?`; the Display impl renders the available dimension info instead of
the width x height / geotransform extent.
@paleolimbot

Copy link
Copy Markdown
Member

You could add a is_band matcher that just matches the band type (or just write all your initial implementations on 2D rasters). I don't love the idea of a bunch of test code written against "rasters" that aren't actually spatial since that's not really what the type is for...do you really need the 1D case?

@james-willis

Copy link
Copy Markdown
Contributor Author

I see your concern. We need to allow 1D bands in the type if we want to support aggregations taht return nonspatial results. Queries that aggregate along 1 or more of the spatial dimensions for example:

What is the mean temp over the whole region for each timestep.

This 1D case can of course be represented as some kind of array, but what about when theres 2 nonspatial dimensions?

What is the mean time over the whole region for each timestep and elevation.

@james-willis

Copy link
Copy Markdown
Contributor Author

Im going to want 0d rasters soon too...

@paleolimbot

Copy link
Copy Markdown
Member

The way the raster type is set up is a very bad fit for returning anything other than 2D things with spatial significance. The "band" as we currently define it is a pretty reasonable way to do this (it's basically just the ndarray with no spatial tag). Can pure ndarray manipulations operate on that data type? They could also accept rasters (they would just not return them because the spatial dimensions would be aggregated away).

There are also some aggregations that make sense to return a Raster (very specifically, ones that aggregate into the existing y, x dimensions), but that can't be known at plan time.

# Conflicts:
#	rust/sedona-raster-functions/src/executor.rs
#	rust/sedona-raster-functions/src/footprint.rs
#	rust/sedona-raster-functions/src/rs_envelope.rs
#	rust/sedona-raster-functions/src/rs_georeference.rs
#	rust/sedona-raster-functions/src/rs_geotransform.rs
#	rust/sedona-raster-functions/src/rs_pixel_functions.rs
#	rust/sedona-raster-functions/src/rs_setsrid.rs
#	rust/sedona-raster-functions/src/rs_size.rs
#	rust/sedona-raster-functions/src/rs_value.rs
#	rust/sedona-raster-functions/src/rs_values.rs
#	rust/sedona-raster-gdal/src/gdal_common.rs
#	rust/sedona-raster-gdal/src/gdal_dataset_provider.rs
#	rust/sedona-raster-gdal/src/rs_as_geotiff.rs
#	rust/sedona-raster-gdal/src/rs_as_raster.rs
#	rust/sedona-raster-gdal/src/rs_clip.rs
#	rust/sedona-raster-gdal/src/rs_frompath.rs
#	rust/sedona-raster-gdal/src/rs_metadata.rs
#	rust/sedona-raster-gdal/src/rs_reproject_match.rs
#	rust/sedona-raster-gdal/src/rs_resample.rs
#	rust/sedona-raster-gdal/src/rs_tile.rs
#	rust/sedona-raster-gdal/src/utils.rs
#	rust/sedona-raster/src/affine_transformation.rs
#	rust/sedona-raster/src/array.rs
#	rust/sedona-raster/src/builder.rs
#	rust/sedona-raster/src/display.rs
#	rust/sedona-raster/src/traits.rs
#	rust/sedona-spatial-join-raster/tests/spatial_join_integration.rs
#	rust/sedona-testing/src/benchmark_util.rs
#	rust/sedona-testing/src/raster_spec.rs
#	rust/sedona-testing/src/rasters.rs
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants