[WIP] feat(raster): view machinery for non-identity band views

[james-willis]

Summary

Integrates the view-machinery layer into the raster type. Bands can be constructed and read with non-identity view entries — slices, broadcasts, axis permutations, and reverse-iteration. The byte-access surface is a single zero-copy accessor, nd_buffer(), which exposes the source buffer plus the visible region's shape/strides/offset; consumers that need flat row-major bytes borrow them via NdBuffer::as_contiguous(), which errors on strided layouts rather than allocating.

Builds on two pieces that already landed on main: the ViewEntries view-spec module (validate / compose / visible_shape / is_identity, #934) and the i64 source_shape column (#927). This PR wires them into band construction and reads.

What's in this PR

Three layers, plus the GDAL bridge:

Trait surface — zero-copy byte access (rust/sedona-raster/src/traits.rs)

• NdBuffer::is_contiguous() — pure function of (shape, strides, data_type). C-order packed-stride check, innermost-first, offset-agnostic so an outer-axis slice from a nonzero offset still counts as contiguous. Broadcast (stride 0), reverse (negative stride), permuted, gapped, and inner-strided layouts are not contiguous; a zero-extent axis is trivially contiguous.
• NdBuffer::as_contiguous() -> Result<&[u8]> — borrows the visible bytes zero-copy when contiguous, else errors directing the caller to RS_EnsureContiguous (RS_EnsureContiguous UDF: materialize strided band views to contiguous bytes via an explicit plan node #899). Never allocates.
• Unit tests for the contiguity classifier.

Builder (rust/sedona-raster/src/builder.rs)

• start_band_with_view() API (args bundled in StartBandWithViewArgs).
• with_view() API (args bundled in WithViewArgs) — compose a new view over an existing band.
• Tests cover the construction and Arrow-serialization paths: identity-view equivalence with start_band, 0-D rejection, view-step validation, the view column being NULL for identity bands, and Arrow IPC round-trip.

Reader (rust/sedona-raster/src/array.rs)

• RasterRefImpl::band resolves the band row, decodes the data type, and composes view -> byte strides + byte offset (compose_band_layout / check_band_buffer_bounds) with checked arithmetic; malformed views and arithmetic overflows route through sedona_internal_datafusion_err! so they surface with the standard "SedonaDB internal error" framing.
• nd_buffer() returns the source buffer + shape/strides/offset (zero-allocation strided access) and is the sole byte accessor. OutDb bands return an error from nd_buffer() (backend resolvers are tracked separately).
• Reader-level byte-layout tests: identity and outer-axis-slice contiguity (zero-copy borrow), strided nd_buffer strides across dtypes, broadcast and negative-step rejection, and permutation+slice composition.

GDAL bridge (rust/sedona-raster-gdal/src/{gdal_common,gdal_dataset_provider}.rs)

• raster_ref_to_gdal_mem borrows each band via nd_buffer()?.as_contiguous()?. is_2d() already requires an identity view, so the borrow always succeeds — no owned-bytes copy is threaded through the dataset provider.

[paleolimbot]

This is very cool!

I think I got there by the end in the inline comments, but I think a struct ViewEntries with associated functions in src/view_entries.rs where the non serialize-to-arrow and deserialize-from-arrow logic behind this lives will make this clearer.

The main issue I have with the implementation here is that it relies on hidden Vec<u8>s that aren't reusable. To manage memory properly we are going to have to register some scratch spaces with the session's memory pool and reuse them between iterations of a loop. Hiding them within various structures for convenience is going to make it tricky to implement a correct pattern going forward.

[paleolimbot]

A BandBuilder may help you quite a bit here since there are a lot of None that a default constructor could fill in

[james-willis]

I am generally unhappy with the band builder interfaces; I agree I am making them substantially worse. Not sure how you feel about deferring this but I started putting items for band builder improvements in this ticket:

#896

[paleolimbot]

These kinds of tests you probably don't want in this file. This file is about serializing stuff into Arrow and it's tricky to see what's going on...I'm not sure what the perfect abstraction is here but it should probably live in src/views.rs with tests and some pub functions that can be benchmarked.

Proabably struct ViewEntries { inner: Vec<ViewEntry> } with associated methods would make this a bit cleaner than the &[ViewEntry] + free functions.

[james-willis]

I reduce the tests here. they should now be just builder-integration specific. will confirm before resolving this comment

[james-willis]

Somewhat offtopic but are you going to have the same concern with lazy outdb loading as well? (Outdb lazy loading + zarr loader is going to be higher on my priority list next week than this PR)

The main issue I have with the implementation here is that it relies on hidden Vecs that aren't reusable. To manage memory properly we are going to have to register some scratch spaces with the session's memory pool and reuse them between iterations of a loop. Hiding them within various structures for convenience is going to make it tricky to implement a correct pattern going forward.

[paleolimbot]

Possibly? I know it seems like a lot of comments but what I'm getting at is:

let mut scratch = Vec::new();
for raster in rasters {
  let data_slice = raster.band(0)?.contiguous_data(&mut scratch)?;
  do_stuff(data_slice);
}

Or on the day we need to track memory:

let mut max_capacity = 0;
for raster in rasters {
  max_capacity = max_capacity.max(raster.band(0)?.contiguous_data_alloc_size());
}

if max_capacity > config_options.raster.max_scratch_alloc {
  // error
}

// If this ends up being a lot, frequently, we may also want to use a MemoryReservation to track it
// Probably this helps contiguous_data be faster since in theory there is only one heap allocation.
// You could probably use some of the unsafe modifiers to also ensure that there are no bounds
// checks in your materializer slowing things down
let mut scratch = Vec::with_capacity(max_capacity);
for raster in rasters {
  let data_slice = raster.band(0)?.contiguous_data(&mut scratch)?;
  do_stuff(data_slice);
}

I'm not sure exactly how loading works yet but you might want something similar (with the hiccup that you need some number of async workers doing IO so you also need the same number of scratch buffers).

(I have the same comment for pretty much everything that heap allocates in a loop, which is why, for example, our geometry writers write WKB directly into the Arrow output instead of returning Vec<u8>).

[james-willis]

subsumed by #1113