Describe the bug
nullif(left, mask) should return null at positions where mask is true.
The current generic implementation adds a top-level null bitmap to the input ArrayData. This works for ordinary arrays, but it does not work for RunEndEncoded or Union arrays:
- Run-end encoded arrays derive logical nulls from their
values child array.
- Union arrays derive logical nulls from the selected child value.
- Neither representation uses a top-level null bitmap to represent logical nulls.
As a result, the null bitmap written by nullif is ignored when the result is converted back to RunArray or UnionArray. The operation succeeds but silently returns non-null values where nulls were requested.
To Reproduce
use arrow_array::{BooleanArray, Int16Array, Int32Array, RunArray};
use arrow_array::types::Int16Type;
use arrow_select::nullif::nullif;
let ree = RunArray::<Int16Type>::try_new(
&Int16Array::from(vec![1, 2]),
&Int32Array::from(vec![10, 20]),
)?;
let mask = BooleanArray::from(vec![Some(false), Some(true)]);
let result = nullif(&ree, &mask)?;
### Expected behavior
[10, null]
### Additional context
[10, 20]
The same issue occurs for UnionArray: a top-level null bitmap is added, but it is ignored by the Union logical-null representation.
Possible direction
nullif should dispatch explicitly for these types instead of using the generic top-level null bitmap path.
- For RunEndEncoded, one possible implementation is to convert mask == true positions into nullable take indices and reuse the Run-End Encoded take implementation.
- For Union, nulls must be represented by selecting a nullable child and writing a null value into that child.
- If a Union has no nullable child field, we need to define whether the kernel should return an error or widen the output field nullability.
Related
- #10909
Describe the bug
nullif(left, mask)should returnnullat positions wheremaskistrue.The current generic implementation adds a top-level null bitmap to the input
ArrayData. This works for ordinary arrays, but it does not work forRunEndEncodedorUnionarrays:valueschild array.As a result, the null bitmap written by
nullifis ignored when the result is converted back toRunArrayorUnionArray. The operation succeeds but silently returns non-null values where nulls were requested.To Reproduce