Repository navigation

[SPARK-60686][CONNECT] Use Arrow columns for supported pandas timestamps - #130

Draft

dbtsai wants to merge 23 commits into

apache:masterfrom

dbtsai:codex/spark-60686-pandas-timestamps

Draft

[SPARK-60686][CONNECT] Use Arrow columns for supported pandas timestamps#130
dbtsai wants to merge 23 commits into
apache:masterfrom
dbtsai:codex/spark-60686-pandas-timestamps

Commits on Oct 29, 2026
- Export Arrow tables through the C Stream interface
  dbtsai
  committed
  d7cebc3
- Use arrow-rs reader conversion for the C Stream handoff
  dbtsai
  committed
  afb69bc
- Simplify Arrow handoff tests and add live-server smoke cases
  
  dbtsai
  and
  Isaac
  committed
  d89c01b
- Skip Arrow handoff tests when protoc is unavailable
  
  dbtsai
  and
  Isaac
  committed
  fe79d54
- [SPARK-60675][CONNECT] Support PyArrow Table input through the C Stream interface
  
  dbtsai
  and
  Isaac
  committed
  c4bbfd3
- [SPARK-60675][CONNECT] Upload large Arrow inputs as cached relations
  
  dbtsai
  and
  Isaac
  committed
  ce89b8d
- [SPARK-60676][CONNECT] Use Arrow columns for signed int64 pandas inputs
  dbtsai
  committed
  4a5807e
- [SPARK-60676][CONNECT] Validate integer upload scales against mock RPC
  dbtsai
  committed
  38568fe
- [SPARK-60676][CONNECT] Cover leading null inference and non-nullable schemas
  dbtsai
  committed
  7009bb7
- [SPARK-60676][CONNECT] Exercise named schemas through mock rename relations
  dbtsai
  committed
  fee0800
- [SPARK-60676][CONNECT] Avoid Arrow configuration overhead for small NumPy frames
  dbtsai
  committed
  50b00d1
- [SPARK-60676][CONNECT] Retain IPC row fallback for NUL field names
  dbtsai
  committed
  a8774f0
- [SPARK-60676][CONNECT] Include pandas regressions in the existing Arrow CI suite
  dbtsai
  committed
  e7e3094
- [SPARK-60686][CONNECT] Use Arrow columns for supported pandas timestamps
  dbtsai
  committed
  979fadc
- [SPARK-60686][CONNECT] Preserve timestamp row fallback for customized pandas input
  
  dbtsai
  and
  Isaac
  committed
  5f31a4c
- [SPARK-60676][CONNECT] Preserve integer pandas schema and fallback semantics
  
  dbtsai
  and
  Isaac
  committed
  8a11a4d
Commits on Oct 30, 2026
- [SPARK-60676][TESTS] Preserve subclass column labels on pandas 3
  
  dbtsai
  and
  Isaac
  committed
  d26e45b
- [SPARK-60676][CONNECT] Merge upstream Arrow handoff to restore CI
  
  dbtsai
  and
  Isaac
  committed
  aff23b8
- [SPARK-60686][CONNECT] Merge upstream Arrow output guards
  
  dbtsai
  and
  Isaac
  committed
  4b92da4
- [SPARK-60676][CONNECT] Preserve customized pandas columns hooks
  
  dbtsai
  and
  Isaac
  committed
  964e16e
- [SPARK-60676][CONNECT] Preserve rebound pandas index storage ordering
  
  dbtsai
  and
  Isaac
  committed
  c4284a3
- [SPARK-60686] Include reviewed canonical integer input dependency
  dbtsai
  committed
  fd115ce
- [SPARK-60686][CONNECT][PYTHON] Preserve timestamp columns hook order
  
  dbtsai
  and
  Isaac
  committed
  7b53529

Commits

Commits on Oct 29, 2026

Commits on Oct 30, 2026