[dbtsai]

What changes were proposed in this pull request?

Add DataFrame.__dir__ and _ipython_key_completions_ in the PyO3 binding, matching Python Connect. dir(df) returns sorted, deduplicated class attributes plus all column names. Key completion returns column names in schema order, including duplicates and non-identifiers. Both methods propagate schema errors and release the GIL during analysis. Class inspection avoids recursive instance dir calls.

Why are the changes needed?

For a DataFrame with column c0, master omits c0 from dir(df) and raises AttributeError for _ipython_key_completions_(). This prevents interactive column discovery and bracket completion.

SPARK-60681

Does this PR introduce any user-facing change?

Yes. Interactive completion includes DataFrame columns while preserving existing methods. This is based directly on master and is independent of schema caching in #120: each completion call still performs schema analysis.

How was this patch tested?

• Added 13 separate-process mock gRPC tests covering ordinary/non-identifier/Unicode/empty/duplicate names, method collisions, fresh result lists, schema errors and successful retries. First-access RPC checks use independent fresh plans so the tests also allow schema caching. All pass against the locked release wheel.
• Completion tests and their uniquely prefixed helpers live in the existing CI-selected python/tests/test_dropin_offline.py; no workflow edit is required. Focused invocation: python -m pytest python/tests/test_dropin_offline.py -k test_completion -q.
• Checked the same name and error behavior against frozen Python Connect; the new test fails on the unchanged Rust baseline as expected.
• Rust: 392 client library tests and 95 core library tests pass; full client test compilation passes with default and wasm-udf features; formatting and diff checks pass. Workspace-wide compilation requires an unavailable wasm32-unknown-unknown target for examples, so that initial failure is retained locally.
• Existing CI-selected offline suite: candidate passes all 287 tests; unchanged baseline passes the original 274 tests and fails all 13 added completion tests as expected. The installed-wheel validation sets PYTHONPATH to the installed package followed by the repository Python root, matching the existing CI module layout.
• Unchanged frozen loopback benchmark, five fresh rotating paired samples per client/case across 18 cases: candidate and Python each pass 90/90; baseline passes all 40 guard samples and fails the 50 completion samples. All ten inspection cases are fixed; guards cover schema, columns, named-column access, select, withColumn and missing-column errors. No speedup is claimed for formerly failing cases. Measured completion loops issue one schema RPC per call (20 for the repeated fixture).

No live Spark server or JVM was used. Original setup failures, raw measurements, commands, source/wheel hashes and dependency freezes are retained in campaign artifacts.

Was this patch authored or co-authored using generative AI tooling?

Generated-by: OpenAI Codex (GPT-6)