Skip to content

feat(python): expose consumer shutdown and offset drain timeout - #4660

Draft
ParthibanRajasekaran wants to merge 7 commits into
apache:masterfrom
ParthibanRajasekaran:fix/python-consumer-shutdown
Draft

ParthibanRajasekaran wants to merge 7 commits into
apache:masterfrom
ParthibanRajasekaran:fix/python-consumer-shutdown

Conversation

@ParthibanRajasekaran

@ParthibanRajasekaran ParthibanRajasekaran commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

Which issue does this PR address?

Closes #4165.

Rationale

Python consumers can stop their consumption loop but cannot invoke the Rust consumer's final offset cleanup and group leave.

What changed?

await consumer.shutdown() delegates to the existing Rust consumer shutdown after consumption has stopped. An appended optional offset_drain_timeout: datetime.timedelta | None uses the existing duration conversion and preserves the Rust default when omitted or None. Generated types and documentation describe the supported call order and timeout semantics. Regression tests verify final stored offsets, group departure before an independently connected replacement resumes, concurrent/repeated shutdown, iterator termination and timeout inputs.

The change is limited to five Python SDK files, in separate implementation, test and documentation commits.

Local Execution

  • Passed full Python SDK suite: 983 tests.
  • Passed Rust SDK library: 504 tests.
  • Passed three-node CLI suite: 177 tests, 1 ignored.
  • Passed connectors suite: 233 tests, 1 ignored.
  • Passed workspace doctests: 48 tests, 14 ignored.
  • Passed workspace/server and Python extension builds, workspace/binding Clippy, formatting, Python lint/type checks and repository checks.
  • Pre-commit and pre-push hooks ran and passed.
  • Post-rebase affected Rust tests: 632 passed.
  • Broad macOS workspace run: 9,936 passed, 1 failed, 15 skipped. The HTTP relay-upload cluster test timed out waiting for hop-target saturation. An exact isolated macOS CI-profile rerun subsequently passed on the first attempt in 13.575s, and the case also passed locally on Linux. The broad run is not reported as passed.
  • Three Linux /proc-dependent integration cases and the reserved-port socket unit case passed locally on Linux.

Draft for tracking: separate Python BDD, README examples and the encryption-disabled Python CI leg still need confirmed local results. Author review of the rationale and explanation of the code is pending. This draft does not request maintainer review yet; the contributor's existing PR #4581 is still open.

AI Usage

  • Repository/API analysis and independent adversarial review against the implementation and tests.
  • The local builds, real-server tests and repository checks

@github-actions

Copy link
Copy Markdown

Note for maintainers: @ParthibanRajasekaran also has #4581 open. New contributors are asked to keep one open PR at a time. See CONTRIBUTING.md.

@codecov

codecov Bot commented Oct 27, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 80.86%. Comparing base (d47ff5d) to head (232b851).

Additional details and impacted files
@@             Coverage Diff              @@
##             master    #4660      +/-   ##
============================================
+ Coverage     80.83%   80.86%   +0.03%     
  Complexity     3107     3107              
============================================
  Files          1467     1467              
  Lines        356533   356549      +16     
  Branches     300421   300421              
============================================
+ Hits         288194   288330     +136     
+ Misses        61617    61375     -242     
- Partials       6722     6844     +122     
Components Coverage Δ
Rust Core 80.16% <ø> (+0.04%) ⬆️
Java SDK 78.15% <ø> (ø)
C# SDK 83.60% <ø> (ø)
Python SDK 85.74% <100.00%> (+0.06%) ⬆️
PHP SDK 90.67% <ø> (ø)
Node SDK 96.44% <ø> (ø)
Go SDK 69.13% <ø> (ø)
C++ SDK 85.51% <ø> (ø)
Files with missing lines Coverage Δ
foreign/python/src/client.rs 99.10% <100.00%> (+<0.01%) ⬆️
foreign/python/src/consumer.rs 79.55% <100.00%> (+1.25%) ⬆️

... and 530 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Python SDK: expose consumer shutdown and offset drain timeout

1 participant