ci(ffi): cross-platform Python wheels + PyPI publishing - #229
Conversation
Adds the release pipeline for the ant-ffi Python bindings: reproducible per-platform wheel builds and a publish workflow using PyPI Trusted Publishing (OIDC, no stored token). Build scripts (single source of truth — CI runs the same script a developer runs locally): - build-wheel-manylinux.sh compiles the native lib inside a manylinux_2_28 container (glibc 2.28) so the wheel installs on RHEL8 / Ubuntu 20.04+ / Debian 10+; auditwheel verifies + retags. Arg: x86_64 | aarch64. - build-wheel-macos.sh builds arm64 + x86_64, lipo-fuses to a universal2 wheel (deployment target 11.0); delocate verifies self-contained. - build-wheel-windows.ps1 native x86_64 DLL; delvewheel bundles non-system DLLs. Workflow (publish-python.yml): - Matrix: linux x86_64 (ubuntu-latest), linux aarch64 (ubuntu-24.04-arm, native — no QEMU), macOS universal2 (macos-latest), windows amd64. - Triggers: push tag `python-v<ver>` -> build+publish to PyPI; or manual dispatch with publish=none|testpypi|pypi (none = matrix smoke test). - Publishes via OIDC (id-token) into a GitHub environment (pypi/testpypi). Proven locally: linux-x86_64 and macOS-universal2 wheels build, are auditwheel/delocate-honest, and install + run in clean environments (incl. debian:11 / glibc 2.31). linux-aarch64 and Windows validate on first CI dispatch (no local hardware for either). Before the first publish, a maintainer must create the GitHub environments and register the PyPI/TestPyPI pending publisher (project ant-ffi, owner WithAutonomi, repo ant-sdk, workflow publish-python.yml). The workflow is inert until then (only runs on a python-v* tag or manual dispatch). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Follows the distribution-name decision on the base branch; wheel globs are name-agnostic so only the workflow title and the pending-publisher setup comment change. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Each platform script now ends by installing its repaired wheel into a throwaway venv and importing it from outside the source tree (via ffi/scripts/check-python-wheel.sh from the base branch; inline PowerShell on Windows). Catches the wheel that installs cleanly but cannot import. The manylinux script skips the check when cross-building (pip refuses a foreign-arch wheel); both CI legs build natively. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
dirvine
left a comment
There was a problem hiding this comment.
Reviewed at f674f57d77358d016910280a1167277754768de2.
Requesting changes for two release-safety gaps:
-
publish=nonecan publish to production PyPI. In.github/workflows/publish-python.yml:102-104,126-127, a manual dispatch whose selected ref is apython-v*tag satisfies the tag clause regardless ofevent_name. Thusworkflow_dispatch(ref=python-v0.0.8, publish=none)runs the publish job in thepypienvironment and executes the PyPI step, contrary to the documented no-upload behaviour. Atestpypidispatch on such a ref also enables both publish steps. Please restrict tag clauses to tag push events, e.g. requiregithub.event_name == 'push'alongsidestartsWith(...). -
No release-version invariant is enforced. The tag suffix,
ffi/python/pyproject.tomlversion, and nativeant_ffi_version()/Cargo package version can disagree while every current check passes. I reproduced this in a scratch copy: a wheel with metadata9.9.9containing native FFI0.0.8built, installed and imported successfully. Please fail before upload unless the wheel metadata and native version agree, and on tag pushes require both to equal thepython-v*suffix.
Verified here: current versions are all 0.0.8; current FFI CI is green; actionlint and git diff --check pass; the macOS universal2 wheel built successfully, contains both x86_64 and arm64, installs in a fresh venv, and reports metadata/native 0.0.8.
Nonblocking follow-ups: the new four-platform matrix has no pre-merge run (Windows and Linux ARM remain unexecuted); clean wheelhouse/ and foreign native libraries before builds (I reproduced a stale .dll being packaged); use Cargo --locked and consider pinning release tooling/actions.
Review panel: five completed independent seats considered the PR merge-safe with caveats; one seat timed out. The panel agreed on the underlying version/matrix/staleness gaps but differed on severity. I am blocking on the two cases above because they can cause an unintended or permanently mis-versioned production PyPI release.
…riant Review findings on this PR: - A workflow_dispatch whose selected ref is a python-v* tag satisfied the tag clauses regardless of the publish input, so publish=none (or testpypi) on a tag ref could upload to production PyPI. Tag clauses now also require the push event. - Nothing tied the wheel metadata version, the native ant_ffi_version(), and the release tag together. The install/import check now fails any wheel whose metadata and native versions disagree, and on tag pushes CI exports EXPECTED_VERSION from the python-v* suffix and both must match it. Verified locally: matching stub passes; metadata!=native and tag!=version stubs both fail. Nonblocking hardening from the same review: build scripts clean stale wheelhouse/ and foreign native libs before staging (a leftover .dll/.so from another platform's build would be packaged by the package-data globs), and cargo builds run --locked. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
dirvine
left a comment
There was a problem hiding this comment.
Re-reviewed at c8b5821a2c0084fdec636b760930d3c68596e56b. The two requested changes are resolved:
- Production publishing now requires a
pushevent for thepython-v*tag path, so manualpublish=noneremains no-upload andtestpypicannot also invoke PyPI, including when dispatching from a tag ref. - Each installed wheel now asserts metadata version equals
ant_ffi_version(), and tag pushes additionally assert both equal thepython-v*suffix.
I reran the adversarial event/input matrix and mismatch tests: both metadata/native and tag/version mismatches fail as intended. The current macOS universal2 wheel built, installed and imported successfully with matching 0.0.8 versions and both x86_64/arm64 slices. Current-head FFI CI and actionlint are green. The added stale-native cleanup and Cargo --locked hardening are sound.
No remaining blockers found.
What
The release pipeline for the Python bindings added in #228: reproducible per-platform wheel builds + publishing via PyPI Trusted Publishing (OIDC, no stored token).
Build scripts
CI runs the same script a developer runs locally — one source of truth:
build-wheel-manylinux.sh— compiles the native lib inside amanylinux_2_28container (glibc 2.28) so the wheel installs on RHEL8 / Ubuntu 20.04+ / Debian 10+;auditwheelverifies + retags. Arg:x86_64|aarch64.build-wheel-macos.sh— builds arm64 + x86_64,lipo-fuses to a universal2 wheel (deployment target 11.0);delocateverifies self-contained.build-wheel-windows.ps1— native x86_64 DLL;delvewheelbundles non-system DLLs.Workflow (
publish-python.yml)ubuntu-latest), linux aarch64 (ubuntu-24.04-arm, native — no QEMU), macOS universal2 (macos-latest), windows amd64.python-v<ver>→ build + publish to PyPI; or manual dispatch withpublish = none | testpypi | pypi(none= matrix smoke test, no upload).pypi/testpypi).Proven
debian:11/ glibc 2.31).Before first publish (maintainer action)
pypiandtestpypi.ant-sdk, ownerWithAutonomi, repoant-sdk, workflowpublish-python.yml, environmentpypi/testpypi.The workflow is inert until then (only runs on a
python-v*tag or manual dispatch). Suggested first run: dispatch withpublish=testpypito exercise the full matrix end-to-end.🤖 Generated with Claude Code