[3.15] gh-151613: Fix remote debugging frame cache ABA - #152448
Merged
Conversation
The remote debugging frame cache previously used only the last_profiled_frame address as its cache anchor. If a frame returned and a later frame reused the same _PyInterpreterFrame address, the profiler could accept a stale cache entry and splice parent frames from a different call chain into the current stack. This adds a last_profiled_frame_seq counter next to last_profiled_frame, increments it when the anchor advances, stores it in frame cache entries, and validates cache hits against both the frame address and the sequence. Cache miss walks now copy stack chunks before storing new cache entries so stored continuations come from a stable snapshot. The new regression test exercises alternating call chains and checks that cached stacks never contain frames from both branches. (cherry picked from commit 8cda6ae)
pablogsal
marked this pull request as ready for review
June 27, 2026 17:15
pablogsal
requested review from
AA-Turner,
FFY00,
ZeroIntensity and
ericsnowcurrently
as code owners
June 27, 2026 17:15
pablogsal
enabled auto-merge (squash)
June 27, 2026 17:15
Contributor
|
I'm sorry but is it intentional that this change breaks extension ABI? If yes, could we please have a warning about such things in release notes? It's not nice when we keep being asked to test things early, then learn the hard way that suddenly everything using |
Member
Author
|
The ABI is only frozen after the first release candidate as per our policy. We don't keep track of ABI breakage changes before we freeze ABI but we can discuss doing it, but as things stand today the only guarantees start after RC1 |
Contributor
|
Sure, just want to make sure that the change is intentional before we tell hundreds of people who already started testing that they have to rebuild everything, and hope nothing goes out of order so things don't start segfaulting randomly. |
Member
Author
The change is intentional in the sense that was required for the fix. As you can imagine we don't want to break people unnecessarily even if we are still allowed to do so. |
Contributor
|
I think tracking extension module ABI breakages in beta versions (and rc1) would be helpful. I'll start a discussion on dpo after I recover from EuroPython. |
Member
|
To avoid ABI breakage in the future, we should move PyInterpreterState structure to the internal C API! But that's a lot of work :-( Currently, multiple 3rd party C extensions access it directly, instead of calling getter or setter functions. |
fametrano
added a commit
to btclib-org/btclib_libsecp256k1
that referenced
this pull request
Jul 29, 2026
This reverts commit 885a7ca6. The library is not what failed: on 3.15.0b4 the whole suite passes, 77 tests, once cffi is built from source. What failed is cffi's own cp315 wheel, compiled against 3.15b3, whose PyThreadState grew a field in b4 (python/cpython#152448). The offset it was compiled with is now wrong, so the process segfaults in take_gil the moment C re-enters Python through a libffi closure — here the illegal_callback that secp256k1_ec_pubkey_parse invokes on invalid input. Upstream is python-cffi/cffi#263, whose fix is a new cffi release. Keeping the jobs would mean thirteen red ones in every run until 3.15 reaches release candidate, for a cause outside this repository, and a permanently red matrix is one nobody reads. Building cffi from source in CI would hide the red, at the price of a workaround nobody would remember to remove. So 3.15 comes back when there is something to learn from it: cffi wheels that match the interpreter, or the release candidate, which is also when cibuildwheel starts producing a cp315 wheel without a flag and test-static can take 3.15 in the same move. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
giacomocaironi
added a commit
to btclib-org/btclib_libsecp256k1
that referenced
this pull request
Aug 1, 2026
…packaging (#21) * Prepare release 0.7.1 - Wrap libsecp256k1 0.7.1 (1a53f49) - Support Python 3.9-3.14 (drop EOL 3.7/3.8, add 3.13/3.14) - Align release numbers with the wrapped libsecp256k1 version - Modernize CI: current runners and actions, cibuildwheel 4.x, test matrix 3.9-3.14 + PyPy - Update all pre-commit hooks to their latest versions - Build the callback stubs as a separate compilation unit instead of mutating the vendored sources, fixing sdist rebuilds with multi-pass PEP 517 frontends such as uv (#20) - Fail fast on build subprocess errors - Use secp256k1_schnorrsig_sign32 instead of the deprecated alias Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Clean up sdist contents and README badges - Exclude vendored-tree build artifacts from the sdist: stale autotools/make output from a local build (Makefile, config.status, .libs, ...) would break or pollute builds on the installing machine; sdist size drops from 5.7MB to 2.1MB - Point the CI badge to the main.yml workflow (build.yml/test.yml no longer exist) and drop the pyright badge (tool not used) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Add macOS arm64 and Linux aarch64 wheels Extend the cibuildwheel build matrix and the static-wheel test matrix with the arm64 GitHub runners (macos-latest, ubuntu-24.04-arm). Also document in HISTORY.md why Python 3.7/3.8 support was dropped. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Derive the dynamic wheel platform tag from the build machine The tag was hardcoded to x86_64 (with a fake macOS 10.16 minimum version): a dynamic wheel built on an arm64 machine was tagged as x86_64, hence wrong and not even installable where it was built. Only the mingw cross-compilation keeps a fixed win_amd64 tag, as the target machine cannot be inspected. Merge the duplicated linux/macos dynamic-wheel jobs into one matrix job covering arm64 too (ubuntu-24.04-arm, macos-latest), move the sdist to its own job, and extend the dynamic-wheel test matrix to the arm64 runners. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Use Python 3.14 as build host, auto-cancel superseded CI runs The host version does not affect the produced wheels; 3.14 is the current stable and the one all local verification runs on. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Add native Windows wheels (CMake/MSVC via cibuildwheel) On native Windows the vendored library is now built with CMake/MSVC (BUILD_SHARED_LIBS=OFF, external default callbacks ON) and the cffi extension with the standard setuptools/MSVC toolchain, defining SECP256K1_STATIC and compiling the callback stubs into the extension itself. The autotools/mingw path is unchanged and still used for POSIX builds and for the cross-compiled dynamic Windows wheel. windows-latest joins the cibuildwheel and static-test matrices (CPython only: PyPy wheels are skipped on Windows for now). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Validate inputs, check return codes, modernize the context - All libsecp256k1 return codes are now checked, raising ValueError with a clear message: malformed keys and signatures were previously parsed into uninitialized structs and verified against garbage - Fixed-size buffers the C side reads blindly (msg32, sig64, seckey) are length-checked before the call - The three per-module contexts are replaced by a single shared one (context.py), created with the modern SECP256K1_CONTEXT_NONE flag instead of the deprecated SIGN|VERIFY (769) and randomized against side-channel leakage, as recommended by libsecp256k1 - New negative tests cover the error paths Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Add Dependabot for GitHub Actions and the secp256k1 submodule Weekly grouped PRs for action bumps, monthly for submodule movement (upstream default branch: release bumps stay manual, aligned with the tagged version). pre-commit hooks are already covered by pre-commit.ci. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Test against official BIP340, RFC6979/DER, and btclib's ECDSA vectors - BIP340: the 19 vectors from bitcoin/bips test-vectors.csv (vendored); signing is exercised where the bindings allow it (32-byte messages), verification on every vector including the invalid-key/signature ones - RFC6979 deterministic ECDSA on secp256k1: (k, r, s) vectors published on bitcointalk, as vendored by trezor-firmware and bitcoinjs-lib; each vector is additionally self-checked asserting r == x(k*G) - the ECDSA vectors used by btclib (vendored there from rustyrussell/secp256k1-py): 199 deterministic signatures reproduced byte-for-byte, plus 199 custom-nonce signatures verified and anchored via r == x(k*G); btclib's rfc6979.json is not imported as it only covers NIST curves, not secp256k1 - DER: canonical encodings from trezor-firmware must parse (and fail verification); trailing garbage and non-minimal padding must be rejected; the leniencies of secp256k1_ecdsa_signature_parse_der (unsigned reading of high-bit integers, zeroing of out-of-range values) are pinned as regression expectations Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Harden CI: least-privilege token, SHA-pinned actions - permissions: contents: read at workflow level: anything running in a job (actions, pip dependencies, build scripts) can read the GITHUB_TOKEN, so it must not be able to write to the repository; artifact upload/download uses the runtime token and is unaffected - all actions pinned to full commit SHAs (with the version as a comment): version tags are mutable and a compromised action repo can repoint them, as in the tj-actions incident of March 2025; Dependabot understands SHA pins and keeps them updated Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Run macos-13 jobs only on master and on PRs targeting master The Intel macOS runners are scarce and queue for a long time: today a dev run sat 40+ minutes with only the two macos-13 build jobs pending. Development pushes now skip them; the full Intel coverage still gates every PR to master and every push to master. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Add tag-triggered PyPI release workflow with Trusted Publishing - release.yml runs on v* tags: it reuses the whole main.yml build/test pipeline via workflow_call (single source of truth for the build jobs), then publishes every wheel and the sdist to PyPI - publishing uses Trusted Publishing (OIDC): no long-lived PyPI token exists anywhere; the job only adds id-token: write, and PEP 740 attestations are generated by default - the publish job runs in the pypi environment, so required reviewers can gate every release behind a manual approval - main.yml push trigger restricted to branches (tags belong to the release workflow) and the macos-13 jobs enabled on tag refs, so releases ship Intel macOS wheels - release process documented in README.md, including the one-time PyPI/GitHub setup Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Wrap the ecdh, recovery, ellswift, and musig modules - every optional libsecp256k1 module is now compiled in and declared to cffi: ecdh, musig and ellswift were already compiled in (they are on by default upstream) but never declared, so their symbols were unreachable; recovery was the only one actually disabled - all the wrapped modules are requested explicitly at configure time on both build paths: upstream defaults are not part of its API, and the autotools and CMake paths were enabling different module sets - new ecdh, recovery, and ellswift binding modules; the header concatenation order now matters, as the #include directives are stripped before preprocessing and musig needs the extrakeys types - MuSig2 has no binding module: its two-round protocol needs an interface that makes secret nonce reuse hard, still to be designed; it is reachable through the raw cffi bindings, and tests/test_modules runs a complete 2-of-2 session as usage example - the new tests cross-check the bindings against each other instead of against vendored constants: the ECDH secret is recomputed from the shared point, the recoverable signature is compared with the deterministic ECDSA one, and the MuSig2 aggregate signature is verified as a plain BIP340 signature by ssa Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Ignore .dylib build artifacts An in-place dynamic build copies the shared libsecp256k1 next to the bindings; *.so and *.dll were already ignored, so the artifact only showed up as untracked on macOS. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Wrap key algebra, taproot tweaking, and ECDSA signature forms - new keys module: private and public key algebra (verify, negate, tweak add and multiply, combine) plus public key parsing and serialization; pubkey_tweak_mul is the multiplication of an arbitrary point, which mult, being limited to the generator, cannot express - new xonly module: the BIP341 taproot primitives, i.e. the tweaking of an x-only public key and of its private key, so that a key path spending can be signed with ssa; tweak_add_check verifies a taproot commitment without recomputing the tweak - dsa now exposes the malleability primitives (normalize, is_low_s) and the DER to compact conversions: signatures coming from a system that does not enforce lower-s must be normalized before verification, as libsecp256k1 rejects them - dsa.verify and ssa.verify return bool instead of int; being a subtype of int, with True == 1, callers cannot tell the difference - dsa.sign and dsa.verify now share the DER parse and serialize helpers with the new conversions, instead of inlining them - the tests cross-check the scalar and point operations against mult modulo the group order, the x-only tweaking against the plain public key one (a distinct libsecp256k1 code path), and run a full taproot key path: tweak the key, sign with the tweaked private key, verify against the tweaked x-only public key; the malleated signature of the low-s test is built through the new compact conversions Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Gate CI on lint and on installing from the sdist Two holes in the pipeline: the pre-commit hooks (formatting, linting, typing) ran only on the developer's machine, so nothing stopped a non-conforming commit from landing; and the sdist was built and uploaded as an artifact without ever being installed. - new lint job: pre-commit run --all-files on a single runner, with the hook environments cached on the hash of .pre-commit-config.yaml; it is the cheap gate that must fail before the expensive build matrix - new test-sdist job: installs the built sdist and runs the test suite on Linux and macOS. Installing from sdist is the only path that has to compile the vendored libsecp256k1 on the target machine, i.e. the path the sdist exclude list in pyproject.toml exists to support; until now a broken sdist would have been discovered on PyPI Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Replace eight linters and formatters with ruff black, isort, flake8 (plus the Flake8-pyproject shim to read its config from pyproject.toml), autoflake, pyupgrade, bandit, pydocstringformatter and yesqa all did one slice of the job, each with its own configuration section and its own exclusion of the vendored secp256k1 tree. ruff does all of it: two hooks instead of nine, one config section, seconds instead of a minute. - the rule families selected in pyproject.toml stand in one to one for the tools they replace (F/E/W for flake8, I for isort, UP for pyupgrade, S for bandit, D for pydocstringformatter, C90 for the McCabe complexity limit), plus C4, SIM, PTH, RET, PL and RUF; the pylint section that no hook ever ran is gone - D202 and PLR2004 are ignored project wide: the blank line after a docstring is the style of this codebase, and comparing against a key, signature or hash length is the domain, not a magic value - the ~150 "# nosec B101" markers and the comment blocks quoting bandit rule ids and URLs are dead with bandit gone: ruff reads suppressions from the per-file-ignores table, where the justification now lives - the copyright-notice hook matched "python" as a filename pattern, so it silently checked no file at all; with the pattern fixed to \.py$ it found noxfile.py and scripts/hatch_build.py missing the notice - three module docstrings ("... according." / "to BIP324: ...") had a summary line split mid sentence: rewritten so the first line stands on its own, as D212 requires - assorted fixes ruff found and the previous chain did not: os.remove and glob.glob replaced by pathlib equivalents, redundant str() inside f-strings, superfluous else after return, if/else blocks that are plain ternaries, an unused glob import - the five README badges advertising isort, black, flake8, pylint and bandit are replaced by the ruff one Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Gate coverage on a ratchet, harden the pytest configuration Coverage was measured by hand, with the source and the report options spelled out on the command line, and never in CI: nothing recorded what the suite reached, so nothing noticed it regressing. - [tool.coverage] in pyproject.toml: branch mode (a wrapper is made of guard clauses, and statement coverage calls a half-tested if fully covered), source, and fail_under = 84, the level the suite reaches today. It is a ratchet, not a target: the remaining gap is error paths that need libsecp256k1 itself to fail, plus the dlopen fallback in __init__.py, unreachable in a static build and the only covered branch in a dynamic one, which is why the static build is the one measured - new coverage job in CI, on a single runner: coverage is a property of the python layer, not of the platform - pytest now runs with --strict-markers and --strict-config (a typo in a marker or in a config key is an error, not a silent no-op), xfail_strict (an xfail that passes is a stale test) and filterwarnings = error: across a 3.9 to 3.14 spread a deprecation warning from cffi or CPython is the cheapest possible early warning - testpaths replaces the python_files / python_functions widening, so tests/test.py, the only file needing it, is renamed test_core.py after what it covers: round-trips, input validation and safe aborts Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Type-check the cffi boundary, turn on mypy strict The package ships py.typed, so its annotations are what btclib type-checks against downstream, but mypy ran with strict = false and, worse, "import _btclib_libsecp256k1 # type: ignore" made ffi and lib plain Any: every wrapper in the package is a call through those two objects, so the whole codebase type-checked vacuously. - new stubs/_btclib_libsecp256k1.pyi, on mypy_path: the extension is generated at install time and has no source to inspect. It declares only the surface the package uses (new, unpack, NULL, dlopen) and keeps lib as Any on purpose, its members being the ~100 C entry points generated at build time. Typing unpack as returning bytes is what removed 20 "Returning Any from function declared to return bytes" errors, i.e. 20 functions whose return type nothing checked - strict = true, and the annotations it demanded: the opaque cffi handles are now spelled CData (an alias exported by the package, so that public signatures like keys.parse say what they return instead of Any), and FFIExtension declares the five attributes its subclass is required to set, which mypy was right to flag as undefined - the mypy mirror injects --ignore-missing-imports, which silently turns any unresolved or misspelled import into Any: dropped, so that the optional comparands of the benchmark (btclib, coincurve, secp256k1) have to be declared as overrides in pyproject.toml. secp256k1 needs an explicit skip either way, or the vendored C tree is picked up as a namespace package of the same name - the extension import path is restructured into _load_lib(): the dlopen fallback assigned lib inside a nested loop, so a failure to load left the name unbound and the import failed with a bare NameError instead of a diagnosable error. It now raises ImportError naming the directory searched, and no longer keeps looking after a successful load - the stub had to be exempted from the .gitignore rule that hides the build artifacts of the extension Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Manage the development environment with uv Three tools overlapped: the hatch environments in pyproject.toml, the noxfile, and pre-commit, each installing its own copy of the same linters. uv replaces the first two; pre-commit stays, as the gate that pins the authoritative revisions of the hooks. - development dependencies become PEP 735 groups (test, lint, build), which are standard and not tied to one project manager: pip reads them too, with --group. The hatch environments and their scripts are gone, and so is the noxfile - hatchling remains the build backend: uv is the installer and resolver, it has nothing to do with how the extension is compiled - uv.lock is committed, so that "uv sync" gives the same development environment to everybody - .python-version pins the interpreter the environment is built on (the newest supported one) and is now tracked instead of ignored: uv installs that interpreter if missing, which is what makes pyenv unnecessary - CI: the lint job runs pre-commit through uvx, without installing the project, since linting does not need libsecp256k1 compiled; the coverage job uses uv sync and inherits the interpreter from .python-version, exactly as a developer does - README documents the uv workflow, including the one sharp edge: uv keys its build cache on the sources, which do not tell it that the compiled extension is ABI specific, so testing against another interpreter needs --no-cache Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Modernize the packaging metadata, validate the distributions - PEP 639 license metadata: license = "MIT" as an SPDX expression plus license-files, replacing the deprecated license table; the License :: OSI Approved classifier is now redundant and removed. The built distributions carry Metadata-Version 2.4 with License-Expression and both LICENSE and COPYRIGHT - the package exposes __version__, read from the installed distribution metadata so that pyproject.toml stays the single place to bump at release time, as the documented release process assumes - new check-dist job: twine check --strict and check-wheel-contents over every wheel and the sdist. It is the same validation PyPI applies on upload, run before the release workflow reaches the point of no return, since broken metadata or a README that does not render cannot be fixed by reuploading, only yanked. check-wheel-contents needs W003 and W009 ignored: a dynamic wheel is meant to ship the shared libsecp256k1 as a non-module at the top level - the changelog project URL pointed at github.com/btclib-org/btclib_libsecp256k1/HISTORY.md, a 404: it is missing the /blob/master/ path, and PyPI shows that link - HISTORY.md records this whole round of infrastructure work Not touched, as it is a decision about the published support matrix rather than hygiene: requires-python is still >= 3.9, though 3.9 has been end of life since October 2025. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Normalize scalar arguments through a single helper The same five lines, serializing an int private key big endian and checking that the result is 32 bytes, existed in nine places: inlined in dsa, ecdh, ellswift (twice), recovery, ssa and mult, and as two byte-identical private _scalar helpers in keys and xonly. The new private _scalar module holds the one copy. Behaviour is unchanged, error messages included: every call site passes the same argument name the inlined message used ("private key", "tweak", "scalar"), so the ValueError texts the tests match on are the same. Coverage goes from 84.4% to 85.5% for free, and the fail_under ratchet with it: eight partly covered length guards became one that the suite exercises on both the int and the bytes path. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Update the outdated Windows disclaimer in README Building and testing on Windows is supported since the native CMake/MSVC path: describe the per-platform toolchains instead. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Document why static PyPy wheels are not built on Windows Not a temporary limitation: PyPy is best served by the ABI-mode dynamic wheel (the cffi mode recommended on PyPy), which pip falls back to; a static pp3 wheel would take precedence over it and perform worse. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Report invalid arguments consistently, align the lint environment dsa.sign did not check the length of its optional nonce contribution, as ssa.sign and recovery.sign do: an ndata longer than 32 bytes left the left padding empty, libsecp256k1 read 32 bytes through the bare pointer, and the excess was dropped in silence. An int scalar outside the 32-byte range surfaced the OverflowError of int.to_bytes, where every other out-of-domain argument raises ValueError; whether the value is a valid scalar, i.e. in [1, n-1], remains for libsecp256k1 to say. The lint dependency group did not carry the stubs the mypy pre-commit hook declares, so mypy run from the development environment reported three errors the gate does not see. The two have to agree, or neither is worth running. The build writes src/btclib_default_callbacks.c into the submodule tree, as automake resolves the sources it is added to relative to it: the untracked file left the submodule permanently reported as modified, which only hides a real change. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Cover the reachable error paths, one lint gate instead of two The coverage ratchet stood at 85% because the validation errors of recovery, ssa, xonly and dsa.verify were never exercised: the gap was not made of unreachable code, it was untested code. Now at 89.9%, and what remains uncovered is only the RuntimeError paths, which need libsecp256k1 itself to fail, plus the dlopen fallback of __init__.py. - recovery: a short message hash in recover, a short signature in to_der, and a signature whose r is zero, which parses but recovers no point - ssa: a short message hash and an over-long aux_rand32 in sign - xonly: an unparsable public key in from_pubkey, and the two tweaks that land on the point at infinity, i.e. by the negation of the private key of the even y point - dsa: a short message hash in verify, and the nonce contribution of sign, which no test passed at all: it now checks that a custom nonce changes the deterministic signature, that the result still verifies, and that a shorter one is left padded. A one-line ternary had hidden this, being covered by the ndata=None path alone - the int out of range guard of scalar(), on both sides of the range - xonly.tweak_add is also exercised with a full public key, which was the only untested branch of _parse: the public key of 11 has odd y, so the result matches the x-only one only because the key is lifted to its even y point first pre-commit.ci and the lint job of the main workflow were both running the hooks, with two badges reporting it. The gate is the lint job, in the same run as the build and test matrices; pre-commit.ci is declared as what it alone provides, the weekly pull request bumping the pinned hook revisions, with autofix disabled (a bot committing to a branch is at odds with a workflow token that cannot write and actions pinned to SHAs) and targeting dev, as Dependabot does. Its badge is dropped. The documentation URL pointed at btclib.readthedocs.io, which documents btclib and not these bindings; it now points at the README. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Cover the last two reachable ValueError paths keys.prvkey_tweak_mul with a zero tweak, whose product is zero and so no private key, and keys.pubkey_tweak_add by the negation of the private key, which lands on the point at infinity. Coverage goes from 89.9% to 90.6%, and the ratchet with it. Every ValueError the bindings raise is now exercised: what is left uncovered is the RuntimeError paths, unreachable through the API since they need libsecp256k1 itself to fail, and the dlopen fallback of __init__.py, which is unreachable code in a static build and the only covered branch in a dynamic one. Raising the ratchet further means testing the untestable. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Add a security policy A cryptographic package without one leaves whoever finds a flaw with no channel but a public issue. There are two: GitHub private vulnerability reporting, now enabled on the repository, and the address btclib already publishes in its own policy, so that the organization has one mailbox rather than one per project. What the policy has to say beyond that is where the boundary of this project lies. The cryptography is libsecp256k1, which has a policy and an address of its own; ours is the layer driving it: the validation of the arguments crossing the cffi boundary, the default callbacks replaced with do-nothing stubs (so that nothing below the wrapper modules catches an illegal argument, and code reaching the raw lib bindings is on its own), which optional modules are compiled in, the commit the submodule is pinned to, and the distributions published to PyPI. It also states what is not a vulnerability, to spare both sides the exchange: secret material lives in immutable Python objects that are never zeroized, and the constant-time properties of libsecp256k1 stop at the boundary. Only the latest release is supported: version numbers track the wrapped libsecp256k1, so a fix is published as a new release and nothing is backported. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Measure only reachable code, and reach all of it The ratchet was gating on a number the suite could not raise. Of the 28 uncovered points, 18 were the RuntimeError paths, which report a libsecp256k1 call failing where no argument can make it fail: excluded from the measure, so that what is left is code a test can reach. The distinction is the one the bindings already draw, ValueError for an argument out of the domain and RuntimeError for the impossible, and counting the two together meant a new untested guard clause could hide inside the slack. The rest was the dlopen fallback of _load_lib, unreachable in a static build because the branch taken is the other one. It now takes the extension as an argument, so a stand-in module pointing at an empty directory drives the search: what is asserted is that a directory holding no library is reported as such, and that a file the loader rejects is skipped rather than ending the search, which is what a wheel repaired by auditwheel or delocate needs. Coverage is now 100% of lines and branches, and the ratchet is set there: the only value that fails on a single new untested one. It assumes the static extension the coverage job builds; on a dynamic one the linked-in branch is the unexecuted one. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Build and test Windows arm64 wheels Windows was covered on x86_64 only: cibuildwheel ran on windows-latest and the mingw cross-compiled dynamic wheel is x86_64 by construction, so Windows on ARM had no wheel at all and fell back to the sdist, i.e. to compiling libsecp256k1 with CMake/MSVC on the user's machine. - windows-11-arm joins the build-cibuildwheel and test-static matrices. No source change is needed: CMake and MSVC target the host, so the win_arm64 wheel comes out of the same native path as win_amd64, and the artifacts reach check-dist on their own, being collected by pattern - cp39 and cp310 are skipped for win_arm64, and excluded from the test matrix: CPython has no Windows arm64 build before 3.11. cibuildwheel does list those identifiers, having cross-compiled them from an x86_64 host in the past, so without the skip it would look for an interpreter that cannot exist on a native arm64 runner - the dynamic wheel stays x86_64: mingw cross-compilation targets it alone, as dynamic_platform_tag already documented Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Test and document concurrent use of the shared context A static wheel is built for the free-threaded interpreter, so these bindings are already shipped where nothing serializes the calls into them, and nothing said whether that was safe. It is: the one context they share is mutated only by secp256k1_context_randomize, which runs at import time before any thread exists, and every call allocates the buffers it writes to. That reasoning is worth no more than what checks it, hence a test driving eight threads through ECDSA and BIP340 signing and verification, generator multiplication, ECDH, x-only tweaking and public key combination. Each is deterministic, ECDSA by RFC6979 and BIP340 by a fixed aux_rand32, so a result differing between threads is a shared buffer rather than a legitimate difference. Verified on CPython 3.14.6 free-threaded, not only on the default build. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Wrap key ordering, tagged hashing, variable length signing Three entry points libsecp256k1 exposes and nothing here reached. keys.pubkey_cmp and keys.pubkey_sort are the lexicographic order of the compressed serialization: the order of a BIP67 multisig script, and the one MuSig2 key aggregation applies when the participants have agreed on no other. Sorting reorders an array of borrowed pointers, so the list they point into is what has to outlive the call. hashes.tagged_sha256 is the BIP340 tagged hash, SHA256(SHA256(tag) || SHA256(tag) || msg), which the taproot tags of BIP341 and the BIP340 challenge are built on: computing it here rather than in python is both faster and the same implementation that signs with it. Cross-checked against hashlib, an implementation of SHA256 sharing nothing with the one inside libsecp256k1. ssa.sign_custom signs a message of any length. BIP340 is not restricted to 32 bytes and ssa.verify already accepted any length, so a signature this package could check was one it could not produce. sign stays the 32-byte entry point, on secp256k1_schnorrsig_sign32, because that is what bitcoin signs and what the BIP340 vectors pin down; its docstring points at tagged_sha256 for the domain separation a raw message of another length gives up. For a 32-byte message the two agree, which the tests assert rather than take from the header. The extraparams magic joins the serialization flags among the constants spelled out here, macros not surviving the preprocessing of the headers into cffi definitions. Its ndata has to stay referenced across the call: cffi keeps alive what a variable points to, not what a struct field does. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Warn that testing on another interpreter replaces the environment uv run --python is documented here as the way to test against another supported interpreter, and what it does beyond running the tests is not obvious: it rebuilds .venv on that interpreter and leaves it there, so the next uv run is on the interpreter of the last such command rather than on the pinned one. Asking for a free-threaded build compounds it, being installed as a managed interpreter that uv then prefers to a system one of the same version. Both cost a while to work out from a failing import of the extension. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Declare what the benchmark script is benchmarked against scripts/benchmark.py imports btclib, coincurve and secp256k1, and none of them was declared anywhere: the script could not be run without working out its comparands from its import lines and installing them by hand. They are a PEP 735 group of their own, outside dev, because btclib depends on this package and cannot become a dependency of developing it. Inside this project that dependency is harmless, and the lock says why: btclib's own requirement on btclib_libsecp256k1 resolves to the editable build of this tree, not to a second copy from PyPI, so what the script times is the code in front of it. Installing the group is still a choice rather than part of uv sync: coincurve and secp256k1 each build a libsecp256k1 of their own, which needs pkg-config on top of the toolchain the README already lists. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Assert that __version__ is exposed, and with a value Nothing referred to it. What that leaves untested is narrow, and the test says so rather than claiming more: reading the version from the distribution metadata happens at import time, so a distribution renamed out from under __init__.py would fail every test in the suite, three times over in CI. It is the attribute itself, part of the public surface btclib reads, that nothing pinned down. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Record why the ECDH hash function is not exposed libsecp256k1 lets the caller replace the SHA256 of the compressed shared point with a hash function of their own, and this is the decision not to wrap that: it is taken as a C callback, so exposing it would mean calling back into python from the middle of the computation, with the shared point passing through python objects on the way. Nothing is lost by leaving it out, because the point itself is already reachable: it is keys.pubkey_tweak_mul(pubkey, prvkey), the multiplication of an arbitrary point, constant time like the ECDH call. A protocol wanting another derivation applies it to that, and SHA256 of it is what shared_secret returns. Written where someone looking for the option will read it, the docstring and the README, and asserted in the ecdh test, so that the sentence cannot quietly stop being true. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Build the vendored library with CMake on every platform - one build path instead of three: autotools on POSIX, CMake/MSVC on native Windows, and autotools with --host for the mingw cross build were three ways to diverge, as the module sets enabled by each of them already had. The cross build now uses the vendored CMake toolchain file, which upstream tests - cmake becomes a build requirement rather than a system one: a PEP 517 frontend provisions it, so installing the sdist no longer demands automake, libtool, pkg-config and autoconf, only a C toolchain. There are cmake wheels for every platform of the build matrix, musllinux and win_arm64 included, and a system cmake >= 3.22 still works with --no-build-isolation - the build writes nothing inside the vendored tree: the callback stubs and the CMake binary directory live under build/, and the stubs are added to the library target through CMAKE_PROJECT_INCLUDE and cmake_language(DEFER), the hook that can reach a target the included file is too early to see. The Makefile.am patching, the generated source inside secp256k1/src, and the git reset --hard and git clean -fxd that undid them are therefore gone: that cleanup used to discard, without warning, any local change to the submodule, and a wheel built on Windows could ship its whole CMake tree in an sdist - the stubs are in the library on every platform now, so the MSVC extension no longer compiles a copy of its own - fixed the shared object lookup of a dynamic build giving up on the first candidate directory: the CMake layout puts POSIX libraries in lib and Windows DLLs in bin, so the mingw wheel would have looked for its DLL among the import libraries and stopped there Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * State that MuSig2 belongs to btclib, rather than promising it here "No dedicated binding module yet ... still to be designed" read as a module owed by this package. It is not one: what the two-round protocol needs is a session whose secret nonce cannot be reused, and that is a property of an object's lifetime, which only whoever owns the session can enforce. Every function here is one libsecp256k1 call with its arguments validated, and holds no state between calls; a musig module would be the first exception, in the one place where getting it wrong leaks a private key. Where the signing state already lives is btclib, whose PSBT is the multi-party machinery MuSig2 plugs into, and the specifications are laid out the same way: BIP327 the protocol, BIP373 its PSBT fields, BIP328 its descriptors. What is left for this package is the part with no state, and it is already here: the key ordering and aggregation of keys, the taproot tweaking of xonly, the tagged hash of hashes, and ssa.verify, since an aggregate MuSig2 signature is a plain BIP340 signature. The 17 musig entry points stay reachable through lib, where the test that drives a 2-of-2 session uses them. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Report what libsecp256k1 says through its callbacks - the vendored build replaces the abort()ing default callbacks with do-nothing stubs, so that an illegal argument cannot take the hosting process down. What that left was a bare 0 and no reason for it, and for the error callback something worse: libsecp256k1 reports there what it holds to be unreachable, and does not expect to be returned to - the shared context now sets both callbacks to record the message, and the new context.check() raises it: a ValueError for a violated precondition, a RuntimeError for an internal error, the message being the precondition itself, as libsecp256k1 stringifies the condition of its own check - no binding calls it, and none needs to: they validate their arguments before calling, so a violated precondition is unreachable through them. It is meant for a call made through lib, which has nothing in front of it, as a MuSig2 session is: partial signing zeroes the secret nonce and signing twice with it is refused through that very callback, which is the failure a session most needs explained - what is recorded is per thread, the callback running on the thread of the call that triggered it - the stubs stay as the defaults: they still cover every context whose callbacks are not set, such as the one test_core.py builds to check that an illegal argument cannot abort - ffi.callback works in ABI mode and in out-of-line API mode alike, so the dynamic and static extensions share the mechanism; the type has to be spelled as a pointer to function, the short form not parsing const - the ruff per-file ignore still described the autotools build Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Test the free-threaded interpreter, not just build for it The cp314t wheels were built on every platform, arm64 Windows included, without free-threading being asked for: cibuildwheel enables it now that 3.14 supports it officially. It also runs the suite against every wheel it builds, cp314t among them, so what was missing was not the tests but what these two matrices add on top: that pip, given a directory holding every wheel, selects the right one. - test-static gains 3.14t, so that the selection of a cp314t wheel over its GIL counterpart is checked - test-dynamic gains it as well, and there it matters more: the dynamic wheel is tagged py3-none, so it is what a free-threaded interpreter installs, and that job has no cibuildwheel behind it running the suite - no exclusion is needed. The actions/python-versions manifest carries a free-threaded 3.14 for every runner in the two matrices, win32-arm64 included, and cffi publishes cp314t wheels for all of them, so no job has to compile it from source Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Stop ignoring untracked content in the submodule The reason for it is gone. It was added because the autotools build wrote src/btclib_default_callbacks.c inside the vendored tree, automake resolving the sources it is added to relative to it, which left the submodule reported as modified forever and hid any real change behind that noise. The CMake build writes the stubs to its own binary directory instead, so there is nothing expected in there any more, and what the setting does now is the opposite of what it was for: hide a stray file rather than the absence of one. Verified by building: the submodule stays clean. The test of the musig module still described its lack of a wrapper as something not designed yet, which the README and HISTORY no longer do. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Assert the invariants over derived inputs, not over chosen ones Every line and branch is covered, which says nothing about a wrong result on an input nobody wrote down. The keys the suite uses are 1, 3, 5, 7, 11 and 13: none of their public keys has an x coordinate starting with a zero byte, and no signature over the one fixed message encodes to DER in 69 bytes. Both happen, at about 1 in 256 and 1 in 200 of arbitrary inputs, and the second is a length nothing here had ever produced. So the inputs are derived from a chain of SHA256 and the assertions are invariants: a round trip returning what it started from, a scalar operation agreeing with the point operation it corresponds to, both parties of ECDH and of BIP324 reaching the same secret, sorting agreeing with sorted(). Nothing is checked against a second implementation of secp256k1, and where an independent reference exists without one, hashlib for the tagged hash and for the ECDH derivation, it is used. Derived rather than random, and hypothesis is deliberately not a dependency. cibuildwheel runs this suite inside every wheel test environment, some forty of them across cp39 to cp314t and PyPy on five platforms, so a test dependency is installed forty times and can fail the release build in any of them; and a sweep that fails once in a thousand runs is not a gate. A SHA256 chain gives the variety with a failure reproducible at the exact iteration. What it gives up is shrinking to a minimal counterexample, which for 32-byte values is worth little, and the bias towards boundary values, which is why the ends of the scalar range and the two rare cases above are pinned by hand. 128 inputs per invariant, 0.06s for the eleven tests. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Let the release workflow rehearse itself on TestPyPI A tag is the point of no return: a version on PyPI can be yanked, never replaced, and this workflow has never run. So it can now be dispatched by hand, and a manual run publishes to TestPyPI. It is the same file that the release uses, deliberately: a trusted publisher is registered for a workflow filename, so a release-test.yml of its own would prove nothing about this one. The rehearsal exercises the OIDC exchange, the approval gate, the artifacts the publish job collects, the PEP 740 attestations, and a real Warehouse accepting the metadata, which is more than twine check --strict can say. The two publish jobs are two jobs, not one parameterized by an expression, because the environment name is what enforces the manual approval: an expression that did not evaluate as intended would send the job to some other environment and publish with no gate in front of it. The event decides which job runs; the index and the environment of each are literals. The matrices treat a dispatch like a tag, so the rehearsal builds the Intel macOS artifacts too. Without that it would skip precisely what a release adds to a development run, and dispatching from master is not an alternative while master is the commit before all of this. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Set the version to 0.7.1rc1, for the TestPyPI rehearsal REVERT THIS COMMIT BEFORE TAGGING. The version that goes to PyPI is 0.7.1; this one exists so that the rehearsal has a version number of its own to consume, TestPyPI refusing a reupload exactly as PyPI does. A second attempt needs rc2, a local version (0.7.1+test1) being refused by both indexes. A pre-release also stays out of the way of anyone pointing pip at TestPyPI without --pre. HISTORY.md is left alone: the release being prepared is 0.7.1, and a rehearsal is not a release. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Download only the wheel each test job installs - every row of the two test matrices downloaded every wheel in order to install one: with a pattern and merge-multiple, four dynamic artifacts or six static ones per job, hundreds of ListArtifacts calls per run. The artifact service answers a fan-out that wide with the occasional 403, which is how the run of the CMake commit failed: one job out of forty-five, the same step then downloading two artifacts successfully, every build job green - each job now asks for its own artifact by name. The x86_64 Windows dynamic wheel is the mingw cross-compiled one, hence the only name that is not the runner's - it also makes the step stricter: a missing artifact now fails naming what it looked for, instead of surfacing later as a pip resolution error, or not at all - it is a mitigation and not a fix, the 403 coming from the artifact service: a retry would mean either a third-party action, against the SHA-pinned hardening of this workflow, or a hand-rolled gh run download, which loses the digest verification the action does Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Record why linking a system libsecp256k1 stays out - what a distribution packager needs is to link the libsecp256k1 already installed, Debian, Fedora, conda-forge, Nix and Alpine all having policies against a vendored copy of a cryptographic library. The README now accounts for the decision not to offer that, in favour and against, rather than leaving it in a closed issue - two of the arguments against were checked rather than assumed: libsecp256k1 has no runtime version function and no version macro in its headers either, so a system build would go on claiming the version it wraps over a library of any vintage, unverifiably; and its abort()ing default callbacks would diverge from the vendored build, where the shared context sets its own, so a context created through lib could take the process down - it stays out until a packager asks for it: the value is in opening a channel, and the costs are paid from the first day, whether anyone walks it or not Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Put README.md back to LF, and keep it there - it was the one CRLF file of the forty-two tracked, having become one in e915656 (Nov 2021, by the look of it an edit through the GitHub web interface). The diff is line endings only: git diff --ignore-cr-at-eol is empty - what let it survive four years is the default of the hook meant to catch it: --fix=auto normalizes a file to its own most frequent ending, so a file entirely in CRLF is consistent by that measure and stays, while the LF lines added to it are converted the wrong way, which is what happened to the section added yesterday. --fix=lf is the argument that would have caught it in 2021 - the BIP340 vectors are excluded from the hook: they are vendored from bitcoin/bips byte for byte, CRLF included, and being the upstream file anyone can fetch and diff is what makes them worth trusting; the csv module, read with newline="" as it is, takes them either way Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Give the release procedure a file of its own - the steps to cut a release were a README section, read by whoever cuts one and by nobody else, in a file that is also the PyPI long description. They move to RELEASING.md, the README keeping the paragraph on what publishes and how (Trusted Publishing, PEP 740) and a pointer to the rest; the Versioning section moves up, next to the install line it qualifies - the checklist grew what doing it twice showed was missing: uv lock after the bump, uv.lock holding the version too; the libsecp256k1 version named at the top of the README, when the submodule moved; git push origin v0.7.1 rather than --tags, which pushes whatever else is lying around; the install and the attestations checked after the upload; the GitHub release, which the workflow does not create though pyproject points its download URL at that page - and what nothing said: the version published is the one in pyproject.toml, the tag only choosing the index, with nothing cross-checking the two. A v0.7.1 tag on a tree still reading 0.7.1rc1 publishes 0.7.1rc1 and burns it - a section on a release found broken, since nothing can be reuploaded: yank, then the fourth number of the versioning scheme - the last step of the rehearsal was a 1. that restarted the list, the markdownlint comment around the long pip line breaking out of it at column 0. Indented, it renders as step 4 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Anchor the extension exclude, so the sdist keeps its stub - the sdist excluded _btclib_libsecp256k1.* unanchored, which also matches stubs/_btclib_libsecp256k1.pyi: hatchling appends the configured patterns after the .gitignore ones in a single GitIgnoreSpec, where a later pattern overrides the earlier !stubs/_btclib_libsecp256k1.pyi negation. The stub the strict mypy gate needs was therefore missing from the sdist - anchored to the root, it excludes the built extension it is meant to exclude and nothing else Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Correct the documentation that PyPI publishes - the README is the PyPI long description, frozen at release time on a page that is not the repository: its two relative links, SECURITY.md and RELEASING.md, resolve to a 404 there. Absolute GitHub URLs, as the license badge already used - the submodule transcript showed the SSH URL of a clone that predates .gitmodules, which registers the HTTPS one: what git prints is the URL it reads from there, and a reader comparing the two would find they disagree - three typos in docstrings that ship in the wheel, Multply twice in mult.py and Schhnorr in ssa.py: help() and the editors of whoever calls these bindings show them verbatim Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Say that a rehearsal is never tagged - the trigger is what picks the index: a v* tag publishes to PyPI, a manual run to TestPyPI. So the rc the rehearsal asks for is one tag away from the real index, and nothing said so: a v0.7.1rc1 tag would publish the pre-release to PyPI, burn that version there, and leave it for whoever installs with --pre - the step that says to run the workflow from the Actions tab is where that belongs, being the step a tag would replace Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Fix two declarations pyproject.toml gets wrong - the metadata is PEP 639 (a license expression and license-files, replacing the deprecated table and the classifier), which hatchling understands from 1.27 and rejects before it. The build requirement had no floor, so a frontend resolving an older hatchling fails on the metadata rather than on the requirement, which is the wrong place to find out - the F401 ignore for __init__.py described a namespace assembled by re-export, and there is none: the four imports of that module are all used, the submodules are imported by whoever wants them, and the ignore never fired. Removed, so that an unused import there is reported like anywhere else Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Give the dynamic macOS wheel a deployment target - a dynamic (cffi ABI mode) wheel compiles no extension, so no toolchain fixes the macOS version in its platform tag: without MACOSX_DEPLOYMENT_TARGET the tag is whatever OS version the runner happens to be on, and rises with it (macosx_26_0 when built on a developer machine already there). pip on anything older refuses a wheel that would have loaded, and what it falls back to is the static wheel, or the sdist - CMake initializes CMAKE_OSX_DEPLOYMENT_TARGET from the same variable, so the vendored library is built for the floor its wheel then claims, and delocate finds the two agreeing - 11.0 is where arm64 macOS starts; 10.13 is what cibuildwheel gives the static x86_64 wheels of CPython 3.12 and later, so the two kinds of wheel now accept the same systems Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Install the sdist on Windows too - the sdist job installed it on Linux and macOS only, and Windows is where that build is least like the others: MSVC compiles the extension, a gcc on the PATH preprocesses the headers for cffi, and the static archive it links is named libsecp256k1.lib rather than libsecp256k1.a. Nothing exercised that path from a source distribution, so a file the sdist fails to ship for it would surface on PyPI, where an sdist cannot be replaced - the install step runs under bash on every runner: pip does not expand the glob itself and PowerShell does not expand it for a native command, so the default shell there would hand pip dist/*.tar.gz verbatim Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Ignore .ruff_cache without relying on ruff - the cache was untracked only because ruff writes a .gitignore inside it: that hides the directory once ruff has created it, and says nothing about a tree where it does not exist yet, or about a ruff that stops doing it. .pytest_cache and .mypy_cache were already listed here Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Document what the scripts folder contains The two build scripts are the build backend of this package, and the only account of them was their own comments: what invokes them, in which order, and which environment variable selects which of the three extension modes was not written down anywhere outside .github/workflows. The new scripts/README.md says it, and states the one non-obvious rule of that code, that host and target are distinct predicates: the choice between the MSVC and the Unix static path keys off the real host, while the shared library suffix and the wheel tag key off the target. It ships in the sdist, next to the scripts it documents, which is where it is needed: building from source runs two of them. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Build the Intel macOS artifacts on macos-26-intel - macos-13 was retired on 4 December 2025: its jobs no longer start, they queue until something cancels them, so every artifact that only it produced was in fact never built. macos-26-intel is the x86_64 image now, generally available since February 2026 and, like every other Intel label, alive until Actions drops the architecture in August 2027, which is the real deadline for the macosx x86_64 wheels - with a working runner the queue-time gating goes: the three conditional exclude expressions that kept the Intel jobs off every branch but master are gone, so the whole matrix runs on every push and no artifact appears for the first time on a release tag. What is left of the exclude in test-static (PyPy on Windows, no CPython arm64 before 3.11) is a plain YAML list again rather than two JSON blobs inside an expression - the deployment target of the dynamic wheel stays 10.13 on Intel: the Xcode of the macOS 26 image still compiles for it, and the static wheels are unaffected either way, cibuildwheel setting their target from the interpreter and not from the host Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * State the design: a boundary, not a library The README argued each decision in its own section, but never said what they add up to. Say it once, up front: the cryptography is upstream's, wrappers can only differ at the boundary, and list what this boundary promises — known version, complete surface, no crashing input, randomized context, typed signatures, independent validation, checkable provenance. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * reordered * Run the matrix once per commit, not once per event A push to a branch with an open pull request fired both triggers, so the whole workflow ran twice on the same commit. The concurrency group could not collapse the pair either: github.ref is refs/heads/dev for the push and refs/pull/N/merge for the pull request, two groups, so neither run cancelled the other and two ~100-job matrices competed for the same runner pool, making both slower than one would have been. The push trigger is now master only. Nothing is tested less: the whole matrix still runs on every commit, and the run that survives is the better of the two, because it tests the merge commit rather than the branch tip and is the only run a fork's pull request produces at all. master keeps a trigger of its own since a merge creates a commit that is not the one the pull request tested, and nothing else would ever test it. workflow_dispatch is the escape hatch for the gap this opens, a branch whose pull request is not open yet. It becomes usable once this reaches master: Actions reads dispatch triggers from the default branch. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Test Python 3.15 where a beta can say something 3.15 is in beta until October 2026, and two jobs can run it today: - test-dynamic gains 3.15 and 3.15t. The dynamic wheel is py3-none, so it is exactly what pip hands an interpreter that new — no cp315 wheel exists on any index — which makes this the wrapper's own compatibility test, the layer where a removed API or a changed exception would bite - test-sdist gains a version dimension, ["3.14", "3.15"], 3.14 being the interpreter .python-version pins for a developer. This is the one job that compiles the extension against the beta's headers, and the only reason it can: cibuildwheel refuses to build a cp315 wheel while 3.15 is not ABI stable, so nothing else would compile against 3.15 until the release candidate lands, which is when a break would have to be fixed against a deadline rather than now test-static is deliberately untouched. There is no cp315 wheel to test, and producing one means enable = ["cpython-prerelease"], which upstream documents as testing-only and asks not to be uploaded to PyPI: since release.yml reuses build-cibuildwheel through workflow_call, enabling it would put wheels of that kind in the artifacts of a release. Once 3.15 reaches release candidate the identifier appears without any flag, and "3.15" can join that matrix with no other change. Mechanics: setup-python needs allow-prereleases, as 3.15 has no general availability release to resolve to; this leaves the rest of the matrix alone, a range only falling back to a prerelease when no GA one exists. continue-on-error is scoped to the 3.15 entries by expression, so a beta reports without holding up a merge. The renamed test-sdist jobs break no required status check: master protection requires none by name. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Revert "Test Python 3.15 where a beta can say something" This reverts commit 885a7ca6. The library is not what failed: on 3.15.0b4 the whole suite passes, 77 tests, once cffi is built from source. What failed is cffi's own cp315 wheel, compiled against 3.15b3, whose PyThreadState grew a field in b4 (python/cpython#152448). The offset it was compiled with is now wrong, so the process segfaults in take_gil the moment C re-enters Python through a libffi closure — here the illegal_callback that secp256k1_ec_pubkey_parse invokes on invalid input. Upstream is python-cffi/cffi#263, whose fix is a new cffi release. Keeping the jobs would mean thirteen red ones in every run until 3.15 reaches release candidate, for a cause outside this repository, and a permanently red matrix is one nobody reads. Building cffi from source in CI would hide the red, at the price of a workaround nobody would remember to remove. So 3.15 comes back when there is something to learn from it: cffi wheels that match the interpreter, or the release candidate, which is also when cibuildwheel starts producing a cp315 wheel without a flag and test-static can take 3.15 in the same move. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Build the vendored library for the interpreter, not for the host The two toolchains in a source build read their target from different places. The cffi extension is compiled by the interpreter's own toolchain, which takes it from sysconfig.get_platform(), i.e. from the interpreter; CMake takes it from the machine it runs on. That is the same answer only as long as the two agree, and on Windows arm64 they need not: uv installs an emulated x86-64 CPython there by default ("support for the native architecture (aarch64) is not yet mature"), so MSVC compiled the extension for x86-64 against an arm64 archive and every secp256k1 symbol stayed unresolved at link time (LNK2001). Requesting the native interpreter by name worked, which is what identified the mismatch as the cause. CMake is now told the architecture, derived from get_platform(): -A on the Visual Studio generators, CMAKE_OSX_ARCHITECTURES on macOS. On POSIX that expression also follows the _PYTHON_HOST_PLATFORM a cross-compiling cibuildwheel sets, so an arm64 macOS wheel built on an Intel runner is built for arm64 throughout. macOS gains a fix of its own besides Rosetta: a universal2 interpreter compiles the extension for both architectures, and could not link a single-architecture archive at all. Linux has no equivalent knob, a 32-bit interpreter on a 64-bit host needing the -m32 of a multilib toolchain rather than a CMake target. The platform tag of a dynamic wheel had the same bug, from the same cause: platform.machine() reports the host, deliberately so (it reads PROCESSOR_ARCHITEW6432 to see past the emulation), while the library that wheel carries is now built for the interpreter. It follows get_platform() too, which also gives 32-bit Windows and universal2 the tag they were missing. test-sdist gains windows-11-arm with architecture x64. No wheel job can catch this: they all build with an interpreter of the runner's own architecture, so the mismatch never arises. That runner is the one place where another architecture can be asked for, the arm64 image having the x64 MSVC tools and the gcc the header preprocessing needs, and the sdist is the path that compiles on the user's machine. test-static keeps its shape: setup-python resolves to the native arm64 interpreter there, as its jobs installing win_arm64 wheels prove, so the matrix needs no job of its own. Verified where a mismatch can be reproduced locally, an x86-64 CPython 3.13 on arm64 macOS: the vendored archive is x86_64 and the 77 tests pass. That case turns out to work before the change as well, CMake itself running translated and /usr/bin/cc launching its x86_64 slice in turn, so the build followed the interpreter by accident; what the change adds there is CMAKE_OSX_ARCHITECTURES naming x86_64 instead of leaving it to that accident. Windows arm64 is what the CI job is for. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Give linting a workflow of its own The pre-commit gate and the build matrix shared a workflow, so a style error and a broken wheel were the same red mark: one badge, one entry in the checks list, and nothing telling them apart until the run is opened. Split into lint.yml and test.yml (the former main.yml, renamed for what is left of it). The release workflow calls both, and both are a gate on publishing, which is what reusing main.yml already meant. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Check what a release cannot survive being wrong about The release process rested on the tag and the tree agreeing, and nothing compared them: a v0.7.1 tag on a tree still reading 0.7.1rc1 published 0.7.1rc1 and burnt that version, which can then only be yanked. RELEASING documented the hazard rather than closing it. A version-check job now runs before anything is built: the tag against the declared version, uv.lock against pyproject.toml, and the libsecp256k1 release named in README.md against the commit the submodule is pinned to (resolv…
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Backport of gh-151613 to Python 3.15.
Original commit: 8cda6ae
Original PR: GH-151614