Skip to content

os: add support for windows - #107

Open
AmyJeanes wants to merge 7 commits into
commaai:masterfrom
AmyJeanes:windows
Open

os: add support for windows#107
AmyJeanes wants to merge 7 commits into
commaai:masterfrom
AmyJeanes:windows

Conversation

@AmyJeanes

@AmyJeanes AmyJeanes commented Sep 7, 2026 •
edited
Loading

Copy link
Copy Markdown

Windows (win_amd64) wheels for all fifteen packages, built with clang, lld and libc++ from an MSYS2 CLANG64 shell, so that openpilot's PC tooling can be built and run natively on Windows for development. The other PRs in the series consume these wheels; openpilot's Windows CI job currently builds against a release of them from my fork and runs the full unit test suite there.

What changes:

  • setup.py (every package): wheels tagged win_amd64; bash resolved through PATH (CreateProcess searches System32 first, which is the WSL launcher); build.sh paths passed with forward slashes; the build interpreter handed to build.sh as PYTHON so the scripts stop assuming a python3 on PATH.
  • build.sh per package, in one commit with the rationale per package in its message. The notable decisions:
    • ffmpeg is static (consumers need no DLL search path handling), hardware decode is Direct3D 11 (h264/hevc_d3d11va, any GPU vendor, mingw SDK headers suffice), and the network layer is off since only the file and pipe protocols are enabled.
    • acados uses the GENERIC BLASFEO/HPIPM targets (the x86 assembly kernels assume the SysV ABI) and the pinned tera_renderer release binary instead of requiring a rust toolchain.
    • zeromq drops the AF_UNIX transport (afunix.h is MSVC-only; openpilot uses tcp:// on Windows).
    • ncurses ships PDCurses' console port under the ncurses names; ncurses itself does not build cleanly for the Windows console with mingw, and the curses API replay uses is covered.
    • capnproto and cppcheck link libc++ statically so the binaries run outside an MSYS2 shell.
    • raylib builds the desktop backend with clang through setuptools' mingw32 compiler (a tracked build_ext config).
    • The Python entry points run the binaries through subprocess on Windows: os.exec* cannot replace the process.
  • setup.sh: pacman packages for the CLANG64 environment; other MSYS2 environments are refused. .gitattributes keeps the scripts LF on Windows checkouts.
  • CI: a fourth matrix entry on windows-latest (GitHub-hosted; the Namespace expression falls through to it) and a test windows job that installs the wheels into a venv of a stock Python with no MSYS2 on PATH and runs the smoketests, the Windows counterpart of the distro test. The publish job uploads the Windows wheels with the others (bootstrap-icons is a py3-none-any duplicate, which --skip-existing drops). A matrix entry can name its shell (the Windows one runs in the CLANG64 shell from setup-msys2); the others keep bash -e {0}, GitHub's implicit default, through the job's run defaults. An explicit shell: bash would add pipefail, which breaks the Xcode select step on macos-latest (macOS 26 has no Xcode 16).

Not included: libdatachannel-py has no Windows wheel on PyPI and is not this repo's package; the openpilot PR keeps teleoprtc off Windows.

Tested: this workflow on this PR, all four platforms plus the distro containers: every wheel builds, the test windows job installs the fifteen win_amd64 wheels into a venv of a stock Python with no MSYS2 on PATH and runs each package's smoketest, and the Linux and macOS jobs are unchanged. The wheels are what openpilot's Windows CI job syncs and builds against, and its full unit test suite passes there (856 tests): commaai/openpilot#38810. Locally, a clean-room uv sync of the wheels into a fresh venv and cache, then openpilot's scons build and the same suite. Cold build times on GitHub-hosted runners: Windows 17 min (11 with ccache).

This PR is part of a larger series of work bringing support for Windows to openpilot:

This PR goes first: the rednose, opendbc and panda PRs are drafts whose Windows CI entries install the wheels it publishes, and turn ready once they are on PyPI. Otherwise each PR merges independently; only the openpilot one depends on all of them.

The changes in this PR were generated by Claude Fable 5.1 but were human reviewed and fully tested end to end both locally and in CI.

AmyJeanes and others added 7 commits September 7, 2026 19:33
@AmyJeanes @claude
setup.py: build wheels on Windows (MSYS2)
3c5859c 
Tag wheels win_amd64, resolve bash through PATH (CreateProcess searches
System32 first, which would launch the WSL bash), pass the build.sh path
with forward slashes since MSYS2 bash treats backslashes as escapes, and
hand the build interpreter to build.sh as PYTHON so scripts stop assuming
a python3 on PATH.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016AgqYZYWLpwE2T5vS3nVEt
@AmyJeanes @claude
Build Windows (win_amd64) wheels under MSYS2 clang64
e546d79 
Per-package changes so every build.sh runs in an MSYS2 CLANG64 shell:
- capnproto/ffmpeg/git-lfs/gcc-arm-none-eabi: .exe names, Windows
  archives, and native C:/ prefixes for pkgconf; ffmpeg is built static
  so consumers do not need DLL search path handling
- acados: GENERIC BLASFEO/HPIPM targets (the x86 assembly kernels assume
  the SysV ABI), the pinned tera_renderer release binary instead of a
  rust toolchain, Windows venv layout when vendoring casadi
- zeromq: no AF_UNIX transport on MinGW (afunix.h is MSVC only)
- raylib: desktop backend with clang and the mingw distutils compiler,
  configured through a tracked build_ext config file; the cffi build
  output under raylib/Release is ignored
- ncurses: PDCurses' console port shipped under the ncurses names,
  decided before the ncurses tarball would be downloaded
- shims: run binaries via subprocess on Windows, os.exec* cannot replace
  the process there

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016AgqYZYWLpwE2T5vS3nVEt
@AmyJeanes @claude
ffmpeg: enable Direct3D 11 video decoding in the Windows wheel
3118765 
replay selects AV_HWDEVICE_TYPE_D3D11VA on Windows; that needs the
h264/hevc d3d11va2 hwaccels, which share their objects with the legacy
d3d11va ones, so both variants are enabled. Only the mingw SDK headers are
needed, no extra libraries.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016AgqYZYWLpwE2T5vS3nVEt
@AmyJeanes @claude
build.sh: run the smoketests on Windows too
de49390 
The venv's interpreter lives in Scripts/ there, and MSYS2 puts its own
mingw CPython first on PATH, so pin uv to the managed 3.12 the wheels
are built for.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016AgqYZYWLpwE2T5vS3nVEt
@AmyJeanes @claude
capnproto, cppcheck: self-contained Windows binaries
bf9133a 
capnp and cppcheck linked libc++ dynamically, so they only ran from a
shell with the MSYS2 CLANG64 bin directory on PATH; link them -static
like ffmpeg. cppcheck also gets the subprocess shim the other packages
use on Windows (os.execvp cannot replace the process there), and the
capnproto smoketest joins PATH with os.pathsep so it runs there too.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016AgqYZYWLpwE2T5vS3nVEt
@AmyJeanes @claude
setup.sh: MSYS2 CLANG64 build dependencies
271360c 
pacman installs the toolchain and build tools by MINGW_PACKAGE_PREFIX;
the other MSYS2 environments are refused since the wheels rely on
clang, lld and libc++. .gitattributes keeps the scripts LF on Windows
checkouts.

The uv shell installer unpacks with unzip, which a fresh MSYS2 lacks (a
native unzip on PATH cannot open MSYS paths). MSYS2's git package is not
installed: openpilot's venv git-lfs cannot drive that Cygwin-style git,
so Git for Windows on the inherited PATH is the git for both repos.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016AgqYZYWLpwE2T5vS3nVEt
@AmyJeanes @claude
CI: build and test the win_amd64 wheels
013ba36 
A fourth matrix entry builds on windows-latest in an MSYS2 CLANG64 shell
with the runner's PATH inherited; ./setup.sh installs the pacman
packages. Only that entry names its shell: the job's run default falls
back to `bash -e {0}`, GitHub's implicit default, so the Linux and macOS
steps are unchanged. A `test windows` job installs the wheels into a venv
of a stock Python with no MSYS2 on PATH and runs the smoketests, the
Windows counterpart of the distro-container test: it is the only place a
wheel that still depends on the toolchain's DLLs fails. The publish job
then uploads the Windows wheels to PyPI with the others (bootstrap-icons
is a py3-none-any duplicate, which --skip-existing drops).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016AgqYZYWLpwE2T5vS3nVEt
This was referenced Sep 7, 2026
Closed
Draft
Draft
Open
Draft
Closed
Closed
Closed
AmyJeanes added a commit to AmyJeanes/panda that referenced this pull request Sep 7, 2026
@AmyJeanes @claude
TEMP: Windows wheels from a pre-release index until commaai/dependenc…
0c5376b 
…ies#107 publishes

--- TODO REMOVE AFTER DEPENDENCY PR MERGES (commaai/dependencies#107) ---

This commit is dropped from the PR once the win_amd64 wheels are on PyPI.
Until then it points uv at a GitHub release of the same wheels, built from
that PR's branch on my fork, so the Windows CI entry runs here meanwhile:
comma-deps-gcc-arm-none-eabi and comma-deps-cppcheck for sys_platform ==
'win32'. Linux and macOS resolve from PyPI as before.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016AgqYZYWLpwE2T5vS3nVEt
AmyJeanes added a commit to AmyJeanes/rednose that referenced this pull request Sep 7, 2026
@AmyJeanes @claude
TEMP: Windows wheels from a pre-release index until commaai/dependenc…
99881b5 
…ies#107 publishes

--- TODO REMOVE AFTER DEPENDENCY PR MERGES (commaai/dependencies#107) ---

This commit is dropped from the PR once the win_amd64 wheels are on PyPI.
Until then it points uv at a GitHub release of the same wheels, built from
that PR's branch on my fork, so the Windows CI entry runs here meanwhile:
comma-deps-eigen for sys_platform == 'win32'. Linux and macOS resolve from
PyPI as before.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016AgqYZYWLpwE2T5vS3nVEt
AmyJeanes added a commit to AmyJeanes/opendbc that referenced this pull request Sep 7, 2026
@AmyJeanes @claude
TEMP: Windows wheels from a pre-release index until commaai/dependenc…
5fd359b 
…ies#107 publishes

--- TODO REMOVE AFTER DEPENDENCY PR MERGES (commaai/dependencies#107) ---

This commit is dropped from the PR once the win_amd64 wheels are on PyPI.
Until then it points uv at a GitHub release of the same wheels, built from
that PR's branch on my fork, so the Windows CI entry runs here meanwhile:
comma-deps-cppcheck for sys_platform == 'win32', and uv.lock relocked
against that index. Linux and macOS resolve from PyPI as before; the lock
is regenerated from PyPI when this commit goes.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016AgqYZYWLpwE2T5vS3nVEt
@AmyJeanes AmyJeanes mentioned this pull request Sep 8, 2026
Draft
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@AmyJeanes