Windows backend for msgq and visionipc - #1
Closed
AmyJeanes wants to merge 4 commits into
Closed
Conversation
Queues and the fake-event state are named shared-memory sections rather than files: Windows has no /dev/shm, a section needs no cleanup (the kernel drops it with its last handle or view, where a Linux queue file stays until reboot) and queue files could not be swept safely, since an NTFS delete succeeds on a file that is only mapped and leaves the mapping attached to an orphan. A section keeps its name only while a handle to it is open, so every mapping holds one for its lifetime: the queue struct carries it and the event state keeps one per view. msgq_shm_dir() picks the queue directory on Linux and macOS (/dev/shm, /tmp) instead of three ifdef ladders; on Windows it locates the visionipc sockets under %TEMP%, "/tmp" being the current drive's root there. Readers block in msgq_poll() on a per-thread named event which publishers signal by thread id, replacing the SIGUSR2/tkill wakeup; the wait loops until the steady clock passes the deadline, since WaitForSingleObject's timeout runs on the interrupt clock and can expire a few microseconds early. Fake events use named Win32 events instead of FIFOs. visionipc uses AF_UNIX stream sockets with length-prefixed messages. The buffers are anonymous sections whose handles the server duplicates into the client process, the Windows form of SCM_RIGHTS; the socket reports the peer's pid (SIO_AF_UNIX_GETPEERPID). The C++ tests now release their queues: a section is created afresh once nobody maps it, where the tests used to rely on deleting the queue file. test_receive_timeout measures with perf_counter: on Windows Python 3.12 time.monotonic() is the 15.6 ms tick clock and read a 5 ms wait as 0 ms about once in a hundred runs. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016AgqYZYWLpwE2T5vS3nVEt
toggle_fake_events() and set_fake_prefix() set CEREAL_FAKE* through the C runtime's putenv. Python's os.environ is a snapshot taken at startup, and on Windows multiprocessing spawns children from that snapshot (in a venv it hands os.environ.copy() to CreateProcess), so the replayed process in process_replay never entered fake mode and the test waited for it forever. Keep os.environ in sync from the Python wrappers; a forked child on Linux inherits both anyway. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016AgqYZYWLpwE2T5vS3nVEt
Python imports extension modules as .pyd on Windows. The cython tool renames the .so targets the SConscripts declare, so neither they nor a SConstruct that loads the tool has to know. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016AgqYZYWLpwE2T5vS3nVEt
Windows checkouts default to CRLF, which breaks the shell scripts and makes every file look modified from an MSYS2 shell. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016AgqYZYWLpwE2T5vS3nVEt
Owner
Author
|
Superseded by commaai#709: comma's CI runs on the upstream PR now. The follow-up (#2) stacks on the windows branch and stays open. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fork-internal draft: runs this repository's own CI on the
windowsseries before it is proposed upstream. Not for merging.🤖 Generated with Claude Code
https://claude.ai/code/session_016AgqYZYWLpwE2T5vS3nVEt