Repository navigation
Conversation
Signed-off-by: Honglie Yi <hyi@amd.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this PR does / why we need it:
Add an opt-in chunk-store server module that returns a completion event for each chunk, allowing explicit RPC clients to reuse source KV blocks as their transfers finish. The module loads through
--server-module, reuses the existing transfer module, and exposes a typed RPC over ZMQ and gRPC. Includes unit tests, cross-process GPU tests, and usage/design documentation.Special notes for your reviewers:
This PR is entirely within
yhl-amd/LMCache. Its base,codex/server-module-plugins, is an exact copy of maobaolong's plugin branch at352ea6bc56ff814c61df5be6332399465cbb0a29. The head adds one signed-off commit. Files changed contains only the chunk-store delta: 9 files, +723/-0 (190 lines of implementation/protocol, 453 test lines, 80 documentation lines). The feature patch is identical to the previously reviewed implementation.Each chunk uses the ordinary store path independently: earlier chunks may remain committed if a later chunk fails. Event handles inherit the existing backend retention contract. Submission overhead has not been benchmarked.
Validation on head
74f63569e8fbbd94a9985fd33d6bb687d159a836, directly on the plugin baseline:mi355-gpu-51through Slurm job7803, with one GPU; allocation released after completion. Python 3.10.12, PyTorch 2.10.0+rocm7.0, HIP 7.0.51831. Reused an existing native build; native extension sources are unchanged.The warnings are the existing OpenTelemetry
LoggingHandlerdeprecation.Reproduce the test selection
If applicable: