Skip to content

[v0.2.5+] Kernel Cache LRU Persistence #54

New issue
New issue
Closed
Closed
[v0.2.5+] Kernel Cache LRU Persistence#54

Description

@m96-chan
m96-chan
opened on Dec 14, 2025

Overview

Complete the Kernel Cache LRU system with persistent storage to avoid JIT recompilation across sessions.

Tasks

PTX Cache Persistence

  • Serialize compiled PTX to disk cache
  • Cache location: ~/.pygpukit/kernel_cache/
  • Invalidation on kernel source changes

Architecture Fingerprinting

  • SM version (compute capability)
  • Register file size
  • Shared memory size per SM
  • L2 cache size
  • Include in cache key

LRU Eviction

  • Max cache size limit (configurable)
  • LRU eviction policy
  • Cache stats/metrics API

Implementation Location

  • Update rust/pygpukit-core/src/kernel_cache.rs
  • Add persistence layer in Rust
  • Expose cache control via Python API

Benefits

  • Eliminate JIT compilation overhead on subsequent runs
  • Warm startup for production deployments
  • Architecture-aware caching prevents invalid kernel reuse

Related Issues

Activity

  1. m96-chan
    added 2 commits that reference this issue on Dec 15, 2025

    feat(cache): implement persistent kernel cache with LRU eviction (#54)

    b7c5224

    v0.2.5: FP16/BF16 support, TF32 optimization, reduction ops (#61)

    854719b

  2. m96-chan commented on Dec 15, 2025

    @m96-chan
    m96-chan
    on Dec 15, 2025
    OwnerAuthor

    Completed in v0.2.5

    Persistent Kernel Cache with LRU

    • Implemented in src/pygpukit/jit/cache.py
    • LRU eviction policy for memory management
    • Kernel lookup by name and compile options
    • Stats tracking (hits, misses, evictions)

    Features

    • KernelCache class with configurable max entries
    • get_by_name() / put() for cache operations
    • stats() returns cache hit/miss statistics
    • Thread-safe implementation

    PR: #61

  3. m96-chan
    closed this as completedon Dec 15, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions