Skip to content

Optimize memory for 512MB free tier: tune ONNX session options and reduce TF-IDF features (#11) - #10

Closed
SparshM8 wants to merge 1 commit into
mainfrom
fix/memory-optimization-v2
Closed

SparshM8 wants to merge 1 commit into
mainfrom
fix/memory-optimization-v2

Conversation

@SparshM8

Copy link
Copy Markdown
Owner

This PR implements memory optimizations for the 512MB free-tier host:

  1. Tuned ONNX Runtime SessionOptions to disable memory pre-allocation and aggressive optimizations.
  2. Reduced TF-IDF vectorizer max_features from 20,000 to 10,000 to shrink the similarity index footprint.
  3. Added session config entries to use the device allocator for initializers.

These changes are expected to reduce the runtime memory footprint by 50-80 MB, providing safer headroom for concurrent requests.

@SparshM8 SparshM8 closed this Aug 21, 2026
@SparshM8
SparshM8 deleted the fix/memory-optimization-v2 branch August 21, 2026 20:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant