Skip to content

Memory Leak Fixes - #258

Open
L1l1thLY wants to merge 1 commit into
asg017:mainfrom
L1l1thLY:main
Open

Memory Leak Fixes#258
L1l1thLY wants to merge 1 commit into
asg017:mainfrom
L1l1thLY:main

Conversation

@L1l1thLY

@L1l1thLY L1l1thLY commented Feb 3, 2026

Copy link
Copy Markdown

I found a few memory leaks in the code and attempted to fix them. Could you please take a look and confirm whether the fixes are correct? I’m not entirely sure, but the changes are as follows:

  1. Line 2537-2538 (vec_eachFilter): pzErrMsg allocated but not freed on error
  2. Line 1651-1652 (vec_slice): Early return without calling cleanup(vector)
  3. Line 1674-1675 (vec_slice): Early return without calling cleanup(vector)
  4. Line 5145-5152 (vec0_init): zSql not freed before goto error
  5. Line 7064 (vec0Filter_knn): knn_data not freed on cleanup when error occurs
  6. Line 8276-8278 (vec0Update_Insert): zSql not freed before goto cleanup
  7. Lines 9409-9468 (vec_static_blob_entriesFilter): Multiple memory leaks - err, knn_data, queryVector, topk_rowids, distances, candidates, taken not properly freed on errors

@L1l1thLY

L1l1thLY commented Feb 3, 2026

Copy link
Copy Markdown
Author
  1. vec0_free function (lines 3601-3638)

Issue: The function did not free partition column names, auxiliary column names, metadata column names, and metadata chunk shadow table names.

Fix: Added three loops to free this memory:

  • paritition_columns[i].name - partition column names
  • auxiliary_columns[i].name - auxiliary column names
  • shadowMetadataChunksNames[i] - metadata chunk shadow table names
  • metadata_columns[i].name - metadata column names
  1. vec0_init error handling (lines 5180-5182)

Issue: In the error path, only vec0_free(pNew) was called to free member variables, but the pNew structure itself was not freed.

Fix: Added sqlite3_free(pNew) after vec0_free(pNew).

  1. vec0Update_UpdateAuxColumn function (line 8666)

Issue: zSql allocated via sqlite3_mprintf was never freed.

Fix: Added sqlite3_free((void *)zSql) immediately after the sqlite3_prepare_v2 call.

  1. vec0_write_metadata_value function (lines 8108 and 8130)

Issue: zSql was not freed in two places:

  • SQL statement for UPDATE or INSERT operations
  • SQL statement for DELETE operations

Fix: Added sqlite3_free((void *)zSql) after each sqlite3_prepare_v2 call.

  1. vec0Update_Delete_ClearMetadata function (lines 8572 and 8580)

Issue:

  • zSql was not freed when sqlite3_prepare_v2 failed
  • stmt was not finalized when sqlite3_step failed

Fix:

  • Added sqlite3_free((void *)zSql) after the sqlite3_prepare_v2 call
  • Added sqlite3_finalize(stmt) in the error path

mceachen added a commit to photostructure/sqlite-vec that referenced this pull request Feb 10, 2026
@mceachen
fix(vec0): fix remaining memory leaks from PR asg017#258
c9be38c 
- vec_eachFilter: free pzErrMsg on error to prevent leak
- vec_slice: use goto done instead of return in INT8 and BIT cases
  to ensure vector cleanup is called on malloc failure

These were the last 3 unfixed memory leaks from upstream PR asg017#258.
The other 4 issues from that PR were already addressed in previous commits.

Upstream-PR: asg017#258
mceachen added a commit to photostructure/sqlite-vec that referenced this pull request Feb 10, 2026
@mceachen
test(error-paths): add comprehensive error path tests
95cc6c8 
Add test-error-paths.py with 30 tests specifically targeting error-handling
code paths that previously had memory leaks (fixed in PR asg017#258).

These tests ensure that error paths are exercised by the test suite so that
memory leaks would be caught by sanitizers (ASan/LSan) if reintroduced.

Tests cover:
- vec_each with invalid inputs (NULL, wrong types, malformed JSON, empty arrays)
- vec_slice error conditions (NULL, invalid types, bad indices)
- vector_from_value error paths (mismatched types/dimensions in distance/add/sub)
- vec0 INSERT/KNN errors (NULL, dimension mismatches, type mismatches)
- Metadata operations with invalid inputs
- Repeated error operations to stress-test cleanup paths

Note: malloc failure paths (OOM conditions) are not tested as they require
fault injection (SQLITE_TESTCTRL_FAULT_INSTALL) which is not available in
the standard build. The fixes ensure proper cleanup via 'goto done' instead
of early 'return' statements.

Related: Upstream PR asg017#258
@murillodutt murillodutt mentioned this pull request Feb 18, 2026
Open
stevemurr added a commit to stevemurr/sqlite-vec that referenced this pull request Feb 18, 2026
@stevemurr @claude
Fix multiple memory leaks in vec0 operations (INSERT, DELETE, KNN que…
12259fa 
…ries)

This commit applies fixes from PR asg017#258 and additional memory leak fixes
identified in issues asg017#245, asg017#259, and asg017#265.

Memory leak fixes include:

1. vec_slice: Early return without calling cleanup(vector) - now uses goto done
2. vec_eachFilter: pzErrMsg allocated but not freed on error
3. vec0_free: Missing cleanup for partition, auxiliary, and metadata column names
4. vec0_init: zSql not freed before goto error, pNew not freed on error path
5. vec0_metadata_filter_text: rowids not freed on error after allocation
6. vec0Filter_knn: knn_data not freed on cleanup when error occurs,
   array cleanup missing on error paths for metadata in-list processing
7. vec0_write_metadata_value: zSql not freed after sqlite3_prepare_v2
8. vec0Update_Insert: zSql not freed after sqlite3_prepare_v2
9. vec0Update_Delete_ClearMetadata: zSql not freed, stmt not finalized on error
10. vec0Update_UpdateAuxColumn: zSql not freed after sqlite3_prepare_v2
11. vec_static_blob_entriesFilter: Complete rewrite to properly handle all
    error paths - knn_data, queryVector, topk_rowids, distances, candidates,
    taken not properly freed on errors

These fixes address memory corruption and database corruption issues that
occurred during vec0 INSERT/DELETE operations and KNN queries.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
stevemurr added a commit to stevemurr/sqlite-vec that referenced this pull request Feb 18, 2026
@stevemurr @claude
Add memory safety tests for vec0 memory leak fixes
4eeb2f2 
Add comprehensive test suite that exercises code paths fixed in the memory
leak fixes from PR asg017#258:

- vec0_free: Tests create/destroy cycles with partition, auxiliary, and
  metadata columns to verify proper cleanup of column names
- vec0_init: Tests error conditions (too many partition keys, too many
  metadata columns, invalid syntax) to verify cleanup on error paths
- vec0Update_Insert: Tests batch inserts with various column types
- vec0Update_Delete_ClearMetadata: Tests DELETE operations with metadata
- vec0Filter_knn: Tests KNN queries with metadata filters
- vec0_metadata_filter_text: Tests KNN queries with text filters
- vec0_write_metadata_value: Tests inserts with metadata values

The tests include stress tests for rapid create/insert/drop cycles and
mixed operations that would surface memory leaks when run under memory
checkers like valgrind or ASAN.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Oldes pushed a commit to Siskin-framework/sqlite-vec that referenced this pull request Feb 23, 2026
@stevemurr @claude @Oldes
Fix multiple memory leaks in vec0 operations (INSERT, DELETE, KNN que…
875f29a 
…ries)

This commit applies fixes from PR asg017#258 and additional memory leak fixes
identified in issues asg017#245, asg017#259, and asg017#265.

Memory leak fixes include:

1. vec_slice: Early return without calling cleanup(vector) - now uses goto done
2. vec_eachFilter: pzErrMsg allocated but not freed on error
3. vec0_free: Missing cleanup for partition, auxiliary, and metadata column names
4. vec0_init: zSql not freed before goto error, pNew not freed on error path
5. vec0_metadata_filter_text: rowids not freed on error after allocation
6. vec0Filter_knn: knn_data not freed on cleanup when error occurs,
   array cleanup missing on error paths for metadata in-list processing
7. vec0_write_metadata_value: zSql not freed after sqlite3_prepare_v2
8. vec0Update_Insert: zSql not freed after sqlite3_prepare_v2
9. vec0Update_Delete_ClearMetadata: zSql not freed, stmt not finalized on error
10. vec0Update_UpdateAuxColumn: zSql not freed after sqlite3_prepare_v2
11. vec_static_blob_entriesFilter: Complete rewrite to properly handle all
    error paths - knn_data, queryVector, topk_rowids, distances, candidates,
    taken not properly freed on errors

These fixes address memory corruption and database corruption issues that
occurred during vec0 INSERT/DELETE operations and KNN queries.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
(cherry picked from commit 12259fa)
Oldes pushed a commit to Siskin-framework/sqlite-vec that referenced this pull request Feb 23, 2026
@stevemurr @claude @Oldes
Add memory safety tests for vec0 memory leak fixes
c4da947 
Add comprehensive test suite that exercises code paths fixed in the memory
leak fixes from PR asg017#258:

- vec0_free: Tests create/destroy cycles with partition, auxiliary, and
  metadata columns to verify proper cleanup of column names
- vec0_init: Tests error conditions (too many partition keys, too many
  metadata columns, invalid syntax) to verify cleanup on error paths
- vec0Update_Insert: Tests batch inserts with various column types
- vec0Update_Delete_ClearMetadata: Tests DELETE operations with metadata
- vec0Filter_knn: Tests KNN queries with metadata filters
- vec0_metadata_filter_text: Tests KNN queries with text filters
- vec0_write_metadata_value: Tests inserts with metadata values

The tests include stress tests for rapid create/insert/drop cycles and
mixed operations that would surface memory leaks when run under memory
checkers like valgrind or ASAN.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
(cherry picked from commit 4eeb2f2)
@josharian josharian mentioned this pull request Apr 5, 2026
Closed
vlasky added a commit to vlasky/sqlite-vec that referenced this pull request Aug 22, 2026
@vlasky
Fix memory leaks in vec0 operations and error paths (fixes #7)
5ae4fed 
Ports the fixes from upstream PR asg017#258 (see also
asg017#265), adapted to this fork, plus a few additional
leaks found while reviewing the same code paths:

- vec_slice: route malloc-failure branches through the cleanup label
  instead of returning early and leaking the parsed vector
- vec_eachFilter: free the error message from vector_from_value
- vec0_free: free partition, auxiliary, and metadata column names and
  metadata shadow-table names, previously leaked on every table close
- vec0_init: free the auxiliary shadow-table SQL string; free the vtab
  struct itself on the error path; assign the column counters at the
  error label so names copied into the vtab during a failed constructor
  parse are freed too
- vec0_metadata_filter_text: free the rowids buffer when the blob read
  fails
- vec0Filter_knn: free knn_data on all error paths, clean up
  metadata-IN arrays and their text entries on error, and free the
  top-k buffers when a step after a successful chunk scan fails
- vec0_write_metadata_value: free the two SQL strings; always close the
  metadata blob handle, including on error paths
- vec0Update_Insert, vec0Update_Delete_ClearMetadata,
  vec0Update_UpdateAuxColumn: free SQL strings after prepare; finalize
  the statement when a step fails
- vec_static_blob_entriesFilter: restructure with a cleanup label; the
  query vector and candidate/taken bitmaps previously leaked even on
  success, and everything leaked on error paths

Verified with macOS leaks(1) on a workload exercising inserts, updates,
deletes, KNN queries with IN constraints, metadata/partition/auxiliary
columns, and table create/drop: 17 leaks (1168 bytes) before, 0 after.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@L1l1thLY