Skip to content

feat(config): add arrayFunctionResultOverwritesData opt-in overwrite flag (HF-305)#1714

Open
marcin-kordas-hoc wants to merge 1 commit into
developfrom
feat/hf-305-overwrite-flag
Open

feat(config): add arrayFunctionResultOverwritesData opt-in overwrite flag (HF-305)#1714
marcin-kordas-hoc wants to merge 1 commit into
developfrom
feat/hf-305-overwrite-flag

Conversation

@marcin-kordas-hoc

@marcin-kordas-hoc marcin-kordas-hoc commented Jul 22, 2026

Copy link
Copy Markdown
Collaborator

What & why

Adds an opt-in boolean config option arrayFunctionResultOverwritesData (default false).

  • false (default): unchanged behavior — an array function spilling onto an occupied cell yields #SPILL! and leaves the occupant intact.
  • true: the spill overwrites the occupied cells (clears occupants, spills the array, reroutes dependents) instead of emitting #SPILL!.

Requested by a customer; the feature + option name were pre-agreed (~8 SP).

How

Reuses the existing spill-placement exchange primitive rather than inventing a mechanism:

  • ConfigParams / Config: new option, mirrored on useArrayArithmetic.
  • DependencyGraph gains config + two guarded early-returns (exchangeOrAddFormulaVertex, setAddressMappingForArrayVertex).
  • Evaluator: recompute no longer emits #SPILL! when overwrite is allowed.

Array-vs-array safety: a spill colliding with another array always keeps #SPILL! and leaves that array intact, even in overwrite mode (canOverwriteArrayResult / overwriteWouldHitArray). Matches Excel; overwrite only clears static data. Default false guarantees no silent data loss.

Tests

test/hf-305-overwrite.spec.ts (OFF parity → #SPILL! + occupant intact; ON overwrite + dependent reroute + recalc; array-vs-array stays #SPILL!). Full array/spill + undo-redo regression green.

Known limitations

The opt-in overwrite behavior is intentionally scoped to the direct spill/recalc path. The following edges are not fully handled yet; each is a nuance to weigh, not a silent-corruption risk (the default false config is fully backward compatible and untouched by all of these):

  • buildFromArray initial-build path: declaring an array formula and a conflicting occupant in the same initial buildFromArray call still lets the occupant win — overwrite only applies on the recalc / setCellContents path (existing data, then an array formula lands on it), not the un-gated initial-build placement path.
  • Undo does not restore cells overwritten via direct setCellContents (verified): undo() removes the array formula but the overwritten occupants are lost, not restored — they are not recorded on the undo stack. Documented in the ConfigParams JSDoc; a proper fix (recording cleared occupants in undo oldData) is follow-up work. Top item for reviewer to weigh given the feature is destructive.
  • Structural insert/delete (addColumns/addRows/removeColumns/removeRows) that grows an array over static data does not snapshot for undo either. When a structural op shifts a dependency so an existing array grows and overwrites a previously-static cell, the overwrite itself is now handled correctly for evaluation and for useColumnIndex (the stale value is dropped from the column index at the moment of overwrite — fixed during review, see rewriteAffectedArrays), but undoing that structural op does not restore the static cell's original value or its column-index entry. This is the same undo gap as the point above, on the structural-op path instead of the direct-edit path.
  • useColumnIndex + undo interaction: because of the two points above, any workflow that relies on undo() to fully roll back an overwrite (whether triggered directly or by a structural op) will see the array formula disappear but the previously-static data stay gone, and (if useColumnIndex is on) MATCH/VLOOKUP/HLOOKUP will correctly not find the old value post-undo, but will also not find it restored — i.e. the index and the data are consistent with each other post-undo, just not consistent with pre-overwrite state.

None of these limitations can produce the crash or stale-lookup-index bugs originally flagged by Bugbot on this PR — both were reproduced with throwaway probes and fixed at the root cause (see review thread replies on commit b877ed4). What remains out of scope is exclusively the undo leg of overwrite bookkeeping.


Note

High Risk
Opt-in destructive sheet mutation in core dependency graph, evaluation, undo, and column-index paths; mistakes could silently clear data or leave stale lookup indexes despite broad new tests.

Overview
Adds opt-in config arrayFunctionResultOverwritesData (default false). When true, array formulas that would #SPILL! onto non-empty cells instead claim the spill range, clear static occupants, and evaluate—array-vs-array collisions still #SPILL!.

Engine: DependencyGraph takes Config, exposes canOverwriteArrayResult / overwriteWouldHitArray, and extends array placement in exchangeOrAddFormulaVertex to allow overwrite and emit content changes for cleared cells (using non-throwing valueOrUndef for uncomputed formula occupants). Evaluator skips #SPILL! when overwrite is allowed.

Operations / undo: setCellContent returns { oldContent, overwrittenCells } via pre-spill snapshots; SetCellContentsUndoEntry stores overwritten cells and undo() restores them after the anchor formula is reverted. Column index stays consistent on overwrite, array expand/shrink restore, and selective rewriteAffectedArrays updates when structural ops grow arrays over static cells (useColumnIndex).

Tests: test/hf-305-overwrite.spec.ts covers default vs explicit flag, dependent reroute, undo/redo, array-array guard, batch uncomputed occupant, and column-index staleness (including addColumns growth).

Reviewed by Cursor Bugbot for commit 467c186. Bugbot is set up for automated code reviews on this repo. Configure here.

@qunabu

qunabu commented Jul 22, 2026

Copy link
Copy Markdown
Contributor

@netlify

netlify Bot commented Jul 22, 2026

Copy link
Copy Markdown

Deploy Preview for hyperformula-dev-docs ready!

Name Link
🔨 Latest commit 467c186
🔍 Latest deploy log https://app.netlify.com/projects/hyperformula-dev-docs/deploys/6a6602deeedd020008331aa9
😎 Deploy Preview https://deploy-preview-1714--hyperformula-dev-docs.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

@marcin-kordas-hoc
marcin-kordas-hoc force-pushed the feat/hf-305-overwrite-flag branch from 9009463 to b2da92e Compare July 22, 2026 16:18
@github-actions

github-actions Bot commented Jul 22, 2026

Copy link
Copy Markdown

Performance comparison of head (467c186) vs base (23a7437)

                                     testName |    base |    head | change
--------------------------------------------------------------------------
                                      Sheet A |  482.06 |  500.19 | +3.76%
                                      Sheet B |  153.19 |  159.11 | +3.86%
                                      Sheet T |  136.52 |  138.96 | +1.79%
                                Column ranges |  462.88 |  464.98 | +0.45%
                                Sorted lookup | 14334.2 | 14093.8 | -1.68%
Sheet A:  change value, add/remove row/column |   14.61 |   14.98 | +2.53%
 Sheet B: change value, add/remove row/column |  130.29 |  121.94 | -6.41%
                   Column ranges - add column |  151.39 |  155.63 | +2.80%
                Column ranges - without batch |  452.04 |  459.15 | +1.57%
                        Column ranges - batch |  114.79 |  116.09 | +1.13%

@marcin-kordas-hoc

Copy link
Copy Markdown
Collaborator Author

bugbot run

Comment thread src/DependencyGraph/DependencyGraph.ts
@marcin-kordas-hoc
marcin-kordas-hoc force-pushed the feat/hf-305-overwrite-flag branch 2 times, most recently from 4c02791 to d2a2671 Compare July 23, 2026 03:53
@marcin-kordas-hoc

Copy link
Copy Markdown
Collaborator Author

bugbot run

Comment thread src/Operations.ts
Comment thread src/DependencyGraph/DependencyGraph.ts Outdated
@marcin-kordas-hoc
marcin-kordas-hoc force-pushed the feat/hf-305-overwrite-flag branch from d2a2671 to 5e397d1 Compare July 24, 2026 11:14
@marcin-kordas-hoc

Copy link
Copy Markdown
Collaborator Author

bugbot run

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit 5e397d1. Configure here.

@marcin-kordas-hoc
marcin-kordas-hoc marked this pull request as ready for review July 25, 2026 10:58
@marcin-kordas-hoc
marcin-kordas-hoc force-pushed the feat/hf-305-overwrite-flag branch from 5e397d1 to 2f48fba Compare July 26, 2026 02:31
Comment thread src/UndoRedo.ts
@marcin-kordas-hoc
marcin-kordas-hoc force-pushed the feat/hf-305-overwrite-flag branch from 2f48fba to b877ed4 Compare July 26, 2026 02:43
@marcin-kordas-hoc

Copy link
Copy Markdown
Collaborator Author

bugbot run

Comment thread src/DependencyGraph/DependencyGraph.ts
Comment thread src/Operations.ts
…flag (HF-305)

Adds a public boolean Config option `arrayFunctionResultOverwritesData`
(default `false`). When `false`, behavior is unchanged: an array spill
onto an occupied cell yields `#SPILL!` and leaves the occupant intact.
When `true`, the spill overwrites the occupied cells (clears occupants,
spills the array, reroutes dependents) instead of emitting `#SPILL!`.

Implementation reuses the existing spill-placement exchange primitive:
- ConfigParams/Config: new option, mirrored on `useArrayArithmetic`.
- DependencyGraph gains `config` + two guarded early-returns in
  exchangeOrAddFormulaVertex / setAddressMappingForArrayVertex.
- Evaluator: recompute no longer emits `#SPILL!` when overwrite is allowed.

Array-vs-array safety: a spill colliding with ANOTHER array always keeps
`#SPILL!` and leaves that array intact, even in overwrite mode
(canOverwriteArrayResult / overwriteWouldHitArray). This matches Excel and
avoids corrupting a pre-existing array; overwrite only clears static data.

Default `false` guarantees no existing embedder loses data silently.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@marcin-kordas-hoc
marcin-kordas-hoc force-pushed the feat/hf-305-overwrite-flag branch from b877ed4 to 467c186 Compare July 26, 2026 12:51
@marcin-kordas-hoc

Copy link
Copy Markdown
Collaborator Author

bugbot run

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit 467c186. Configure here.

@codecov

codecov Bot commented Jul 26, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 96.72131% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 97.21%. Comparing base (23a7437) to head (467c186).

Files with missing lines Patch % Lines
src/DependencyGraph/DependencyGraph.ts 94.73% 1 Missing ⚠️
src/Operations.ts 96.96% 1 Missing ⚠️
Additional details and impacted files

Impacted file tree graph

@@             Coverage Diff             @@
##           develop    #1714      +/-   ##
===========================================
- Coverage    97.21%   97.21%   -0.01%     
===========================================
  Files          178      178              
  Lines        15611    15660      +49     
  Branches      3460     3441      -19     
===========================================
+ Hits         15177    15224      +47     
+ Misses         434      428       -6     
- Partials         0        8       +8     
Files with missing lines Coverage Δ
src/Config.ts 94.17% <100.00%> (+0.05%) ⬆️
src/CrudOperations.ts 99.20% <100.00%> (+<0.01%) ⬆️
src/Evaluator.ts 100.00% <100.00%> (ø)
src/UndoRedo.ts 100.00% <100.00%> (ø)
src/DependencyGraph/DependencyGraph.ts 98.70% <94.73%> (-0.11%) ⬇️
src/Operations.ts 98.81% <96.96%> (-0.13%) ⬇️

... and 4 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants