Repository navigation
perf(eval): select percentile and median ranks instead of sorting the window - #619
Merged
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #594.
value_percentileandvalue_mediancorrelations sorted every finite value in the group's window on each event fed into the group, and chain bookkeeping deep-cloned every fired correlation result only to read its source index and group key.Changes
select_nth_unstable_by(f64::total_cmp)instead of sorting. An interpolated percentile takes its upper neighbour as the minimum of the right-hand partition, and an even-count median takes its lower middle value as the maximum of the left-hand partition, so results are bit-identical to the sorted computation.check_conditionandcurrent_valuenow sharewindow_percentileandwindow_median. Window state and snapshots are unchanged.chain_correlationscarries(source_idx, group_key)inpendinginstead of fullEvaluationResults, which removes the per-fire clone of every result (including retained events in full event mode) and the header clone for each chained parent.Benchmark
New
correlation_numeric_windowgroup inbenches/correlation.rs: one group held at a steady window size, 1,000 events per iteration, each checked against the full window.value_percentile, 1,000 valuesvalue_percentile, 5,000 valuesvalue_median, 1,000 valuesvalue_median, 5,000 valuesTests
test_percentile_selection_matches_sorted_referencecompares against a full-sort reference bit for bit, over window sizes 1 to 64 plus 99, 100, 1,000, and 5,001, at every integer rank and a few fractional ones. The samples include repeats and signed zeros.test_window_median_matches_sorted_referencedoes the same for odd and even medians through bothcheck_conditionandcurrent_value.test_chaining_in_full_event_modechecks that a child carrying retained events still drives its parent, which fires with the child's group key and no events.main.