Repository navigation
fix(prune)!: honour Schedule start_pct/end_pct in Pruner and PruneCallback - #98
Merged
Merged
Conversation
…back The levels handed to torch-pruning mapped step i of n straight into the schedule's shape function, so start_pct and end_pct were never read: one_shot pruned at the first step, agp and iterative ramped from the first step. Pruner._to_tp_scheduler now builds the levels from Schedule.progress(i / n) for both entry points; the two _build_pruning_schedule copies are removed. Schedules with start_pct=0 and end_pct=1 produce the same levels as before.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Breaking (targets 0.5.0).
PruneCallbackandPruner(schedule=Schedule)now follow the schedule'sstart_pctandend_pct.The bug
Two identical
_build_pruning_schedulefunctions, one inPrunerand one inPruneCallback, evaluatedsched_func(start_val, end_val, i/steps)from the first step and never readstart_pct/end_pct. As a result:one_shotdeclaresstart_pct=0.5but pruned at the first step of the fit;agpanditerativeramped from step 1 instead of from 20 %.The fix
Pruner._to_tp_scheduler(schedule)builds torch-pruning's levels fromcopy(schedule).progress(i/steps) * ratio. The shared module-level schedule objects are never mutated, and a plain callable scheduler passes through unchanged._build_pruning_schedulecopies are deleted.PruneCallbackpasses itsScheduleto thePrunerand reads its logged levels from the pruner. The per-layer dict path keepssparsity_levels == [].Effect
one_shotprunes at the first step at or past 50 % of training.agpanditerativebegin at theirstart_pct.end_pctends the ramp.start_pct=0, end_pct=1(one_cycle,cos,lin,dsd) are unchanged, level for level (golden test). No public notebook uses a non-zerostart_pct, so no stored output changes.Evidence
test_pruner,_levels(one_shot)gives[0.5]*9on master vs[0]*4+[0.5]*5expected;test_prune_callback, the channel trace gives[8]*6on master vs[16,16,8,8,8,8]expected.end_pctlevels;Schedule(sched_agp);reg > 0path getting the same levels;nbdev-testpasses in a clean clone with CI's nbdev 3.3.21, slow prune tests included. Clean and export leave the tree unchanged.Not included
PruneCallback.after_epochprints the level of the previous step: it readslevels[completed_steps - 1], while torch-pruning applieslevels[completed_steps]. That one-index fix comes in a small follow-up PR stacked on this one, because it requires re-executing theprune_callbacktutorial.