[nathanhubens]

Release 0.5.0: changelog section dated and version bumped to 0.5.0. Export and clean are idempotent; nbdev-test passes locally.

Changelog

Breaking Changes

• PruneCallback and Pruner(schedule=Schedule) now honour the schedule's start_pct and end_pct. The levels handed to torch-pruning used to map step i of n straight into the schedule's shape function, so start_pct and end_pct were never read: one_shot (start_pct=0.5) pruned at the first step, and agp and iterative (start_pct=0.2) began ramping from the first step. The levels are now the schedule's own progress(i / n), built in one place (Pruner._to_tp_scheduler) for both entry points: one_shot prunes once, at the first step at or past start_pct; agp and iterative begin at their start_pct; the full ratio is reached at end_pct and held after it. A schedule with start_pct=0 and end_pct=1 (one_cycle, cos, lin, Schedule(f)) produces the same levels as before, level for level, and a plain torch-pruning scheduler callable is passed through unchanged — Migration: a Schedule with start_pct > 0, such as one_shot or agp, now waits for it; pass a schedule with start_pct=0 to keep the old timing

New Features

• Pruner(..., reg=1e-4) builds torch-pruning's GroupNormPruner instead of its MetaPruner, and the new Pruner.regularize(scale=1.) adds DepGraph's group penalty to the gradients already computed: every channel of a dependency group receives reg * gamma * w on each tensor it spans — the convolution's filter, the batch-norm weight and the input slices of the layers that read it. gamma runs from 1 on the group's most important channel to 2**alpha on its least important one (alpha=4 by default), measured on the square root of the pruner's own criteria score. It is called after backward and before the optimizer step, and scale multiplies the penalty for gradients that are themselves scaled (a mixed-precision loss scale, gradient accumulation). Only the groups the pruner will prune are penalized, so a per-layer dict leaves the layers it does not target alone; a group with a constant importance (a single channel among them) is skipped instead of producing nan gradients, with one warning if every group is; the groups are rebuilt after each prune_model(). Biases, frozen parameters and a depthwise convolution's own kernels are not penalized. reg and alpha are keyword-only and reg < 0 is refused. reg=0, the default, builds the same MetaPruner as before and prunes identically apart from the device fix below. PruneCallback refuses reg: use GroupRegularizeCallback
• GroupRegularizeCallback(pruner, schedule=None) applies a Pruner's group penalty during a fastai fit: before_step calls pruner.regularize(), after GradientAccumulation has skipped the batches that do not step and before MixedPrecision unscales the gradients, so the penalty is added once per optimizer step and multiplied by the loss scale of a torch.amp.GradScaler when one is active. A Schedule multiplies the penalty by its progress and is copied, so the module-level schedules other callbacks share are not advanced. Pruning stays a separate call after the fit: pruner.prune_model(), then learn.fit(n, reset_opt=True), because the prune replaces the parameters the optimizer holds. A Pruner with reg=0 is refused, and so are, at before_fit, a Pruner built on another model than learn.model, NonNativeMixedPrecision and a PruneCallback in the same fit
• RegularizeCallback(norm=2) penalizes the L2 norm of each group of the chosen granularity (group lasso) instead of every weight separately. The default, norm=1, is unchanged byte for byte and is an elementwise L1 whatever the granularity, since the groups partition the weights; groups are taken within one layer and do not follow cross-layer couplings such as the next layer's input channels or BatchNorm parameters

Bug Fixes

• Pruner.group_importance returns its scores on the device of the layers it scored instead of fastai's default device, which put them on the GPU for a model on the CPU whenever one was available. The ranking a prune step makes is unchanged: torch-pruning moves the scores to the CPU before ranking them
• PruneCallback logs, at the end of each epoch, the pruning ratio torch-pruning actually applied. torch-pruning's step() advances its step counter before reading the level, so after k steps the model is at level k; the log printed level k - 1, one step behind, and the last epoch did not print the final target
• PruneCallback hands the batch it takes from learn.dls.one_batch() to the Pruner as example_inputs, so the dependency graph is traced on the fit's own data. It was read and then dropped, and the Pruner traced on its default torch.randn(1, 3, 224, 224), which failed on a model that does not accept a 224x224 image. An example_inputs passed to the callback still wins

Dependencies

• torch-pruning>=1.5