From 48e0eb461d3c3925bcb5aa781b7a52bf3f7c7dc4 Mon Sep 17 00:00:00 2001 From: nathanhubens Date: Fri, 25 Sep 2026 10:46:42 +0200 Subject: [PATCH] release: 0.5.0 --- CHANGELOG.md | 8 ++++---- fasterai/__init__.py | 2 +- 2 files changed, 5 insertions(+), 5 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 85a2ae71..8a7e1636 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,16 +2,16 @@ -## Unreleased +## 0.5.0 (2026-09-25) + +### Breaking Changes +- `PruneCallback` and `Pruner(schedule=Schedule)` now honour the schedule's `start_pct` and `end_pct`. The levels handed to torch-pruning used to map step `i` of `n` straight into the schedule's shape function, so `start_pct` and `end_pct` were never read: `one_shot` (`start_pct=0.5`) pruned at the first step, and `agp` and `iterative` (`start_pct=0.2`) began ramping from the first step. The levels are now the schedule's own `progress(i / n)`, built in one place (`Pruner._to_tp_scheduler`) for both entry points: `one_shot` prunes once, at the first step at or past `start_pct`; `agp` and `iterative` begin at their `start_pct`; the full ratio is reached at `end_pct` and held after it. A schedule with `start_pct=0` and `end_pct=1` (`one_cycle`, `cos`, `lin`, `Schedule(f)`) produces the same levels as before, level for level, and a plain torch-pruning scheduler callable is passed through unchanged — **Migration**: a `Schedule` with `start_pct > 0`, such as `one_shot` or `agp`, now waits for it; pass a schedule with `start_pct=0` to keep the old timing ### New Features - `Pruner(..., reg=1e-4)` builds torch-pruning's `GroupNormPruner` instead of its `MetaPruner`, and the new `Pruner.regularize(scale=1.)` adds DepGraph's group penalty to the gradients already computed: every channel of a dependency group receives `reg * gamma * w` on each tensor it spans — the convolution's filter, the batch-norm weight and the input slices of the layers that read it. `gamma` runs from `1` on the group's most important channel to `2**alpha` on its least important one (`alpha=4` by default), measured on the square root of the pruner's own `criteria` score. It is called after `backward` and before the optimizer step, and `scale` multiplies the penalty for gradients that are themselves scaled (a mixed-precision loss scale, gradient accumulation). Only the groups the pruner will prune are penalized, so a per-layer dict leaves the layers it does not target alone; a group with a constant importance (a single channel among them) is skipped instead of producing `nan` gradients, with one warning if every group is; the groups are rebuilt after each `prune_model()`. Biases, frozen parameters and a depthwise convolution's own kernels are not penalized. `reg` and `alpha` are keyword-only and `reg < 0` is refused. `reg=0`, the default, builds the same `MetaPruner` as before and prunes identically apart from the device fix below. `PruneCallback` refuses `reg`: use `GroupRegularizeCallback` - `GroupRegularizeCallback(pruner, schedule=None)` applies a `Pruner`'s group penalty during a fastai fit: `before_step` calls `pruner.regularize()`, after `GradientAccumulation` has skipped the batches that do not step and before `MixedPrecision` unscales the gradients, so the penalty is added once per optimizer step and multiplied by the loss scale of a `torch.amp.GradScaler` when one is active. A `Schedule` multiplies the penalty by its progress and is copied, so the module-level schedules other callbacks share are not advanced. Pruning stays a separate call after the fit: `pruner.prune_model()`, then `learn.fit(n, reset_opt=True)`, because the prune replaces the parameters the optimizer holds. A `Pruner` with `reg=0` is refused, and so are, at `before_fit`, a `Pruner` built on another model than `learn.model`, `NonNativeMixedPrecision` and a `PruneCallback` in the same fit - `RegularizeCallback(norm=2)` penalizes the L2 norm of each group of the chosen `granularity` (group lasso) instead of every weight separately. The default, `norm=1`, is unchanged byte for byte and is an elementwise L1 whatever the granularity, since the groups partition the weights; groups are taken within one layer and do not follow cross-layer couplings such as the next layer's input channels or BatchNorm parameters -### Breaking Changes -- `PruneCallback` and `Pruner(schedule=Schedule)` now honour the schedule's `start_pct` and `end_pct`. The levels handed to torch-pruning used to map step `i` of `n` straight into the schedule's shape function, so `start_pct` and `end_pct` were never read: `one_shot` (`start_pct=0.5`) pruned at the first step, and `agp` and `iterative` (`start_pct=0.2`) began ramping from the first step. The levels are now the schedule's own `progress(i / n)`, built in one place (`Pruner._to_tp_scheduler`) for both entry points: `one_shot` prunes once, at the first step at or past `start_pct`; `agp` and `iterative` begin at their `start_pct`; the full ratio is reached at `end_pct` and held after it. A schedule with `start_pct=0` and `end_pct=1` (`one_cycle`, `cos`, `lin`, `Schedule(f)`) produces the same levels as before, level for level, and a plain torch-pruning scheduler callable is passed through unchanged - ### Bug Fixes - `Pruner.group_importance` returns its scores on the device of the layers it scored instead of fastai's default device, which put them on the GPU for a model on the CPU whenever one was available. The ranking a prune step makes is unchanged: torch-pruning moves the scores to the CPU before ranking them - `PruneCallback` logs, at the end of each epoch, the pruning ratio torch-pruning actually applied. torch-pruning's `step()` advances its step counter before reading the level, so after `k` steps the model is at level `k`; the log printed level `k - 1`, one step behind, and the last epoch did not print the final target diff --git a/fasterai/__init__.py b/fasterai/__init__.py index 3d26edf7..3d187266 100644 --- a/fasterai/__init__.py +++ b/fasterai/__init__.py @@ -1 +1 @@ -__version__ = "0.4.1" +__version__ = "0.5.0"