Skip to content

Raise the LSQR penalty cap to 1e6 - #86

Merged
timholy merged 1 commit into
mainfrom
teh/lsqr-kcap
Oct 10, 2026
Merged

timholy merged 1 commit into
mainfrom
teh/lsqr-kcap

Conversation

@timholy

@timholy timholy commented Oct 10, 2026

Copy link
Copy Markdown
Member

On sparse nonsymmetric lognormal matrices cover_transversal was 1.5 to 29 times slower than cover_min, the ratio growing with size. Its multiplier iteration contracted by less than a factor of two per update once the penalty reached the LSQR cap of 1e5, stalled with a KKT residual of 1e-6 to 1e-4, and left the active-set finish tens to hundreds of steps.

The cause is the shape of the active set. At the minimizer the active constraints of the transversal-tight problem form one large tree: on a 10000x10000 matrix with six entries per row, 2792 rows with diameter 163 and Laplacian Fiedler value 2.6e-5, against trees of at most 41 nodes and Fiedler value 1.3e-2 for cover_min on the same matrix. The method of multipliers contracts at a rate near 1/(1 + κ s), with s bounded by that eigenvalue, so κ = 1e5 barely contracts while 1e6 gains a factor of two to four per update. Neither problem is degenerate: both have a clean split between active and slack constraints and one positive multiplier per active constraint.

With the cap at 1e6, cover_transversal certifies with 0 to 2 finish steps instead of 10 to 119 on four such matrices (three at n = 10000, one at n = 31623), in equal or shorter time; at n = 31623 the time drops from 14.2 s to 8.8 s. cover_min is unchanged within three solves. At 1e7 the iteration levels off near 1e-7, as does the exact dense path, so 1e6 is the usable ceiling in Float64.

The drain test moves to a banded matrix whose slack multipliers still need a penalty above the new cap. The cross-check of the symmetric finish against the asymmetric solve loosens to 1e-9: at the higher cap the asymmetric multiplier iteration can meet its own KKT tolerance and skip the finish, which leaves its scaled entries about 1e-10 from the certified minimizer.

Assisted-by: Claude Fable 5.1 noreply@anthropic.com

On sparse nonsymmetric lognormal matrices `cover_transversal` was 1.5 to
29 times slower than `cover_min`, the ratio growing with size. Its
multiplier iteration contracted by less than a factor of two per update
once the penalty reached the LSQR cap of 1e5, stalled with a KKT
residual of 1e-6 to 1e-4, and left the active-set finish tens to
hundreds of steps.

The cause is the shape of the active set. At the minimizer the active
constraints of the transversal-tight problem form one large tree:
on a 10000x10000 matrix with six entries per row, 2792 rows with
diameter 163 and Laplacian Fiedler value 2.6e-5, against trees of at
most 41 nodes and Fiedler value 1.3e-2 for `cover_min` on the same
matrix. The method of multipliers contracts at a rate near
1/(1 + κ s), with s bounded by that eigenvalue, so κ = 1e5 barely
contracts while 1e6 gains a factor of two to four per update. Neither
problem is degenerate: both have a clean split between active and slack
constraints and one positive multiplier per active constraint.

With the cap at 1e6, `cover_transversal` certifies with 0 to 2 finish
steps instead of 10 to 119 on four such matrices (three at n = 10000,
one at n = 31623), in equal or shorter time; at n = 31623 the time drops
from 14.2 s to 8.8 s. `cover_min` is unchanged within three solves. At
1e7 the iteration levels off near 1e-7, as does the exact dense path,
so 1e6 is the usable ceiling in Float64.

The drain test moves to a banded matrix whose slack multipliers still
need a penalty above the new cap. The cross-check of the symmetric
finish against the asymmetric solve loosens to 1e-9: at the higher cap
the asymmetric multiplier iteration can meet its own KKT tolerance and
skip the finish, which leaves its scaled entries about 1e-10 from the
certified minimizer.

Assisted-by: Claude Fable 5.1 <noreply@anthropic.com>
@codecov

codecov Bot commented Oct 10, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 98.79%. Comparing base (67fd0a8) to head (34fcc13).

Additional details and impacted files
@@           Coverage Diff           @@
##             main      #86   +/-   ##
=======================================
  Coverage   98.79%   98.79%           
=======================================
  Files          23       23           
  Lines        6286     6286           
=======================================
  Hits         6210     6210           
  Misses         76       76           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@timholy
timholy merged commit b80567c into main Oct 10, 2026
5 checks passed
@timholy
timholy deleted the teh/lsqr-kcap branch October 10, 2026 16:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant