NTN: warn and flag when sigma_v collapses to the boundary

[davidhbernstein]

Closes the actionable half of a defect found while scoping gap A56. Not reported externally.

The symptom

sfm(model_name = "NTN") can return lambda = 972468, six NA standard errors, and no message of any kind.

tHN has warned and set $thn_sigma_u_at_bound for the mirror case — sigma_u collapsing onto zero — since 1.2.0. The other component was never covered.

The mechanism: the fit is correct, the silence is the defect

On such a sample NTN has no interior maximum in lambda. The profile log-likelihood rises monotonically toward a finite limit as lambda grows without bound. Measured by continuation from the reported solution (rand = 2):

lambda	profile logLik
1e1	−190.3374
1e2	−188.5223
1e5	−188.1011
9.72e5 — what sfm() returns	−188.1003
1e9 … 1e15	−188.1001876 (flat to 7 dp)

So the boundary is the maximum likelihood estimate and running there is right. But the likelihood is flat in lambda at that point, so the Hessian is singular and no finite standard error for lambda exists — which is why all six come back NA. That now gets said, instead of being left for the user to infer from missing numbers.

A methodological note, since it changed my own conclusion: a fixed-start profile reports −209.13 at lambda = 1e6 where the truth is −188.10, because it fails to find sfm()'s own point. That reads as "the optimizer ran away" and is the opposite of the right answer. Taking the lambda = Inf limit analytically is also invalid — dropping the log Phi term unbalances the -n log Phi(mu/sigma) normalising constant, leaving something unbounded as sigma -> 0 (it ran to sigma = 1.5e-149). Continuation from the reported solution is what works, and needs no analytic limit.

Two things verified before writing any code

Scope. I scanned the models that report a lambda/sigma reparameterization rather than assuming this was general. Over 8 seeds, only NTN runs away (3 of 8). NHN shares the parameterization and never does; NE, NR, NU and NG report sigv/sigu directly. So this is model-specific, matching the existing thn_* and NGB2 warnings in the same file rather than inventing a package-wide check from one model's evidence.

The threshold is not tuned. The criterion is sigma_v/sigma < 1e-3, the same scale-free ratio thn_sigma_u_at_bound uses. Over 12 samples it separates the regimes by five orders of magnitude:

	sigma_v/sigma	standard errors
boundary fits (rand 2, 4, 5, 11, 12)	4.9e−08 … 1.3e−06	0 of 6
interior fits (rand 1, 3, 6–10)	0.126 … 0.442	6 of 6

The threshold falls in empty space. The second test block asserts that clearance explicitly, so a change which narrows the gap fails there rather than quietly making the flag sensitive to the cut.

Implementation note

The flag is appended to the results list rather than folded into it. NTN shares its results block with seven other models, and re-listing twelve values against twelve names to add one field is how a name/value misalignment gets introduced. $ntn_sigma_v_at_bound is always present for NTN and always logical, so callers need not test for its existence.

Evidence

• Fails on main, passes here — 12 failures against c94721a, 0 on this branch, 40 assertions. main was pinned with git archive c94721a rather than a worktree, after a worktree silently drifted to an older commit twice — which would have made "fails on main" a claim about the wrong tree.
• tools::checkDocFiles() clean with the new \value item; 50 Rd files parse. No \usage change.
• Full suite (installed build, NOT_CRAN=true): 0 failures, 5295 passing, 43 warnings, 8 skips, 39.5 min. The warning count is unchanged from main's, so the new warning fires in no existing test.
• The four plm::pdata.frame: already a pdata.frame errors documented as expected locally did not occur.
• CI on five platforms.

Deliberately not in this PR

The same scan found a second and more serious NTN failure: 3 of 12 fits stop 0.27–1.67 log-likelihood units below the supremum while reporting conv 0 and a full set of standard errors — confident output at a point that is not the MLE. That needs a multistart over lambda (the shape of fix #40 applied to NNAK), is a behavioural change rather than a diagnostic, and belongs in its own PR. Also untouched: NW, whose runaway is sigu -> 4e-5 with shape k -> 0.13, a different boundary that I have not profiled.

🤖 Generated with Claude Code

[davidhbernstein]

Updated for the NEWS.md conflict with #54, and for the comment-length rule.

Conflict resolved by keeping both entries with a blank line between, #54's merged entry first. Verified by content rather than by a clean merge: no conflict markers, each entry present exactly once, and both sides' code and tests actually in the tree — #54's bhhh_ok/bhhh_hint and test-vcov-bhhh-advice.R, and this branch's ntn_sigma_v_at_bound and test-ntn-sigma-v-bound.R.

Long comment moved out of R/. CLAUDE.md puts the long explanatory blocks in notes/code_history/ and keeps one or two lines in R/. I had written a 16-line block above the assignment — the profile table, the threshold separation, the measurement traps — which is what that rule exists to prevent. R/sfm.R now reads:

## sigma_v on its boundary: lambda -> Inf is the supremum, so the fit is
## right but flat and has no standard errors. notes/code_history/sfm.md.
ntn_sigma_v_at_bound <- isTRUE(sig_v / sqrt(sig_u^2 + sig_v^2) < 1e-3)

The full account is now a section in notes/code_history/sfm.md (outside the package, so it reaches neither the tarball nor this repo). Worth noting the pointer I first wrote was false — that detail lived in FUNCTIONALITY_GAPS.md, not in code-history — so the note had to be written rather than just referenced.

Evidence after the merge: full suite 0 failures, 5312 passing, 43 warnings, 8 skips, 39.1 min — 5255 from main plus 17 from #54's test and 40 from this one. test-ntn-sigma-v-bound.R re-run after the comment trim is still 40/40.

Two of my long blocks are already on main (data.processing.R 20 lines, sfm.R 9 lines, ~6 in sfareg_methods.R). Those get their own PR once this lands — mixing comment cleanup into a bug fix, or creating a self-conflict on sfm.R, both seemed worse than waiting.

---

Generated by Claude Code