Netwise - Arsh update #5

@patelankeet2

Netwise — Arsh update #5

Follows #153. Sprint 4 closed yesterday. Nothing is outstanding on my side, and this time the queue is the whole update — fourteen pull requests are open, every one of them is green against current main, and thirteen of them merge together. What they are waiting on is reviews, and I cannot give them to my own.

main  735a4b9   391 tests · ruff clean
open  14 PRs (12 mine), 14 issues
merged this morning   #167 (Samika), #169 (Ankeet)

The queue, measured against M-1's own test

An approval later than the last commit. Nothing recalled, nothing inferred:

CURRENT      #130 #148 #149 #150 #151 #154 #162 #173
STALE        #146 #160
NO REVIEW    #165 #166 #168 #174

What I need, from whom, in one line each

PR	Who	What
#160	@patelankeet2 or @shubhamkataria2005	One re-approval. This is the blocker for eight others.
#146	@SamikaPerera	Your `CHANGES_REQUESTED` — both items now fixed
#174	@SamikaPerera	Your `CHANGES_REQUESTED` — coverage gap now closed
#165 #166 #168	anyone	First look, all small

#160 first if you only do one. It ratifies M-1 in the record, and the eight CURRENT PRs above are held behind it by my own undertaking, not by anything technical.

Its approval went stale because I pushed a correction after it — which is M-1 applying to the PR that ratifies M-1, and is the right outcome rather than an annoyance.

I ran the whole queue against current `main` rather than trusting old ticks

This is the check whose absence turned main red twice. Every green tick on these was earned against a main that no longer exists.

Each branch, merged onto 735a4b9 individually:

#130 391   #146 400   #148 391   #149 409   #150 391   #151 391   #154 391
#160 391   #162 397   #165 391   #166 391   #168 391   #173 416   #174 394

All thirteen non-conflicting ones merged together:

449 passed · ruff All checks passed

One real conflict, found before anyone hit it at the button: #174 collides with #162 in tests/test_preflight.py — both append to it. Order is #162, then #174 rebases. Clean against the other twelve.

The two that merged this morning, and why

#169 — @patelankeet2. Verified two ways rather than by reading the diff.

Coverage, because a narrow regex that matches nothing is a safeguard that never fires — swept every fixture, not the handy one:

rtr-us5-insecure  PC-001 PC-002 PC-003   FORBIDS    covered
rtr-us5-messy     PC-004 PC-005          REQUIRES   covered
                                         5/5

That number is what decided it. When I looked at #145 I measured that three of the five policy findings use the FORBIDS wording my own earlier fix did not touch — PC-001 was simply the first case @SamikaPerera happened to observe while rating for #90, not the only one. Ankeet's pattern covers both shapes. The half I left open is closed.

Mutation, inverting the branch that decides which side gets which verb: 4 failed, 387 passed. Restored: 391 passed.

#167 — @SamikaPerera. Two current approvals, docs only, and it carries the PC-001 inversion record that started #145.

Both were merged together off main first and tested as a combination before either went in.

#146 — three people asked, and I did not read the threads

@shubhamkataria2005 raised the duplicate-id gap on 16 August. @patelankeet2 repeated it on 17. @SamikaPerera made it a CHANGES_REQUESTED on 19. Fixed in 82051f8, using his exact demonstration:

['AC-001', 'PC-000', 'RT-001', 'PC-000', 'AC-002', 'RK-001']
duplicates: True

the seven tests that existed  ->  7 passed, 1 deselected
with the new test             ->  1 failed, 7 passed
whole suite                   ->  1 failed, 378 passed

Two PC-000, and nothing in 379 tests noticed — sitting inside the pull request whose stated justification is that this file has already drifted once without anyone noticing.

It asserts against pipeline.duplicate_id_findings(), the guard the product actually runs, rather than a re-implementation. Shubham's suggestion and the better half of it: #84 is why that distinction is not pedantic.

Also corrected: @SamikaPerera's v0.1.0 claim, flagged three times, re-derived from the tags rather than copied from his comment.

Why it took three asks: I did not read the review threads on my own open PRs for two days. Same failure that had me reporting "2 of 4" on #160 while the branch in front of me said four. Reading my own PRs is now first in my working order rather than last.

#174 — @SamikaPerera found a coverage gap and he was right

Quoting him: "Removing that fix entirely still passes all 386 tests here, because Docker is down on this machine."

Reproduced by stopping the container:

batfish up    fix removed -> 1 failed     caught
batfish down  fix removed -> 12 passed    NO coverage

CI has no Docker and no Batfish — so on CI, the one place it most needed to hold, the test was a no-op that read as coverage. I missed it because Batfish was running every time I ran the mutation, having written "run the mutation under more than one environment" into my own working standard the day before.

Fixed with a test that reaches the mechanism directly and needs no Batfish at all. Now:

batfish up    fix removed -> 2 failed
batfish down  fix removed -> 1 failed, 1 skipped

His cost objection is addressed too: skipping before running the subprocesses rather than after cut the batfish-down case by about 17 seconds, which is CI's case and was the expensive one returning nothing.

#93 — measured, and its own precondition is wrong

It says "between sprints, not during one". Sprint 4 ended yesterday, so by that test today qualifies. The queue disagrees:

ruff format would rewrite            31 files
files also touched by an open PR      3   (#146 #162 #174)

An empty queue is the real precondition, not the calendar. Full measurement, including the rule-set sizes (E,W,F = 98 findings, E,W,F,I,B,UP,SIM = 325) and a proposal to split it in two, is on #93.

I also posted a correction there: my first run reported zero collisions, which was a bug in my measuring script — lstrip("ab/") strips characters, so a/ai/explain.py became i/explain.py and no filename matched. "Zero collisions, safe to proceed" came within one paste of being posted as fact.

What is not mine and is waiting

Decision request: the five policy-format questions — five letters, one sitting (#87) #159 — @SamikaPerera @patelankeet2 @shubhamkataria2005, the five policy-format letters. Policy loader: the refusals first, to the five answers on #159 (#87) #173 implements the refusal paths and is approved and current; the format itself still needs the sitting.
The converter's limits are restated in three documents instead of read from one place #155 — @patelankeet2, the converter's limits restated in three documents. Correct the PF Sense limits — stale in two of three places, and it reached a status update #154 corrects two of them, which treats the symptom; The converter's limits are restated in three documents instead of read from one place #155 is the cure.
Explanations are regenerated on every /api/findings call #92 — unassigned. Explanations regenerate on every /api/findings call.

Everything on my side is committed, pushed, integration-tested and boarded. The only thing standing between fourteen green pull requests and main is a review, and #160 is the one that unlocks the rest.

@SamikaPerera

Update since posting — one more PR, and two issues measured. Keeping this thread accurate rather than letting it go stale the way everything else in this project does when a fact lives in two places.

#176 — @SamikaPerera's #172 fixed the file, not the property

Found by trying to run tools/stranger_config.py while re-measuring the #87 gap for the report. It died before measuring anything:

ModuleNotFoundError: No module named 'analysis'

Same bug #172 fixed in preflight.py, still live in a second file for as long as it had been fixed in the first:

script                as module   as script
preflight.py          ok          ok                      (fixed by #172)
stranger_config.py    ok          IMPORT FAILS
pfsense_shape.py      --          --   (imports no package, unaffected)

tests/test_tools_invocation.py now enumerates tools/*.py rather than naming a script, so the next tool added is covered on the day it lands. Mutation caught in both Batfish states. 397 passed, 2 skipped.

Worth saying plainly: the tool that measures the largest gap in the product could not be run one of its two documented ways, and nobody noticed.

#92 measured — and the bigger cost is not the one the issue names

hit 1: 5 found  ->  5 model calls
hit 2: 5 found  ->  5 model calls
hit 3: 5 found  ->  5 model calls

But /api/findings also re-runs the whole Batfish pipeline, 3.61s median, every hit — and that costs the same whether or not Ollama is installed, which on CI and on every demo machine makes it the entire observable cost. loadFindings() runs at script load (app.js:559), so every page load pays it, not just an explicit rescan.

The issue's cache-key warning is now demonstrated rather than asserted: AC-001 and AC-002 are used by both rtr-us5-insecure and rtr-us5-messy, so an id-keyed cache would show one network's explanation on the other's finding.

I also threw away a number I had already written up — "the fallback costs 2s per card" turned out to be this machine refusing any closed loopback port in 2.036s, including ports nothing has ever listened on. Details on #92.

@patelankeet2 @SamikaPerera it sits across both your layers; I measured it rather than took it.

#93 measured

31 files reformatted, colliding with 3 open PRs. An empty queue is its real precondition, not the calendar. Full numbers on #93, including a correction to my own first run.

Queue, unchanged where it matters

CURRENT      #130 #148 #149 #150 #151 #154 #162 #173
STALE        #146 #160
NO REVIEW    #165 #166 #168 #174 #176

#160 is still the one that releases eight of them. #146's two outstanding items are both fixed now and it needs @SamikaPerera's re-look.

@SamikaPerera

Two days to the client demo, and the queue has turned over completely since
this thread was written. Short update, one ask.

main  5c00714   825 passed, 6 skipped   CI green
merged 28 Aug   #246 #251 #181 #233 #253   (702 -> 825 tests)

That merge means the demo now shows things it could not on the 20th: a user's
own policy actually changing the findings (#181), a report you can download
(#233), and propose-a-change on screen (#253). Ollama is also installed and
working on the demo machine — findings render real model-written
explanations rather than deterministic fallback text.

The one ask: #260, before Thursday

@SamikaPerera @patelankeet2 @shubhamkataria2005 — one of you, not all three.

It is the only open PR that changes something the client will read during the
demo. Rehearsing beat 6 in a real browser yesterday:

upload a policy   "... staged. Not yet applied -- wiring lands with #181."
click Scan Now     6 problems found  ->  4 problems, 1 checked clean

The message denies the policy is applied. The next click applies it. #181
landed that morning; the sentence did not move.

Two tests were holding the false sentence in place — and the frontend one
asserted a string that lived in its own fixture, so it compared the harness to
itself. That is the part worth a second pair of eyes, more than the message.

I have deliberately NOT asked for review on my other two (#259 docs, #248
.gitignore). They can wait until after Thursday. If you have time for one
thing from me this week, make it #260.

What I have done rather than asked for

Per #249, where I measured that I author 61% of PRs and give 24% of reviews:

Multi-device fixture, and the scale figure #218 asked for (closes #218) #257 approved (@shubhamkataria2005). Reproduced both scale tables
independently — coverage matches exactly at every size, and I confirmed the
numbers survive Wire the policy loader to a check, so #87's gap actually moves #181, which changed the very line that computes covered.
Three mutations, three caught.
Phase 1.2, phase A: link explanations to the evidence they came from #255 changes requested (@patelankeet2). The provenance design is right
and survived three mutations. The problem is an integration one neither PR
could see alone: he added a 5th parameter to renderFinding(), and
renderImpact() from the merged Dashboard UI for propose-a-change (#232): the full response, with pushback that cannot be missed #253 still calls it with four, so the
propose pane renders id="undefined" twice and two links to #undefined.
deps: bump the runtime group across 1 directory with 2 updates #204 changes requested (dependabot). The pybatfish bump produces
identical results on three fixtures, but the pin's contract is that client
and container versions MATCH, and the container is still on 2025.07.07.
Pull the image first, then bump.

Two things needing a decision, not a review

Business-context risk scoring: escalate severity by asset criticality (#237) #254 (@SamikaPerera) is approved and current but now CONFLICTING —
Findings that can leave the screen: HTML and CSV export #233 and Dashboard UI for propose-a-change (#232): the full response, with pushback that cannot be missed #253 landed under it. All three conflicts are additive, not
semantic; details are on the PR. It was clean against the main it was
approved on, so nothing went wrong on your side.
preflight reports Ollama OK when the netwise-warden model does not exist #234 — preflight reports Ollama OK when netwise-warden does not exist.
I proposed defer until after Thursday, with the reason on the issue:
the demo machine autostarts Ollama and the model persists on disk, so it
cannot reach the broken state. Object by Wednesday midday or that is the
decision.

One more thing, which I am not fixing today

CONTRIBUTING.md (§6, §744) and .github/CODEOWNERS both still say the repo
is private and that branch protection needs GitHub Pro. Measured:

private = false     visibility = public

#246 corrected that claim in CLAUDE.md and #251 in M-2. Two files were
missed — one fact in four places, two updated. It feeds directly into #245,
which is still an open decision, so I would rather it be settled there than
patched quietly by me now.

@patelankeet2

Since this morning: six more merged, three reviews given, my queue is empty.

main   00daec2   987 passed, 8 skipped   ruff clean   CI green

Merged

#202 #259 #248 #260 #261 — plus #254 and #257 from your side.
Twelve since the 28th, taking main from 702 to 987 tests. I have zero open
PRs, which is the first time that has been true this month.

Thank you all three for turning #260 around — it was the only thing that
changed what the client reads on Thursday, and it went from unreviewed to
merged in a few hours once I asked for one of you by name rather than all
three. That is worth remembering as the pattern that works.

Reviews given

PF Sense: skip NAT-generated filter rules rather than converting them blindly #265 approved (@patelankeet2). The state I came to attack — every rule on
an interface being NAT-generated, leaving an ACL that fails open against
PF Sense's fail-closed default — you had already handled with a named
refusal. Three mutations, three caught. Then end to end: the NAT rule's
translated addresses appear nowhere in the output, and what survives parses
cleanly in real Batfish.
Phase A: deterministic remediation text for known evidence shapes #263 approved (@patelankeet2). Silent on error, none, unrecognised shape
and empty evidence — checked all four before approving, because remediation
is prescriptive in a way explain() is not. One non-blocking note:
remediate_with_source() has no status guard of its own, so the safety sits
entirely at one caller. And your conflict resolution has an ordering decision
in it — remediation ends up outside business-context escalation.
Report export UI: the download control for #222 #262 changes requested (@SamikaPerera). Details on the PR. Short version:
the conflict is your own Business-context risk scoring: escalate severity by asset criticality (#237) #254 moving under you, and a sixth harness now
exists that Report export UI: the download control for #222 #262 does not know about. Eleven lines fixes it —
988 passed / 25 errors becomes 1013 passed / 0 errors. Do not resolve
the conflict by concatenating both sides; I tried, and it left one unclosed
because they share a trailing brace pair.

Two decisions still open, both with deadlines

A check with no rules to run is tallied as "checked, nothing found" while its own text says nothing was checked #266 — a check with no rules to run is tallied as "checked, nothing
found" while its own card says nothing was checked. Proposed option 1, fix
after Thursday. Deadline Wednesday midday.
preflight reports Ollama OK when the netwise-warden model does not exist #234 — preflight reports Ollama OK with no model built. Proposed defer,
same deadline.

Neither has had a reply. If nobody objects they are decided as proposed.

One correction to something I said this morning

I told you #254 was conflicting. @SamikaPerera rebased it and it merged at
09:40 — while I was mid-batch, which moved main under a combination I had
already tested. I re-tested #260 and #261 on the new main before merging,
because #254 touches web/main.py and so does #260. Worth flagging as the
thing to watch: a batch tested at one commit is not tested at the next.

Filed

#266, #267 (a generated document that went stale within hours of landing,
and why the CI check we proposed for it would be permanently red), #258.
All unassigned, none urgent before Thursday.

@patelankeet2

Sprint 5 cannot close this week. Proposing we close it at what it achieved
and move the remainder, rather than let it drift a third week.

Object by Wednesday midday; silence is agreement. Same pattern as #234 and
#266.

Measured, not felt

Sprint 3   8/8    100%   closed
Sprint 4   27/27  100%   closed
Sprint 5   34/43   79%   OPEN, due 26 August -- overdue by 5 days
Sprint 6    9/20   45%   open, no due date

Nine issues remain in Sprint 5. Every one has been idle 4-7 days, and the
only movement this week came from work that was not in the sprint:

#78   @patelankeet2        idle 7d   PR #206, changes-requested, 2 commits since
#195  @shubhamkataria2005  idle 6d   PR #268 -- approved, will close on merge
#191  @patelankeet2        idle 7d   no PR
#227  @ARSH871-bot         idle 4d   no PR
#80   unassigned           idle 7d   blocked: client
#87   unassigned           idle 7d   the largest gap in the product
#92   unassigned           idle 7d
#93   unassigned           idle 7d   chore
#185  unassigned           idle 7d

Five of nine are unassigned. An unassigned issue in an overdue sprint is
not late work; it is work nobody started.

What can realistically close

The policy format has no example a user can copy, and README says it does not exist #195 — Ship a policy example a user can actually copy (closes #195) #268 is approved and its commit carries Closes #195, so it
closes on merge. That is one.
The client's real export: zero quick rules, multi-interface, and 1998 elements #78 — Add tools/live_check.py -- the manually-triggered live check #206 is mine to re-review; @patelankeet2 has pushed twice since I
blocked it. Possible, not certain.

That is 34/43 → 36/43 at best. Seven would still be open on a sprint that
ended five days ago.

The proposal

Close Sprint 5 once Ship a policy example a user can actually copy (closes #195) #268 lands, at whatever it reaches. A sprint that
closes at 84% honestly is worth more than one held open until it reads
100%, and Sprints 3 and 4 both closed at 100% because they were scoped to
what fitted.
Move A rule with no <protocol> silently widens to 'ip'; a rule with no <type> refuses #80, Explanations are regenerated on every /api/findings call #92, Widen the lint rules and run ruff format — between sprints, not during one #93, policy.py: the did-you-mean in _suggest() can never fire #185, Convert policy module-level state (D, #181) to an explicit run(bf, policy) argument #191, verify_docs.py cannot catch a stale contrib.json -- it reads the same file the documents were built from #227 to Sprint 6. Six items. Sprint 6
goes from 11 open to 17, which is a real cost and the reason for point 3.
Take The policy is ours, not the user's — there is no way to state your own #87 out of sprint planning entirely. It has now been carried
through two sprints without being started, because it is not a sprint-sized
item — it is "two of three checks ignore a user policy", and finishing it
means rewiring access_control and routing. Putting it in a third sprint
it cannot finish is exactly how it has drifted this far. It belongs in
Phase 1, alongside the capabilities it actually blocks.

And the part of this that is mine

Sprint 5 was mine to run. Two things I would do differently:

Five unassigned items should never have entered a sprint. docs/work-distribution.md says an issue without an assignee "is a note, not an assignment", and I wrote that sentence.
No sprint record exists for Sprint 5 or 6. docs/sprint1/ through sprint4/ each have one, written inside the sprint. Sprint 3's entry boasts that it was written the day after rather than reconstructed later, and that practice has now lapsed twice. If we close Sprint 5 this week I will write its record before closing it, not after.

@patelankeet2 @shubhamkataria2005 @SamikaPerera — object to any of the three
points and it changes. Silence by Wednesday midday and I will re-milestone
accordingly.

Superseded by #274 — Netwise Arsh update #6, posted just now.

Eleven days of work since this thread, and the queue has turned over completely: 62 PRs merged, the AI's second direction (#13/#14) landed, a user's own policy now reaches a check (#181), and the client's own vendor produces a real finding for the first time (#216).

Leaving this open rather than closing it — the decisions still running have their deadlines here, and moving them would break the links people already have.

Wednesday-midday deadline has passed with no objections visible. Executed the Sprint 5 proposal recorded here and in #273: moved #80, #92, #93, #185, #191, and #227 to Sprint 6; removed #87 from sprint planning; closed the Sprint 5 milestone. #78 remains as the one open issue still attached to the now-closed milestone, matching the proposal rather than silently moving the blocked client item.

Navigation Menu

Repository navigation

Netwise - Arsh update #5 #175

Netwise — Arsh update #5

The queue, measured against M-1's own test

What I need, from whom, in one line each

I ran the whole queue against current `main` rather than trusting old ticks

The two that merged this morning, and why

#146 — three people asked, and I did not read the threads

#174 — @SamikaPerera found a coverage gap and he was right

#93 — measured, and its own precondition is wrong

What is not mine and is waiting

#176 — @SamikaPerera's #172 fixed the file, not the property

#92 measured — and the bigger cost is not the one the issue names

#93 measured

Queue, unchanged where it matters

The one ask: #260, before Thursday

What I have done rather than asked for

Two things needing a decision, not a review

One more thing, which I am not fixing today

Merged

Reviews given

Two decisions still open, both with deadlines

One correction to something I said this morning

Filed

Measured, not felt

What can realistically close

The proposal

And the part of this that is mine

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Netwise - Arsh update #5 #175

Description

Netwise — Arsh update #5

The queue, measured against M-1's own test

What I need, from whom, in one line each

I ran the whole queue against current main rather than trusting old ticks

The two that merged this morning, and why

#146 — three people asked, and I did not read the threads

#174 — @SamikaPerera found a coverage gap and he was right

#93 — measured, and its own precondition is wrong

What is not mine and is waiting

Activity

ARSH871-bot commented on Aug 20, 2026

#176 — @SamikaPerera's #172 fixed the file, not the property

#92 measured — and the bigger cost is not the one the issue names

#93 measured

Queue, unchanged where it matters

ARSH871-bot commented on Aug 28, 2026

The one ask: #260, before Thursday

What I have done rather than asked for

Two things needing a decision, not a review

One more thing, which I am not fixing today

ARSH871-bot commented on Aug 31, 2026

Merged

Reviews given

Two decisions still open, both with deadlines

One correction to something I said this morning

Filed

ARSH871-bot commented on Aug 31, 2026

Measured, not felt

What can realistically close

The proposal

And the part of this that is mine

ARSH871-bot commented on Aug 31, 2026

ARSH871-bot commented on Aug 31, 2026

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions

I ran the whole queue against current `main` rather than trusting old ticks