Skip to content

The client's real export: zero quick rules, multi-interface, and 1998 elements #78

Open

Description

@ARSH871-bot

The quick question is answered, and it is the answer that reorders the sprint. Senaka provided an anonymised export on 10 August.

No client data appears in this issue, and none entered any AI context. Everything below comes from tools/pfsense_shape.py (#74), which prints element names and counts and never a value, plus one deliberately redacted error message.

The answer

filter rules              : 7
  marked <quick/>         : 0
  NOT marked quick        : 7
  => NO rules are quick -> last-match-wins applies.

Zero of seven. pfSense evaluates last-match-wins; a Cisco ACL — and so our converter — is first-match-wins. Nothing in his rule set short-circuits that difference. Issue #47's risk is no longer theoretical for this network.

It refuses before it gets there

PfSenseConversionError: filter rules span multiple interfaces [4];
                        only a single-interface rule set is supported

So rule order is not even the first blocker. Three interfaces, rules across four interface values. The converter handles one.

Scale

Our fixture His export
elements 54 1,998
filter rules 3 7
interfaces 2 3
top-level sections 4 36

Sections present that we do not handle: nat, openvpn, ipsec, aliases,
dhcpd, shaper — plus captiveportal, vlans, gateways, staticroutes.

Fourteen rule fields outside our vocabulary, including associated-rule-id
on two rules. Those are NAT-generated and cannot be understood without the NAT
section we deliberately skip.

Two anomalies to ask him about rather than guess

  • Only 5 of 7 rules carry <type>. convert() requires it.
  • Only 3 of 7 carry <protocol>.

Either pfSense omits defaults, or the anonymisation dropped them. Do not
assume
— this is the field whose absence would silently change what a rule
means, and guessing is the failure this converter exists to avoid.

What analysing his firewall would now take, in order

  1. Multi-interface rule sets — several ACLs, each bound to its own
    interface, rather than one acl_in.
  2. Real pfSense evaluation order — last-match-wins, or a defensible
    per-pair refusal. PF Sense conversion assumes first-match-wins; real exports may be last-match-wins #47's _check_rule_order_is_unambiguous() currently
    refuses; with zero quick rules it would refuse on every overlapping pair.
  3. A decision on NAT — associated-rule-id rules are meaningless without
    it, and NAT is explicitly out of scope today.
  4. Rules with omitted type / protocol — after asking, not after
    guessing.

Why this is bigger than "a piece of work nobody has scoped"

CLAUDE.md §7 said the remaining question was a client question, not a code
one, and that teaching the converter real pfSense ordering "becomes a piece of
work nobody has scoped."
That was true and it understated it. Rule ordering is
item 2 of four, and item 1 blocks item 2.

This is Sprint 4, and plausibly more than one sprint. Better to know now.

What it does not change

The refusals are working exactly as designed. The converter did not produce a
plausible, wrong ACL for a real firewall — it stopped and said which construct
it could not handle. That is #53, #54 and #58 doing their job on the first real
file they have ever seen, which is the strongest evidence yet that refusing
rather than guessing was the right call.

Activity

  1. ARSH871-bot commented on Aug 10, 2026

    @ARSH871-bot
    OwnerAuthor

    Followed up on the two field-omission anomalies with synthetic inputs — no client data read.

    no <type>       -> refused cleanly:  "unsupported rule <type>: None"
    no <protocol>   -> CONVERTED:        permit ip any any
    

    The missing-type case is fine. The missing-protocol case silently defaults to any -> Cisco ip, the widest possible match — and four of the seven rules in the export have no <protocol>, so it is most of his rule set.

    Filed separately as #80. I am deliberately not claiming pfSense is wrong to treat an absent protocol as "any" — it may well be. The narrower problem survives either answer: the converter cannot distinguish "pfSense omitted a default" from "the field was stripped", and it resolves that ambiguity silently, in the permissive direction, which is the one place this module does not refuse.

    That also sharpens question 1 to Senaka. It is no longer a curiosity about anonymisation — the answer decides whether four of his rules convert correctly or become wide-open ones.

  2. added
    layer: analysisLayer 1 — Batfish orchestration, checks, the pipeline
    blocked: clientWaiting on an answer from Senaka
    safetyTouches F-4, grounding, or the refuse-rather-than-guess boundary. Review with care
    on Aug 12, 2026
  3. ARSH871-bot commented on Aug 12, 2026

    @ARSH871-bot
    OwnerAuthor

    Sprint 4 (13–19 Aug): item 1 ONLY — multi-interface rule sets. Assigned @patelankeet2.

    This is timeboxed with an explicit stop. Do not start items 2–4; when item 1 lands we decide from what you learned whether the rest is a sprint or a term.

    Two reasons, and the second is the one that settles it:

    1. @shubhamkataria2005 on Sprint 4 planning proposal — the client's file already decided the shape #86: "Committing to all four before item 1 lands is estimating work we haven't scoped."
    2. The original brief told us to timebox exactly this, and the constraint was deleted rather than overridden. CLAUDE.md §7 as first written (88dbdfb): "This is a known hard problem — timebox it and fall back to supported-vendor sample configs if it stalls." It vanished in b2456fd, a documentation PR of mine, while writing about the thing it constrained.

    Item 1 is the real blocker: his rules span four interface values across three interfaces and the converter supports one. Everything else in this issue sits behind it.

  4. added this to the Sprint 4 milestone on Aug 12, 2026
  5. ARSH871-bot commented on Aug 12, 2026

    @ARSH871-bot
    OwnerAuthor

    Measured: item 1 alone does not unblock this file

    I raised the DHCP question reviewing #104 and could not answer it from the code. The shape tool can now (#107), and the answer is no. Structure only, never a value:

    wan    addressing=dhcp                               rules=yes  <-- REFUSED
    lan    addressing=<static address present, redacted>  rules=yes
    opt1   addressing=<static address present, redacted>  rules=no   -> deny-all
    => STILL REFUSES after #78 item 1. Rules name ['wan', 'WireGuard', 'openvpn'],
       which the converter cannot model.
    

    Three blockers remain after multi-interface support lands, and none of them is rule ordering:

    why it refuses
    DHCP WAN carrying rules _parse_interfaces() skips interfaces with no static <ipaddr>; rules naming one then hit "no static address configured"
    rules naming WireGuard not present in <interfaces> at all
    rules naming openvpn same

    What this changes about this issue

    The original ordering was: multi-interface → real evaluation order → NAT decision → missing fields. That ordering was written before we knew which interfaces carried rules. Two of the three blockers above are not on it.

    Worth re-scoping before item 2 starts, @patelankeet2 — specifically:

    • Is a DHCP interface modellable at all? A Cisco ACL needs an address to express rules against. The honest answer may be a narrower refusal — "this interface is DHCP; its rules cannot be modelled" — rather than support. That would still be progress, because it names one interface instead of refusing the whole file.
    • openvpn and WireGuard rules are tunnel policy, not LAN filtering. They may be out of scope entirely rather than work, and saying so explicitly is a legitimate outcome.

    The framing that still holds

    The converter refuses this file for four distinct reasons now rather than one, and every refusal names the construct. That remains the right behaviour and it is the evidence #53/#54/#58 were correct.

    But it means "we can analyse the client's firewall" is further away than the item list implied, and the record should say so rather than let item 1 landing read as the goal being met.

    Timebox still applies — item 1 with an explicit stop, then re-decide with this in hand.

  6. 10 remaining items

  7. added this to the Sprint 5 milestone on Aug 23, 2026
  8. added a commit that references this issue on Aug 23, 2026
  9. modified the milestones: Sprint 5, Sprint 6 on Sep 1, 2026
  10. ARSH871-bot commented on Sep 1, 2026

    @ARSH871-bot
    OwnerAuthor

    Moved to Sprint 6.

    This was the single issue left attached to Sprint 5 after that milestone closed, which is a state worth not leaving: an open issue on a closed milestone reads as done in a burndown and is not.

    The other six carried-over items (#80, #92, #93, #185, #191, #227) went to Sprint 6 when Sprint 5 closed; this one was called out as an exception and then left where it was. Now consistent with them.

    Nothing about the work changes. #206 is still open against it and still has my changes-requested on it.

  11. ARSH871-bot commented on Sep 3, 2026

    @ARSH871-bot
    OwnerAuthor

    The four open questions for Senaka are now written out in one place, in his language rather than ours: #294

    They are the same four already recorded here and in CLAUDE.md §7 — an address for the DHCP WAN that carries rules, a decision on the two undeclared VPN interfaces, NAT, and whether the missing <type>/<protocol> fields are real or an artefact of anonymising. Nothing new, just somewhere sendable.

    @patelankeet2 you own this — please correct anything I have got wrong before it goes to him. When he answers, the answer belongs in docs/design/pfsense-nat-support.md and here; the discussion just gets a link to it, so this issue stays the record.

  12. ARSH871-bot commented on Sep 3, 2026

    @ARSH871-bot
    OwnerAuthor

    @patelankeet2 — this is the one thing I need from you before Thursday, and it is the only discussion of the four I posted that needs anyone this week.

    The four questions for Senaka are written out in Discussion #294 in his language rather than ours. You own this issue, so please read them and correct anything I have got wrong or left out — particularly question 4, where I have said that our converter treats an absent <protocol> as any and emits ip. I want to be sure that is stated as our behaviour and not as a claim about what pfSense means.

    Once you have checked it, it can go to him. Nothing else on my four discussions needs you this week: #292 is a rule that needs no reply, #293 is moot because the repo is private again so branch protection is unavailable, and #296 is a backlog that explicitly should not start before the 10th.

    Also, separately and much smaller: A-3 is waiting on you and me — Shubham and Samika have both signed, our two rows are blank. #238 is blocked on it.

  13. patelankeet2 commented on Sep 5, 2026

    @patelankeet2
    Collaborator

    Picked up the rest of this without waiting for Senaka's answers to the four questions in Discussion #294 -- a decision genuinely blocked on him stays blocked, but nothing else needed to sit idle in the meantime.

    Item 4 (missing type/protocol) is now split cleanly, and its non-semantic half is fixed. PR #302: a single rule anywhere that fails to convert (missing/unsupported <type>, unsupported <protocol> value, an unresolved alias, a bad port, an unresolvable network reference) used to abort the ENTIRE file, not just that rule. At the client's real scale (1,998 elements against our 54-element fixture) that meant one anomaly anywhere meant zero analysis of anything. Fixed the same way #104 fixed the interface-level version of this: exclude and name the one rule, convert everything else normally. This is not a guess about what any field means -- every refusal still names the same cause it always did, only the blast radius shrank.

    What did NOT move: <protocol>'s default-to-any behaviour is completely untouched -- that's #80's own semantic question, and #80 says explicitly not to fix it before Senaka answers. This PR doesn't touch it.

    Item 3, NAT, checked and reaffirmed, not touched. Went through docs/design/pfsense-nat-support.md's own analysis again against today's "proceed without the client" instruction specifically, and it doesn't clear the bar the blast-radius fix did: there's no version of "convert the NAT rule" that doesn't invent translation semantics never observed in any real or synthetic <nat> shape. Detection-and-exclusion (#264) stays exactly as it is. Recorded in the design doc that this was revisited today, not left stale.

    One more small idea, not built: disclosing an inferred <protocol> via a comment (#80's own preferred Option C -- no behaviour change, just makes the existing silent default visible) was raised as a question for the team rather than decided solo, since it touches #80 directly. Still open.

    Items 1 and 2 (DHCP WAN, undeclared VPN interfaces) were already fully resolved before this, via #104/#190.

    So: #78 stays open, correctly -- NAT is the one thing left that's still genuinely a decision, not a task.

  14. added
    phase: 3Market readiness -- what stands between us and a product someone could buy.
    on Sep 10, 2026
  15. added a commit that references this issue on Sep 28, 2026
  16. ARSH871-bot commented on Sep 28, 2026

    @ARSH871-bot
    OwnerAuthor

    Reopened: auto-closed by merging #302, which fixed only item 4 (one bad rule no longer sinks the whole file). The export itself has not been run end to end yet; NAT, the DHCP WAN and the undeclared VPN interfaces are still open.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

blocked: clientWaiting on an answer from SenakabugSomething isn't workinglayer: analysisLayer 1 — Batfish orchestration, checks, the pipelinephase: 3Market readiness -- what stands between us and a product someone could buy.safetyTouches F-4, grounding, or the refuse-rather-than-guess boundary. Review with care

Projects

No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions