Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
40 changes: 40 additions & 0 deletions docs/design/phase-3-backlog.issues.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
{
"US-20": 313,
"US-21": 314,
"US-22": 315,
"US-23": 316,
"US-24": 317,
"US-25": 318,
"US-26": 319,
"US-27": 320,
"US-28": 78,
"US-29": 321,
"US-30": 322,
"US-31": 323,
"US-32": 324,
"US-33": 325,
"US-34": 326,
"US-35": 327,
"US-36": 328,
"US-37": 329,
"US-38": 330,
"US-39": 331,
"US-40": 332,
"US-41": 333,
"US-42": 334,
"US-43": 335,
"US-44": 336,
"US-45": 337,
"US-46": 338,
"US-47": 339,
"US-48": 340,
"US-49": 341,
"US-50": 342,
"US-51": 343,
"US-52": 344,
"US-53": 220,
"US-54": 345,
"US-55": 346,
"US-56": 347,
"US-57": 308
}
213 changes: 213 additions & 0 deletions docs/design/phase-3-backlog.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,213 @@
<!-- GENERATED by tools/make_phase3_backlog.py. Do not hand-edit.
The stories live in that script once, and both this document
and the GitHub issues are produced from them, so the two
cannot disagree. Hand-editing removes that guarantee. -->

# Phase 3 — market readiness

*Everything between Netwise today and a product that could be sold.*

Phase 0 was consolidation. Phase 1 was the four capabilities. Phase 2 was direction only. This is the first backlog written against a different question: not *what would make this a better capstone* but **what would stop a stranger who paid for it from asking for a refund**.

---

## Where these came from

| Range | Source |
|---|---|
| `US-20` – `US-39` | **Discussion #296**, unchanged. Twenty stories designed and argued in early September and never turned into work. Their acceptance criteria are copied rather than rewritten. |
| `US-40` – `US-56` | **New.** Each one closes a gap a paying customer hits in the first week, or one a buyer's security review asks about and we currently cannot answer. |

Three stories already existed as issues and are folded in rather than created again — creating them twice would be this project's own recurring defect, committed by the backlog written to avoid it:

| Story | Existing issue |
|---|---|
| `US-28` | [#78](https://github.com/ARSH871-bot/Netwise/issues/78) — Convert my whole pfSense export, including NAT |
| `US-53` | [#220](https://github.com/ARSH871-bot/Netwise/issues/220) — One real compliance benchmark pack (CIS Cisco IOS) |
| `US-57` | [#308](https://github.com/ARSH871-bot/Netwise/issues/308) — Coverage on every surface that shows findings, or none |

Nothing here is a feature because it sounds impressive. Every story names the thing that is currently untrue, unmeasured or unreachable.

---

## The distribution, stated rather than arranged

**38 stories. Arsh has 28. Ankeet has 4, Shubham has 3, Samika has 3.**

That is deliberate, and it is a decision rather than an accident, so it is written down where the people affected by it can read it.

**Why.** The remaining calendar is short and most of this milestone is sequential: US-22 waits on US-33, US-45 waits on US-35, US-46 waits on US-22 and US-44. A dependency chain moves at the speed of whoever is slowest to pick up their link. Concentrating the chain on one person removes the hand-off latency, which is the largest single cost in a four-person team working part-time around other papers.

**What each of the other three gets** is a short, self-contained slice inside the vertical they already own, with no story in it that blocks anybody else. They can finish, stop, and not be the reason something else is waiting.

**What it costs, which is the part worth saying out loud.** #249 measured that Arsh authored 61% of pull requests and consumed 62% of the team's review capacity, and — more importantly — that **review, not authorship, is where this project's defects have actually been caught**. The fixture corruption in #304, the student IDs in #311, the stale amendment table, the false claim in the portfolio: every one was found by somebody reading somebody else's work.

So this split does not reduce the team's work. It moves it from authoring to reviewing. **Every story here still needs a current approving review from someone who did not write it (M-1), and new work still voids an approval (M-2 rule 1).** If the other three treat a light authoring load as a light total load, this milestone will ship faster and be worse, and the failure will not be visible until something built on an unreviewed foundation has to be unpicked.

The honest summary: **three people are being asked to write less and read more.** That is the trade, and it is only a good trade if the reading actually happens.

---

## The backlog

### Honesty

*The product's central claim is that it never says “clean” when it means “unchecked”. These are the places that claim is still only half-kept.*

| | Story | Issue | Size | Owner |
|---|---|---|---|---|
| `US-23` | Make `access_control` read my policy | [#316](https://github.com/ARSH871-bot/Netwise/issues/316) | M | Arsh |
| `US-24` | Show me what Netwise could not read | [#317](https://github.com/ARSH871-bot/Netwise/issues/317) | M | Arsh |
| `US-26` | Make `routing` read my policy | [#319](https://github.com/ARSH871-bot/Netwise/issues/319) | M | Arsh |
| `US-40` | Prove that nothing left my machine | [#332](https://github.com/ARSH871-bot/Netwise/issues/332) | M | Arsh |
| `US-57` | Coverage on every surface that shows findings, or none *(already open)* | [#308](https://github.com/ARSH871-bot/Netwise/issues/308) | M | Arsh |

### Distribution

*Nobody can buy something they cannot install.*

| | Story | Issue | Size | Owner |
|---|---|---|---|---|
| `US-41` | Install Netwise with one command | [#333](https://github.com/ARSH871-bot/Netwise/issues/333) | M | Arsh |
| `US-42` | Upgrade without losing my data | [#334](https://github.com/ARSH871-bot/Netwise/issues/334) | M | Arsh |
| `US-49` | Install it with no internet at all | [#341](https://github.com/ARSH871-bot/Netwise/issues/341) | M | Arsh |

### Scale

*Speed was measured in #218 and is not the problem: fifty devices analyse in 6.9 seconds. What `docs/scale.md` found instead is that the analysis does not GROW with the network -- fifty devices in, one device examined. These stories are about that.*

| | Story | Issue | Size | Owner |
|---|---|---|---|---|
| `US-21` | Analyse my whole network at once, not device by device | [#314](https://github.com/ARSH871-bot/Netwise/issues/314) | M | Arsh |
| `US-43` | Tell me how big a network you can actually handle | [#335](https://github.com/ARSH871-bot/Netwise/issues/335) | L | Arsh |
| `US-44` | Run a long scan in the background | [#336](https://github.com/ARSH871-bot/Netwise/issues/336) | L | Arsh |

### Multi-tenant

*One person on one laptop is a tool. Several people in one company is a product.*

| | Story | Issue | Size | Owner |
|---|---|---|---|---|
| `US-35` | Sign in, so my scans are mine | [#327](https://github.com/ARSH871-bot/Netwise/issues/327) | L | Samika |
| `US-45` | Give my team shared access, with roles | [#337](https://github.com/ARSH871-bot/Netwise/issues/337) | L | Arsh |
| `US-52` | Tell me what you keep, and delete it when I say | [#344](https://github.com/ARSH871-bot/Netwise/issues/344) | M | Arsh |

### Automation

*A scan somebody has to remember to run is a scan that stops happening in week three.*

| | Story | Issue | Size | Owner |
|---|---|---|---|---|
| `US-20` | Check my configs in CI, before the change merges | [#313](https://github.com/ARSH871-bot/Netwise/issues/313) | M | Arsh |
| `US-46` | Scan on a schedule, without me starting it | [#338](https://github.com/ARSH871-bot/Netwise/issues/338) | M | Arsh |
| `US-47` | Read my configs from where they already are | [#339](https://github.com/ARSH871-bot/Netwise/issues/339) | M | Arsh |
| `US-48` | Send findings to the tools my team already uses | [#340](https://github.com/ARSH871-bot/Netwise/issues/340) | M | Arsh |

### Content

*Our rules are ours. A buyer wants a standard's rules, and a way into writing their own.*

| | Story | Issue | Size | Owner |
|---|---|---|---|---|
| `US-34` | Draft a policy from a config I already have | [#326](https://github.com/ARSH871-bot/Netwise/issues/326) | M | Arsh |
| `US-53` | One real compliance benchmark pack (CIS Cisco IOS) *(already open)* | [#220](https://github.com/ARSH871-bot/Netwise/issues/220) | L | Arsh |

### Remediation

*Finding the problem is half the value. The other half is the line of config that fixes it.*

| | Story | Issue | Size | Owner |
|---|---|---|---|---|
| `US-54` | Give me the fix, not just the problem | [#345](https://github.com/ARSH871-bot/Netwise/issues/345) | L | Arsh |

### Vendors

*Four vendor fixtures exist and three of them -- Arista, Juniper and NX-OS -- already produce a real finding end to end. What is left is the fourth, and saying so anywhere a user can see it.*

| | Story | Issue | Size | Owner |
|---|---|---|---|---|
| `US-27` | Analyse a vendor that is not Cisco, end to end | [#320](https://github.com/ARSH871-bot/Netwise/issues/320) | L | Ankeet |
| `US-28` | Convert my whole pfSense export, including NAT *(already open)* | [#78](https://github.com/ARSH871-bot/Netwise/issues/78) | L | Ankeet |

### AI

*The layer with the most attention on it and the least measurement behind it.*

| | Story | Issue | Size | Owner |
|---|---|---|---|---|
| `US-25` | Let me ask a follow-up | [#318](https://github.com/ARSH871-bot/Netwise/issues/318) | L | Ankeet |
| `US-55` | Show me the explanation layer is any good | [#346](https://github.com/ARSH871-bot/Netwise/issues/346) | M | Ankeet |

### Policy

*The user's own rules are a first-class artefact and deserve the same scrutiny we give their configs.*

| | Story | Issue | Size | Owner |
|---|---|---|---|---|
| `US-30` | Tell me if my own policy contradicts itself | [#322](https://github.com/ARSH871-bot/Netwise/issues/322) | M | Shubham |
| `US-31` | Check a proposed change against my whole policy | [#323](https://github.com/ARSH871-bot/Netwise/issues/323) | M | Shubham |
| `US-32` | Tell me which of my policy rules never fire | [#324](https://github.com/ARSH871-bot/Netwise/issues/324) | S | Shubham |
| `US-33` | Recognise the same problem across two scans | [#325](https://github.com/ARSH871-bot/Netwise/issues/325) | M | Arsh |

### UX

*Most of these are cheap. Together they are the difference between a demo and a thing someone uses on a Tuesday.*

| | Story | Issue | Size | Owner |
|---|---|---|---|---|
| `US-22` | Tell me only what changed since the last scan | [#315](https://github.com/ARSH871-bot/Netwise/issues/315) | M | Arsh |
| `US-36` | Rank by what it would cost me, not just what kind of bug it is | [#328](https://github.com/ARSH871-bot/Netwise/issues/328) | M | Arsh |
| `US-37` | Give me something I can hand to an auditor | [#329](https://github.com/ARSH871-bot/Netwise/issues/329) | M | Arsh |
| `US-38` | Let me use it with a keyboard and a screen reader | [#330](https://github.com/ARSH871-bot/Netwise/issues/330) | M | Samika |
| `US-39` | Show me progress during a long scan | [#331](https://github.com/ARSH871-bot/Netwise/issues/331) | S | Samika |
| `US-56` | Make my first five minutes work | [#347](https://github.com/ARSH871-bot/Netwise/issues/347) | S | Arsh |

### Trust

*The questions a buyer's security and procurement teams ask before they ask anything about features.*

| | Story | Issue | Size | Owner |
|---|---|---|---|---|
| `US-29` | Explain related findings as one story | [#321](https://github.com/ARSH871-bot/Netwise/issues/321) | M | Arsh |
| `US-50` | Give me an SBOM and a stated dependency policy | [#342](https://github.com/ARSH871-bot/Netwise/issues/342) | S | Arsh |
| `US-51` | Point Netwise at its own deployment | [#343](https://github.com/ARSH871-bot/Netwise/issues/343) | S | Arsh |

---

## Order of work

Not a schedule. A dependency order, so nothing is started before the thing it needs exists.

```
1 install and honesty US-41 US-40 US-56 US-23 US-26 US-24
US-57
a stranger can run it, and it tells the truth about what it read

2 identity and persistence US-35 US-42 US-33 US-22
results stop being disposable, and a finding keeps its name

3 scale and automation US-21 US-43 US-44 US-20 US-46 US-47
it survives a real estate and runs without being asked

4 breadth and content US-27 US-28 US-53 US-34 US-54 US-31
more vendors, real benchmarks, and the fix as well as the fault

5 the buyer's questions US-50 US-51 US-52 US-45 US-37 US-48
US-49
procurement, security review, and the team-sized product

throughout US-25 US-29 US-30 US-32 US-36
US-38 US-39 US-55
independent of the chain; can start any time
```

---

## What would make this milestone a failure

Worth writing down now, while it is cheap to say.

1. **A story marked done that widened a claim rather than a capability.** US-24, US-40 and US-43 are all one careless commit away from asserting more coverage than they deliver. That is the exact failure this product exists to catch in other people's networks.
2. **Reviews thinning as authoring concentrates.** See the section above. This is the most likely way it goes wrong.
3. **This document going stale.** It is generated from a single source along with the issues themselves, precisely so it cannot disagree with them. If it is ever hand-edited, that guarantee is gone and it becomes one more copy of a fact — which is the defect family this project has been recording since the first sprint.

100 changes: 100 additions & 0 deletions tests/test_phase3_backlog_doc.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,100 @@
"""`docs/design/phase-3-backlog.md` must be what the generator produces.

WHY THIS FILE EXISTS
The document says, in its own last section, that it cannot disagree with
the GitHub issues because both are generated from one source. That claim
is only true while nobody hand-edits the document. Nothing enforced it,
which made it exactly the kind of sentence this project keeps writing and
then falsifying -- a described practice standing in for a performed one.

So the claim is checked instead of asserted.

WHY THIS IS NOT #267
#267 is the reason to be careful here. `docs/traceability.md` carries
per-file test counts, so a CI check on it would be red every time anybody
adds a test -- permanently red, and therefore ignored.

This document carries no count that `main` can move. Its content is a
function of the story list and the recorded issue numbers, both of which
live in files in this repository. It changes when somebody edits the
generator, and at no other time. That is what makes the check stable
rather than noise.

WHAT FAILURE MEANS
Either the document was edited by hand -- regenerate it and put the change
in the generator instead -- or the generator changed and the document was
not regenerated. Both are the same defect and both have the same fix:

python -m tools.make_phase3_backlog doc

These need neither Batfish nor Ollama.
"""

from __future__ import annotations

import importlib.util
import sys
from pathlib import Path

import pytest

REPO_ROOT = Path(__file__).resolve().parent.parent
GENERATOR = REPO_ROOT / "tools" / "make_phase3_backlog.py"
DOCUMENT = REPO_ROOT / "docs" / "design" / "phase-3-backlog.md"


def _generator():
"""Import the script by path -- tools/ is not a package on sys.path."""
spec = importlib.util.spec_from_file_location("_phase3", GENERATOR)
module = importlib.util.module_from_spec(spec)
sys.modules["_phase3"] = module
spec.loader.exec_module(module)
return module


@pytest.fixture(scope="module")
def gen():
if not GENERATOR.exists(): # pragma: no cover - the file is tracked
pytest.skip("generator not present")
return _generator()


def test_document_matches_the_generator(gen):
"""The committed document is byte-identical to a fresh generation."""
on_disk = DOCUMENT.read_text(encoding="utf-8")
assert on_disk == gen.document(), (
"docs/design/phase-3-backlog.md is not what the generator produces. "
"Run: python -m tools.make_phase3_backlog doc"
)


def test_every_story_has_an_issue_number(gen):
"""A backlog row with no issue behind it is a plan, not work."""
import json

numbers = json.loads(gen.NUMBERS.read_text(encoding="utf-8"))
every = {row[0] for row in gen.STORIES} | {e[0] for e in gen.EXISTING}
assert set(numbers) == every, (
f"stories with no issue: {sorted(every - set(numbers))}; "
f"issues with no story: {sorted(set(numbers) - every)}"
)


def test_story_ids_are_unique(gen):
"""Two stories sharing an id is the duplicate-`id` bug in a backlog."""
ids = [row[0] for row in gen.STORIES] + [e[0] for e in gen.EXISTING]
assert len(ids) == len(set(ids)), "duplicate story id"


def test_every_story_is_assigned_to_a_real_member(gen):
"""Nothing here is owned by nobody -- that is how a backlog rots."""
members = {gen.A, gen.N, gen.S, gen.K}
for row in gen.STORIES:
assert row[5] in members, f"{row[0]} assigned to {row[5]!r}"


def test_every_story_states_why_it_is_in_the_milestone(gen):
"""A story with no argument behind it is a wish, and gets cut."""
for row in gen.STORIES:
assert row[3], f"{row[0]} has no acceptance criteria"
assert len(row[4].split()) >= 12, f"{row[0]} has no real rationale"
Loading