diff --git a/.gitattributes b/.gitattributes index 3a5d9bf5a..458c6be88 100644 --- a/.gitattributes +++ b/.gitattributes @@ -17,6 +17,10 @@ # cannot be run. wisdom/extract/workflow.mjs text eol=lf +# The help add-in's resources are checked out byte for byte: every build compares its +# copy of the symbol index with the LF file it writes (builder/addin-index.mjs). +add-in/Resources/** -text + # A bug's reproducer is checked out byte for byte: its .twinproj is packed # from its source tree, and the two must agree in every checkout. bugs/** -text diff --git a/.github/actions/run-gates/action.yml b/.github/actions/run-gates/action.yml index 92e15dfc8..dcd2021f1 100644 --- a/.github/actions/run-gates/action.yml +++ b/.github/actions/run-gates/action.yml @@ -96,6 +96,13 @@ runs: - name: Unit-test check_examples' batches (test/example-batches.test.mjs) shell: bash run: node --test test/example-batches.test.mjs + # Unit tests for scripts/lib/tb-ports.mjs, which claims the DevTools ports + # the harness's IDEs use: two claims at once are disjoint, a port in use + # and a live lock are passed over, a stale lock is taken over. Starts no + # IDE. No browser, no built tree, about 2 s. + - name: Unit-test the harness's port claims (test/ports.test.mjs) + shell: bash + run: node --test test/ports.test.mjs # A regex that backtracks exponentially does not fail a build, it stops # one: the corpus passes until a page happens to contain the trigger, and # then a render worker sits inside String.replace forever. Nothing else diff --git a/BOOKPLAN.md b/BOOKPLAN.md index 490dde92f..1a1d5a6b1 100644 --- a/BOOKPLAN.md +++ b/BOOKPLAN.md @@ -18,7 +18,7 @@ > > One entry under [Rendering gotchas](#build-environment) has stopped applying and is the one most likely to be acted on: **"PowerShell cannot invoke `npx` directly" no longer affects `book.bat`.** That batch file calls `node`, not `npx`, and a `.bat` is interpreted by `cmd.exe` whichever shell launches it — so no `.ps1` shim is reached and the execution policy is never consulted. The paged.js and CSS Paged Media gotchas above it still hold; the forked bundle lives at `book/lib/paged.browser.js`. > -> **Current documentation.** The two-stage pipeline, the renderer's flags and phases, and how to triage a failed render are on the site: [PDF Generation](docs/Documentation/PDF-Generation.md) and [Book Configuration](docs/Documentation/Book-Configuration.md), with the vendored-library patches in [Fixes-PagedJS.md](docs/Documentation/Fixes-PagedJS.md) and [Fixes-PDFLib.md](docs/Documentation/Fixes-PDFLib.md). The Phase 8 specification is [builder/PLAN-8.md](builder/PLAN-8.md), and the Jekyll era as a whole is recorded in [WIP.OldJekyll.md](WIP.OldJekyll.md). +> **Current documentation.** The two-stage pipeline, the renderer's flags and phases, and how to triage a failed render are on the site: [PDF Generation](docs/Documentation/PDF-Generation.md) and [Book Configuration](docs/Documentation/Book-Configuration.md), with the vendored-library patches in [Fixes-PagedJS.md](docs/Documentation/Fixes-PagedJS.md) and [Fixes-PDFLib.md](docs/Documentation/Fixes-PDFLib.md). The Phase 8 specification is [builder/PLAN-8.md](builder/PLAN-8.md). Currently `book.bat` produces a ~1500-page PDF that is a flat concatenation of every reference page. It works, but it doesn't read like a book — no front matter, no parts, no global TOC, every chapter is `

`, and "See Also" cross-references point at standalone `file://` URLs rather than jumping within the PDF. diff --git a/BUGS-TO-REPORT.md b/BUGS-TO-REPORT.md index ef05bce1c..7cc82db9e 100644 --- a/BUGS-TO-REPORT.md +++ b/BUGS-TO-REPORT.md @@ -347,7 +347,7 @@ Every one of those runs ends `... FAILED` and exits 0. Exporting `WebView2Packag What does not reproduce it: a long input path to `export`, and any output folder short enough that no file path reaches 260 characters and no folder path 248. - + --- @@ -888,7 +888,7 @@ Steps to reproduce the behavior: w.Boom 1 (early bound): error 5, ran 1 time(s), Property Get ran 0 ``` 3. See `Hello` run twice, and `Property Get Prop` run after the `Property Let` that failed. The error from the assignment is `&H80020009` (`DISP_E_EXCEPTION`) where the `Property Let` raised 5. -4. To see the second call, implement `IDispatch` in a `NotDispatchable` class (see `bugs/callbyname-membernotfound-retried/`, which has one, and whose `Invoke` can return another code) whose `Invoke` prints `wFlags`, `pDispParams.cArgs` and whether `pVarResult` is null, and returns a failure code. With `Fail3` returning `DISP_E_TYPEMISMATCH` and `Fail5` raising error 5: +4. To see the second call, implement `IDispatch` in a `NotDispatchable` class whose `Invoke` prints `wFlags`, `pDispParams.cArgs` and whether `pVarResult` is null, and returns a failure code. With `Fail3` returning `DISP_E_TYPEMISMATCH` and `Fail5` raising error 5: ``` o.Fail3 1 Invoke flags=1 cArgs=1 result=null, then Invoke flags=3 cArgs=0 result=set o.Fail3 = 5 Invoke flags=4 cArgs=1 result=null, then Invoke flags=3 cArgs=0 result=set @@ -914,41 +914,6 @@ The second call resembles VB's rule for `o.Member(args)` on a property that retu --- -## `CallByName` calls `Invoke` a second time, with no result, when the first call returns DISP_E_MEMBERNOTFOUND - -**Describe the bug** -`CallByName` on an object whose `IDispatch.Invoke` returns `DISP_E_MEMBERNOTFOUND` calls `Invoke` twice: first with a result variant (`pVarResult` supplied), then again with the same identifier and flags and a null `pVarResult`. A late-bound statement for the same member, `o.Anything`, calls it once. Seen with a class that implements `IDispatch` itself and counts the calls. - -**To Reproduce** -Steps to reproduce the behavior: -1. Open `callbyname-membernotfound-retried.twinproj` (attached as `callbyname-membernotfound-retried.zip`). Its one source file, `Startup.twin`, declares a copy of `IDispatch` (the `stdole` one cannot be called) and a `NotDispatchable` class `Recorder` that implements it. Its `GetIDsOfNames` returns 1 for any name, and its `Invoke` prints its `wFlags`, says whether `pVarResult` is null, counts the call, and returns `DISP_E_MEMBERNOTFOUND` with `Err.ReturnHResult`. -2. Run it. `Main` calls `CallByName o, "Anything", vbMethod` and then `o.Anything`, with `On Error Resume Next`, through `Dim o As Object = New Recorder`: - ``` - Invoke 1: wFlags 1, result supplied - Invoke 2: wFlags 1, result null - CallByName: error 438, Invoke called 2 time(s) - Invoke 1: wFlags 1, result null - o.Anything: error 438, Invoke called 1 time(s) - ``` - -**Expected behavior** -One `Invoke` per `CallByName`, as for the statement form. A method that did its work and then reported `DISP_E_MEMBERNOTFOUND` (a scripting host whose members are resolved inside `Invoke` is an example) runs again, and a call with arguments and side effects is repeated. - -**Desktop:** - - OS: Windows 10 Pro 22H2 (build 19045) - - twinBASIC compiler version: BETA 995 - -**Additional context** -Severity: low. It only shows with an `IDispatch` that returns `DISP_E_MEMBERNOTFOUND` after doing work. - -What does not reproduce it: `CallByName` with `vbGet` or `vbLet` against `DISP_E_TYPEMISMATCH`, `DISP_E_BADPARAMCOUNT`, `E_FAIL`, an error raised in `Invoke`, or success (one `Invoke` each); `Invoke` returning `DISP_E_MEMBERNOTFOUND` to a late-bound statement or to `x = o.Member` (one `Invoke` each). `vbGet`, `vbMethod` and `vbLet` all repeat after `DISP_E_MEMBERNOTFOUND`. - -A separate behaviour, in its own entry, repeats a failed late-bound statement that passes arguments as a property read: see the entry "A late-bound call that passes arguments and fails is issued a second time, without them". The two are not the same: that one is for calls with arguments and needs no `DISP_E_MEMBERNOTFOUND`, and this one is `CallByName` with or without arguments. - - - ---- - ## A late-bound call to a member that does not exist raises &H80020006, and `CallByName` raises &H80004005, where VB6 raises 438 **Describe the bug** @@ -1020,6 +985,12 @@ Steps to reproduce the behavior: **Expected behavior** The call goes through the interface: `GetTypeInfoCount` returns the count (1 for a twinBASIC class), and a wrong number or type of argument is a compile error, as for any other interface method. If `stdole.IDispatch` is meant to be an alias of `Object`, `d.GetTypeInfoCount` should still not compile, or the type should not list four methods that cannot be called. A project's own declaration of the interface, with the same `[InterfaceId]`, calls them correctly. +VB6 refuses the declaration itself. The same program in VB6 (attached as `stdole-idispatch-late-bound-vb6.zip`), with a reference to OLE Automation, does not build: +``` +Compile Error in File '...\Module1.bas', Line 16 : Function or interface marked as restricted, or the function uses an Automation type not supported in Visual Basic +``` +`Dim d As IDispatch` is refused the same way. With `Dim d As Object` in its place, VB6 runs `d.Hello` and raises 438 for the other three statements, so twinBASIC's `stdole.IDispatch` behaves as VB6's `Object` does, apart from the error number (see the entry about `&H80020006` and 438). + **Desktop:** - OS: Windows 10 Pro 22H2 (build 19045) - twinBASIC compiler version: BETA 995 @@ -1029,7 +1000,7 @@ Severity: low, since a project can declare its own copy of the interface, but th What was tried: `GetIDsOfNames` and `Invoke` with zero arguments, and `GetTypeInfo` with a null pointer, each end the run the same way; calls with arguments of the wrong type compile (`d.GetTypeInfoCount "a"`, `d.GetTypeInfo "a", "b", "c"`, `d.Invoke "a", "b", "c", "d", "e", "f", "g", "h"` all compile). An unhandled `Err.Raise 5` in the same harness ends the run the same way, so the silent end is how an unhandled error shows there, not a separate fault. Running the built exe with `--exe` was not possible on this machine (the harness could not start it). - + --- @@ -1053,6 +1024,16 @@ Steps to reproduce the behavior: **Expected behavior** `DISP_E_BADINDEX` (`&H8002000B`), which the `IDispatch::GetTypeInfo` documentation gives for an index that is not valid, for an index the object does not have. `E_UNEXPECTED` (*Catastrophic failure*) is the code for a call made at a time when the object cannot take it, and it reads as a fault in the object. +VB6 does not return `E_UNEXPECTED`. The same calls on a VB6 class, made through `DispCallFunc` (the VB6 project is attached as `gettypeinfo-badindex-vb6.zip`), return `TYPE_E_ELEMENTNOTFOUND` (`&H8002802B`) for every index but 0: +``` +GetTypeInfoCount = 1 (HRESULT 0) +GetTypeInfo(0): HRESULT 0, pointer returned True +GetTypeInfo(1): HRESULT 8002802B, pointer returned False +GetTypeInfo(2): HRESULT 8002802B, pointer returned False +GetTypeInfo(-1): HRESULT 8002802B, pointer returned False +``` +Either `DISP_E_BADINDEX` or VB6's `TYPE_E_ELEMENTNOTFOUND` says that the index names nothing; `E_UNEXPECTED` does not. + **Desktop:** - OS: Windows 10 Pro 22H2 (build 19045) - twinBASIC compiler version: BETA 995 @@ -1060,7 +1041,7 @@ Steps to reproduce the behavior: **Additional context** Severity: minor. A caller that checks for `DISP_E_BADINDEX` to learn that an object has no more type information gets a different code; `GetTypeInfo(0)` and `GetTypeInfoCount` are right. - + --- @@ -1182,16 +1163,18 @@ Steps to reproduce the behavior: **Additional context** Severity: low in practice, since a program rarely advises a null sink, but a COM client written in any language can send one, and the connection point crashes the process that hosts the class instead of refusing the call. -What was tried: an object that is not a sink gives an ordinary error (`E_NOINTERFACE`, `&H80004002`, see the entry about `Advise` and `Unadvise` error codes), so only a null pointer crashes. The address of the crash is the same in the standalone probe and in this project. +What was tried: an object that is not a sink gives an ordinary error (`E_NOINTERFACE`, `&H80004002`), so only a null pointer crashes. The address of the crash is the same in the standalone probe and in this project. + +VB6 crashes too. The same call on the connection point of a VB6 class with an event, made through `DispCallFunc` (the VB6 project is attached as `advise-nothing-crashes-vb6.zip`), ends the exe with an access violation (`0xC0000005` in `MSVBVM60.DLL`, in the Application event log) after it writes `before`. A fix therefore costs no compatibility: no program depends on the crash. - + --- -## `Advise` with a sink that lacks the outgoing interface fails with `E_NOINTERFACE`, and `Unadvise` with a cookie that names no connection succeeds +## `Unadvise` with a cookie that names no connection succeeds, where VB6 returns CONNECT_E_NOCONNECTION **Describe the bug** -On the connection point of a twinBASIC class that has an `Event`, two calls return a different code from the one the COM contract names. `Advise` with a sink that does not answer `QueryInterface` for the outgoing interface fails with `E_NOINTERFACE` (`&H80004002`), where the contract returns `CONNECT_E_CANNOTCONNECT` (`&H80040201`). `Unadvise` with a cookie that names no connection (0 and 99 were tried) succeeds silently and does nothing, where the contract returns an error. +On the connection point of a twinBASIC class that has an `Event`, `Unadvise` with a cookie that names no connection (0 and 99 were tried) succeeds silently and does nothing. The COM contract returns an error, and VB6 returns `CONNECT_E_NOCONNECTION` (`&H80040200`). **To Reproduce** Steps to reproduce the behavior: @@ -1210,17 +1193,26 @@ Steps to reproduce the behavior: ``` **Expected behavior** -`Advise` fails with `CONNECT_E_CANNOTCONNECT`. The Windows SDK page for [IConnectionPoint::Advise](https://learn.microsoft.com/en-us/windows/win32/api/ocidl/nf-ocidl-iconnectionpoint-advise) lists it as "The sink does not support the interface required by this connection point", and says to implementers: "The connection point must query the *pUnkSink* pointer for the correct outgoing interface. If this query fails, this method must return CONNECT_E_CANNOTCONNECT." `Unadvise` reports the bad cookie: the page for [IConnectionPoint::Unadvise](https://learn.microsoft.com/en-us/windows/win32/api/ocidl/nf-ocidl-iconnectionpoint-unadvise) lists `E_POINTER` for "The value in *dwCookie* does not represent a valid connection". It is not `S_OK`. +`Unadvise` reports the bad cookie, as VB6 does. The page for [IConnectionPoint::Unadvise](https://learn.microsoft.com/en-us/windows/win32/api/ocidl/nf-ocidl-iconnectionpoint-unadvise) lists `E_POINTER` for "The value in *dwCookie* does not represent a valid connection", and `CONNECT_E_NOCONNECTION` is the code the `CONNECT_E_` range has for it. It is not `S_OK`. + +The same calls on the connection point of a VB6 class with an event, made through `DispCallFunc` (the VB6 project is attached as `advise-unadvise-hresults-vb6.zip`): +``` +Advise, a sink without the outgoing interface: error=80004002 + cookie=-1 +Unadvise 99: error=80040200 +before Unadvise 0 +``` +`Unadvise 0` comes last because it ends the VB6 exe with an access violation (`0xC0000005` in `MSVBVM60.DLL`). twinBASIC's 0 for that cookie is no worse than VB6's crash; the cookie 99 is the clean comparison. **Desktop:** - OS: Windows 10 Pro 22H2 (build 19045) - twinBASIC compiler version: BETA 995 **Additional context** -Severity: low. A client that tests for `CONNECT_E_CANNOTCONNECT` to tell a refused sink from other failures never sees it, and a client that releases a connection twice, or with a wrong cookie, is told it worked. +Severity: low. A client that releases a connection twice, or with a wrong cookie, is told it worked. -What was tried: a sink that is an ordinary twinBASIC class gives the same `E_NOINTERFACE` whether it has no members or implements `IDispatch` (the outgoing interface's identifier is generated for each build, so a twinBASIC class cannot answer for it). Nothing is connected afterwards, and no event reaches the sink. Calling `Unadvise` a second time with the cookie of the failed `Advise`, which is 0, also returns without an error. - +`Advise` with a sink that does not answer `QueryInterface` for the outgoing interface fails with `E_NOINTERFACE` (`&H80004002`), where the Windows SDK page for [IConnectionPoint::Advise](https://learn.microsoft.com/en-us/windows/win32/api/ocidl/nf-ocidl-iconnectionpoint-advise) says the method "must return CONNECT_E_CANNOTCONNECT" (`&H80040201`). VB6 returns `E_NOINTERFACE` too, so that is not reported here; the reproducer prints it as a control. A sink that is an ordinary twinBASIC class gives the same `E_NOINTERFACE` whether it has no members or implements `IDispatch` (the outgoing interface's identifier is generated for each build, so a twinBASIC class cannot answer for it). Nothing is connected afterwards, and no event reaches the sink. Calling `Unadvise` a second time with the cookie of the failed `Advise`, which is 0, also returns without an error. + --- diff --git a/README.md b/README.md index 8cd0773f4..9076e15ab 100644 --- a/README.md +++ b/README.md @@ -32,6 +32,7 @@ book.bat # renders the PDF book; run build.bat first examples.bat # compiles the twinBASIC code samples in the pages (Windows + a twinBASIC install) addin-test.bat # tests IDE add-ins by operating an IDE (Windows + a twinBASIC install) ide-test.bat # tests the IDE itself by operating it (Windows + a twinBASIC install) +try-help-addin.bat # opens an IDE with the help add-in, to try it by hand (Windows + a twinBASIC install) ``` A clean `build.bat && check.bat` is the bar for "ready to commit"; add `test.bat` when the change touched anything outside `docs/`. Each wrapper names the gates it runs, in order, on [Tools and Scripts](https://docs.twinbasic.com/Documentation/Development/Tools). On Linux or macOS, run the `node` command inside each batch file directly --- they are thin wrappers. `examples.bat`, `addin-test.bat` and `ide-test.bat` are the exceptions to both: they drive the twinBASIC IDE, so they are Windows-only and deliberately outside every gate and outside CI. diff --git a/WIP.A11y.md b/WIP.A11y.md index 102deca59..d2c708c6e 100644 --- a/WIP.A11y.md +++ b/WIP.A11y.md @@ -1,7 +1,7 @@ # twinBASIC Documentation --- The Accessibility Scan What the axe scan actually is, how its sample is chosen, and why tuning it is -dangerous. Split out of [WIP.md](WIP.md), which keeps the gate roster under [The +dangerous. [WIP.md](WIP.md) keeps the gate roster under [The gates, and where their internals are](WIP.md#the-gates-and-where-their-internals-are). `check.bat` runs [scripts/check_tree_fresh.mjs](scripts/check_tree_fresh.mjs), @@ -12,13 +12,13 @@ which refuses a tree older than the sources that produced it, then [scripts/check_a11y.mjs](scripts/check_a11y.mjs), which runs puppeteer + axe-core over thirteen sample pages against WCAG 2.0/2.1/2.2 at Level A + AA (plus the `heading-order` best-practice rule) and exits non-zero on any -violation. +violation. It requires `build.bat` to have produced an up-to-date `_site/`. What the scan *is* --- the page list, the viewports, the themes, the blocked requests and the axe run options --- lives in [scripts/lib/axe-scan.mjs](scripts/lib/axe-scan.mjs), shared with the correctness gate and the measurement rig; `check_a11y.mjs` itself is only the -reporting front end. Note that all five WCAG tags --- `wcag2a`, `wcag2aa`, +reporting front end. All five WCAG tags --- `wcag2a`, `wcag2aa`, `wcag21a`, `wcag21aa`, `wcag22aa` --- must be listed: axe matches tags literally with no version rollup, so a rule tagged only `wcag21aa` does **not** match `wcag22aa`. @@ -26,40 +26,40 @@ with no version rollup, so a rule tagged only `wcag21aa` does **not** match Four details of that script matter and are easy to break: - It scans **`_site-offline/`, not `_site/`**. The online tree references its assets with root-absolute URLs (`/assets/css/…`), which resolve to nothing under `file://` — every page would load unstyled and every colour-contrast result would be a meaningless black-on-white pass. The offline tree uses relative asset paths and renders for real. -- It scans each page **in both themes and at two viewports** (`--theme`, `--viewport`). Dark mode is a separate stylesheet with its own palette, and defects such as horizontally scrolling code blocks only appear once the layout is narrow enough to overflow. The dark half is not a formality: the dark compilation re-emits every JTD base rule under `html[data-theme=dark]`, which raises its specificity from (0,0,1) to (0,1,2) -- so a root-level single-class rule in `custom/custom.scss` that overrides a bare element selector **applies in light mode and silently does not in dark**. That is exactly how the footnote-underline fix shipped half-broken, and only the dark pass caught it. Prefix such rules with `.main-content` to clear the bar. +- It scans each page **in both themes and at two viewports** (`--theme`, `--viewport`). Dark mode is a separate stylesheet with its own palette, and defects such as horizontally scrolling code blocks only appear once the layout is narrow enough to overflow. The dark half is not a formality: the dark compilation re-emits every JTD base rule under `html[data-theme=dark]`, which raises its specificity from (0,0,1) to (0,1,2) -- so a root-level single-class rule in `custom/custom.scss` that overrides a bare element selector **applies in light mode and silently does not in dark**. Prefix such rules with `.main-content` to clear the bar. - It injects a **patched** axe bundle. `SOURCE_PATCHES['plain-color-fields']` in `axe-scan.mjs` replaces `Color2`'s six WeakMap-emulated `#private` fields with plain own properties, worth **-26 %** across a realistic page set and **-30 %** on large pages. Patches need the unminified bundle, which costs ~6 ms more per page to inject. Two obligations come with it: every axe-core upgrade re-runs both `check_a11y_fingerprint.mjs --patches plain-color-fields` and `check_axe_patch_equiv.mjs` (CI and `test.bat` run the second for you), and if a result ever looks wrong, re-run with `--stock-axe` first -- that injects the unmodified bundle and says in one command whether the patch is implicated. -- It **blocks the search index** (`search-data.js` + `lunr.min.js`) via request interception — see `BLOCKED_REQUESTS`. Every page pulls in ~3.4 MB of index that never touches the DOM axe walks; loading it was 18.9 s of a 27.1 s run, and aborting it cuts the scan to ~9 s with byte-identical results (every rule id and node count, violations and incomplete alike, across every page/theme/viewport combination -- 24 of them when that was measured, 60 audits today). Do **not** extend the block list to `just-the-docs.js` — it installs the search combobox ARIA, and blocking it makes axe see *less* (colour-contrast nodes on `Select-Case` drop 54 → 2), silently masking coverage. +- It **blocks the search index** (`search-data.js` + `lunr.min.js`) via request interception — see `BLOCKED_REQUESTS`. Every page pulls in ~3.4 MB of index that never touches the DOM axe walks; loading it is 18.9 s of a 27.1 s run, and aborting it cuts the scan to ~9 s with byte-identical results (every rule id and node count, violations and incomplete alike, across every page/theme/viewport combination -- 60 audits today). Do **not** extend the block list to `just-the-docs.js` — it installs the search combobox ARIA, and blocking it makes axe see *less* (colour-contrast nodes on `Select-Case` drop 54 → 2), silently masking coverage. -**A local pass on the geometry rules used not to be authoritative, and the reason is worth keeping in mind.** `target-size` measures rendered boxes, and an inline element's measured height is its font's content area --- so it moved with whatever `system-ui` resolved to. Measured at the mobile h3 size: Segoe UI 19px, Inter / Verdana / Tahoma 17px, Arial 16px, Liberation Sans / DejaVu Sans / Roboto 15px. A heading link topped up with `padding-block: 3px` therefore cleared the 24px floor by 0.8px on Windows and missed it on CI's Linux fonts, and the local scan reported a clean pass throughout. +**A local pass on the geometry rules is authoritative only because the face is pinned.** `target-size` measures rendered boxes, and an inline element's measured height is its font's content area, so it moves with whatever `system-ui` resolves to (15--19px at the mobile h3 size; see [WIP.Typography.md](WIP.Typography.md#why-this-is-not-only-a-cosmetic-change)). [Self-hosting the text face](WIP.md#typography) pins that number at 17px on every machine, which makes a local pass match CI's. It does not licence trimming the paddings back: `font-display: swap` puts the first frames of every load on the fallback, and a reader whose font request fails stays there, so the 15--19px band is still the range a fix has to clear. Give a font-dependent measurement margin against the *smallest* of those numbers, prefer a rule that meets the floor on declared size where the layout allows one, and check the whole site for the rule rather than the sample --- one rule over every page at one viewport takes about two minutes. -[Self-hosting the text face](WIP.md#typography) pins that number at 17px on every machine, which is what closed the gap between a local pass and CI's. It does not licence trimming the paddings back: `font-display: swap` puts the first frames of every load on the fallback, and a reader whose font request fails stays there, so the 15--19px band is still the range a fix has to clear. Give a font-dependent measurement margin against the *smallest* of those numbers, prefer a rule that meets the floor on declared size where the layout allows one, and check the whole site for the rule rather than the sample --- one rule over every page at one viewport takes about two minutes. +**axe does not evaluate whether a focus ring is actually visible**, only that focusable things are reachable and labelled --- so a ring that is drawn and then clipped away passes every rule. The aux nav is where that matters. `.aux-nav` is `overflow-x: auto` (navigation.scss:182), and the overflow spec turns the other axis from `visible` to `auto` when one axis is not `visible`, so the nav is a scroll container that clips at its padding box on all four sides; its items are `height: 100%` and the first one starts at the left content edge. An outset ring on anything in there therefore loses every side that sits on the clip edge. Only a manual keyboard pass sees it. -**axe does not evaluate whether a focus ring is actually visible**, only that focusable things are reachable and labelled --- so a ring that is drawn and then clipped away passes every rule. The aux nav is where that matters. `.aux-nav` is `overflow-x: auto` (navigation.scss:182), and the overflow spec turns the other axis from `visible` to `auto` when one axis is not `visible`, so the nav is a scroll container that clips at its padding box on all four sides; its items are `height: 100%` and the first one starts at the left content edge. An outset ring on anything in there therefore loses every side that sits on the clip edge. The theme toggle shipped that way --- `.btn-reset`'s 2px ring at 2px offset survived only on the right, where the aux-nav link leaves room --- and a manual keyboard pass is what found it. The fix is an inset ring (`outline-offset: -2px` on `#theme-toggle`, in `custom/custom.scss`), which needs an id selector to out-rank the dark compilation's `html[data-theme=dark] .btn-reset:focus-visible` at (0,3,1). +The theme toggle therefore gets an inset ring (`outline-offset: -2px` on `#theme-toggle`, in `custom/custom.scss`), which needs an id selector to out-rank the dark compilation's `html[data-theme=dark] .btn-reset:focus-visible` at (0,3,1). -**Chrome's UA `outline: auto` does not rescue a link in there either**, which is easy to assume and wrong. Measured at 1280x900 in both themes, the nav's box is top 0 / bottom 59 and the "twinBASIC Home" link's is identical, so the UA ring's `+1px` offset puts its top and bottom segments outside the clip box exactly as the toggle's did; only the left and right bars survive, because the link starts 50px inboard of the nav's left edge. It takes an author ring at `outline-offset: -2px` too (`.aux-nav a.site-button:focus-visible`), with a dark-mode colour override -- the link is not a `.btn-reset`, so nothing in either compilation competes with it. +**Chrome's UA `outline: auto` does not rescue a link in there either.** At 1280x900 in both themes, the nav's box is top 0 / bottom 59 and the "twinBASIC Home" link's is identical, so the UA ring's `+1px` offset puts its top and bottom segments outside the clip box exactly as the toggle's did; only the left and right bars survive, because the link starts 50px inboard of the nav's left edge. It takes an author ring at `outline-offset: -2px` too (`.aux-nav a.site-button:focus-visible`), with a dark-mode colour override -- the link is not a `.btn-reset`, so nothing in either compilation competes with it. ## Heading permalinks -The chain icon beside every heading is **deliberately `aria-hidden="true" tabindex="-1"`**, which looks like a defect and is not. It used to carry `aria-labelledby` pointing at its own enclosing heading, so Chrome computed its accessible name as the heading text verbatim: every heading turned up a second time in a screen reader's links list, named identically, with nothing marking it as a permalink. Over 7,000 of them across the site's 869 content pages -- 867 carry at least one -- and 172 on `tB/Gloss.html` alone. axe passed `link-name` throughout, because the rule asks whether a name exists, not whether it is worth announcing. +The chain icon beside every heading is **deliberately `aria-hidden="true" tabindex="-1"`**, which looks like a defect and is not. With a name (for example `aria-labelledby` pointing at its own enclosing heading) every heading would appear a second time in a screen reader's links list, named identically, with nothing marking it as a permalink --- over 7,000 extra links across the site's 869 content pages, 172 on `tB/Gloss.html` alone. axe would still pass `link-name`, because the rule asks whether a name exists, not whether it is worth announcing. -It stays a real `` so the mouse affordances that people actually use to copy these -- right-click Copy Link Address, middle-click, the status-bar URL preview -- are untouched. The keyboard and screen-reader equivalent is `renderSectionLinks` in [builder/template.mjs](builder/template.mjs): one `
` per page, listing every heading. Since `e045ab5` it sits at the top of the page footer, immediately after `` closes (`renderFooter` places it; see `template.mjs`), and it is omitted on a page with fewer than two headings -- 452 of the 1,159 built files lack it, 290 redirect stubs and 162 content pages. **A closed `
` subtree is `notRendered`** -- it contributes nothing to the accessibility tree and no tab stop beyond the `` -- so the whole feature costs one tab stop per page and expands on demand. Verified on the built tree: `Gloss.html` exposes 166 links closed and 338 open. +It stays a real `` so the mouse affordances that people actually use to copy these -- right-click Copy Link Address, middle-click, the status-bar URL preview -- are untouched. The keyboard and screen-reader equivalent is `renderSectionLinks` in [builder/template.mjs](builder/template.mjs): one `
` per page, listing every heading. It sits at the top of the page footer, immediately after `` closes (`renderFooter` places it), and is omitted on a page with fewer than two headings. **A closed `
` subtree is `notRendered`** -- it contributes nothing to the accessibility tree and no tab stop beyond the `` -- so the whole feature costs one tab stop per page and expands on demand (`Gloss.html` exposes 166 links closed and 338 open). -Two things about that placement are load-bearing: +Two things about that placement are essential: - **It is emitted from the template, not from the markdown render.** The PDF book assembles its chapters from `page.renderedContent` ([book.mjs](builder/book.mjs)) and the search index reads the same field ([search.mjs](builder/search.mjs)); both are upstream of the template, so the block reaches neither and nothing has to be marked or stripped downstream. The same is true of the anchor icons themselves, and of `renderChildrenNav`. If a future page-level addition *does* need to reach the book, that is the field to put it in -- not a marker attribute. - **`min-height` for target-size belongs to `.main-content summary`**, which is same-specificity and later in `custom/custom.scss`. A local `min-height` on `.section-links > summary` silently loses to it. And the summary keeps the UA's `display: list-item`: giving a `` `display: flex` makes Chrome drop the disclosure triangle, which is the only thing marking it as openable. -One caveat on how this was checked. axe excludes `aria-hidden` subtrees from rule evaluation wholesale, so the icon vanishing from `link-name` and `target-size` is axe seeing less, not the markup being better -- exactly the pattern the rest of this section warns about. What makes the change sound is that the element was confirmed gone from the accessibility tree (Chrome CDP: ignored, `ariaHiddenElement`) and from the tab order (real Tab presses), independently of what axe reports. +axe excludes `aria-hidden` subtrees from rule evaluation wholesale, so the icon vanishing from `link-name` and `target-size` is axe seeing less, not the markup being better. What makes the design sound is that the element is gone from the accessibility tree (Chrome CDP: ignored, `ariaHiddenElement`) and from the tab order (real Tab presses), independently of what axe reports. ### State audits, and the hole they close -**A closed `
` is invisible to the scan.** Its subtree is `notRendered`, so axe never walks it -- the same property that makes the section-links disclosure cheap is what hides it. The construct went out on 707 pages with `target-size` violations on every link inside it (69.6x14 against a 24px floor) and `check.bat` reported a clean pass, because the state a reader sees after one click was never audited at all. +**A closed `
` is invisible to the scan.** Its subtree is `notRendered`, so axe never walks it -- the same property that makes the section-links disclosure cheap is what hides it. A defect inside it (for example `target-size` on every link) would pass `check.bat`, because the state a reader sees after one click is never audited. `STATE_AUDITS` in [scripts/lib/axe-scan.mjs](scripts/lib/axe-scan.mjs) closes that: entries layered onto the page x theme x viewport matrix that apply a DOM mutation from `PAGE_STATES` after navigation and before the audit. Two entries, each run across both themes and both viewports --- eight extra audits on top of the 52 the thirteen sample pages produce, for the 60 the scan reports. -Three things about it are load-bearing: +Three things about it are essential: -- **Every `PAGE_STATES` function must assert it found what it expected.** A state that silently no-ops degrades into a second audit of the default page: slower, still green, covering nothing. The `section-links-open` applier throws unless it finds exactly one disclosure holding at least two links, and returns the count so `check_a11y.mjs` can print what was actually exposed. Verified both ways -- it throws on a page with no disclosure and on one whose disclosure has been emptied. +- **Every `PAGE_STATES` function must assert it found what it expected.** A state that silently no-ops degrades into a second audit of the default page: slower, still green, covering nothing. The `section-links-open` applier throws unless it finds exactly one disclosure holding at least two links, and returns the count so `check_a11y.mjs` can print what was actually exposed. It throws on a page with no disclosure and on one whose disclosure has been emptied. - **The gate cannot vouch for this.** `check_a11y_fingerprint.mjs` compares a candidate against a baseline produced by that same scheme's element set, so a change to *which DOM is walked* is its documented blind spot. Adding elements is the safe direction, but it still has to be argued from source and demonstrated: the open audit walks 57 more nodes than the closed one, and with the fix reverted in-page it reports `target-size` x14 where the closed audit reports nothing. Run the A/A control (`--baseline production --candidate production`) after touching the matrix -- a nondeterministic state audit would surface there. - **Item count buys nothing; pick the cheap host.** Menu/Window hosts it at 14 items and 0.41 s per audit rather than Pipeline-Stages at 72 and 1.28 s. The defect class is per-link geometry, identical for every item -- spacing between two consecutive items does not change with how many follow, and the long labels that make the big page look like the stress case only *wrap* at mobile, which makes a target taller and easier to pass. Both catch a reverted fix at both viewports. @@ -67,13 +67,11 @@ Three things about it are load-bearing: Syntax-highlight token colours are kept above 4.5:1 automatically: [builder/highlight-theme.mjs](builder/highlight-theme.mjs) clamps any colour from the vendored `.theme` files that fails against the code-block background, moving lightness away from the background while preserving hue and saturation. The vendored themes stay faithful to the IDE; the emitted rule carries a `raised to 4.5:1` comment naming the original colour. -Requires `build.bat` to have produced an up-to-date `_site/`. - ## Choosing the sample -The scan audits thirteen pages out of ~1,160, so its page list decides what it can report at all -- and a page list that stops being representative fails *silently*. That happened: the original six were picked by hand, the site grew around them, and by the time anyone measured, all six sat between 2,175 and 2,694 elements against a site maximum of 5,231 and not one carried a table, an image, a `
` widget or a video card. `image-alt`, the table rules and `scrollable-region-focusable` were all in the run options with nothing to run on, and the gate reported a clean pass. +The scan audits thirteen pages out of ~1,160, so its page list decides what it can report at all -- and a page list that stops being representative fails *silently*. A hand-picked list goes stale as the site grows around it: a sample with no table, image, `
` widget or video card leaves `image-alt`, the table rules and `scrollable-region-focusable` in the run options with nothing to run on, and the gate reports a clean pass. -A full-site sweep ([scripts/sweep_a11y.mjs](scripts/sweep_a11y.mjs) — every page, both themes, both viewports, 3,476 audits, ~20 min) settled what that pass had been hiding: **six violation classes on 54 pages**, every one of them in a construct the sample could not see. `scrollable-region-focusable` on 44 pages (table wrappers carried no `tabindex`), `link-in-text-block` on the footnote back-links, `target-size` on the FAQ's `` elements at the phone viewport, `heading-order` on five pages, `role-img-alt` on two unlabelled diagrams, and one `color-contrast` failure at 4.43:1 inside a Mermaid export (both Mermaid diagrams have since been redrawn as DOT). All fixed; the sweep is clean. +The full-site sweep ([scripts/sweep_a11y.mjs](scripts/sweep_a11y.mjs) — every page, both themes, both viewports, 3,476 audits, ~20 min) is the check on the sample. It shows which constructs the sample has to contain: table wrappers (`scrollable-region-focusable`), footnote back-links (`link-in-text-block`), the FAQ's `` elements at the phone viewport (`target-size`), heading structure (`heading-order`), diagrams (`role-img-alt`) and code-block colours (`color-contrast`). So the sample is derived rather than maintained by hand. [scripts/pick_a11y_sample.mjs](scripts/pick_a11y_sample.mjs) holds a list of **construct families** — markup shapes some axe rule keys on, each recording the rule that would otherwise have nothing to run on — and three modes: @@ -81,13 +79,13 @@ So the sample is derived rather than maintained by hand. [scripts/pick_a11y_samp - `--propose`: greedy set cover, seeded from the current `SAMPLE_PAGES`, so it prints what to *add*. `--fresh` ignores the current set and covers from scratch, which is how to ask whether the existing pages still earn their place. - `--census`: what each family is, how many pages use it, which page uses it most. -Cost is part of the choice, not an afterthought: audit cost is super-linear in element count (k = 2.73), so the cheapest cover is not the smallest one, and `--propose` ranks candidates by measured per-page cost from the sweep's JSONL when one has been run. Measured across the full matrix, the thirteen-page sample costs 18.7 s of audit against 6.3 s for the original six -- 2.96x for a bit over twice the pages, because cost tracks element count super-linearly rather than page count. `Pipeline-Stages.html` alone is 26 % of it, and earns that as the site's largest and most table-dense page. +Cost is part of the choice: audit cost is super-linear in element count (k = 2.73), so the cheapest cover is not the smallest one, and `--propose` ranks candidates by measured per-page cost from the sweep's JSONL when one has been run. Across the full matrix, the thirteen-page sample costs 18.7 s of audit against 6.3 s for a small six-page sample -- 2.96x for a bit over twice the pages, because cost tracks element count rather than page count. `Pipeline-Stages.html` alone is 26 % of it, and earns that as the site's largest and most table-dense page. **When the docs start using a construct they have not used before** — a new admonition shape, a figure, a widget — add a family for it in `FAMILIES` and let `--check` say whether the sample already covers it. Leaving it out is not neutral: it means the rule for that construct never runs anywhere. ## Changing the scan -axe is the site's correctness oracle, which makes it a dangerous thing to tune: a change can make axe see *less* and still report a clean pass. This nearly happened once -- blocking `just-the-docs.js` during the scan looked like a 130 ms win and quietly dropped the colour-contrast node count on `Select-Case` from 54 to 2. +axe is the site's correctness oracle, which makes it a dangerous thing to tune: a change can make axe see *less* and still report a clean pass. Blocking `just-the-docs.js` during the scan, for example, looks like a 130 ms win and quietly drops the colour-contrast node count on `Select-Case` from 54 to 2. So any change to *what the scan runs* goes through [scripts/check_a11y_fingerprint.mjs](scripts/check_a11y_fingerprint.mjs) first. It runs the full page x theme x viewport matrix twice, once under each of two named configurations from the scheme registry in `axe-scan.mjs`, against one build in one process, and diffs the findings audit by audit -- violations by `ruleId:nodeCount`, incomplete by rule-id set. @@ -96,13 +94,12 @@ node scripts/check_a11y_fingerprint.mjs --list node scripts/check_a11y_fingerprint.mjs --candidate no-html ``` -It has one blind spot worth knowing: it compares a candidate against a baseline produced by that same scheme's element set, so it cannot detect a change that stops auditing elements *entirely*. Anything touching which DOM is walked -- viewport, visibility, request blocking -- has to be argued from source instead. +It has one blind spot: it compares a candidate against a baseline produced by that same scheme's element set, so it cannot detect a change that stops auditing elements *entirely*. Anything touching which DOM is walked -- viewport, visibility, request blocking -- has to be argued from source instead. -One thing the gate structurally cannot catch: it compares *which* findings axe produces, never their shape. The `no-html` scheme passes the gate on every audit and would still crash the reporter, because `noHtml: true` makes `node.html` null and `check_a11y.mjs` calls `.slice()` on it. Treat the gate as necessary, not sufficient. +It also compares *which* findings axe produces, never their shape. The `no-html` scheme passes the gate on every audit and would still crash the reporter, because `noHtml: true` makes `node.html` null and `check_a11y.mjs` calls `.slice()` on it. Treat the gate as necessary, not sufficient. -Cost attribution for the scan lives in [perf/ab-axe.mjs](perf/ab-axe.mjs), with [perf/probe-axe-dom.mjs](perf/probe-axe-dom.mjs) for DOM operation counts and [perf/probe-axe-scaling.mjs](perf/probe-axe-scaling.mjs) for the size curve. The investigation is [builder/PLAN-axe-perf.md](builder/PLAN-axe-perf.md), and it is **complete**. Its conclusion: one vendored source patch is worth **26 %** across a realistic page set and **30 %** on large pages -- `SOURCE_PATCHES['plain-color-fields']` in `axe-scan.mjs`, which replaces `Color2`'s WeakMap-emulated `#private` fields with plain properties. It is gated audit-by-audit by [check_a11y_fingerprint.mjs](scripts/check_a11y_fingerprint.mjs) and verified value-by-value by [check_axe_patch_equiv.mjs](scripts/check_axe_patch_equiv.mjs) -- both are needed, because the fingerprint gate compares `incomplete` as a rule-id set and would not notice a colour error that shifted ratios without flipping a pass/fail. The patch asserts an exact occurrence count at each substitution point, so an axe-core bump fails loudly rather than silently reverting. Three results are worth knowing before touching the scan: +Cost attribution for the scan lives in [perf/ab-axe.mjs](perf/ab-axe.mjs), with [perf/probe-axe-dom.mjs](perf/probe-axe-dom.mjs) for DOM operation counts and [perf/probe-axe-scaling.mjs](perf/probe-axe-scaling.mjs) for the size curve. The investigation, [builder/PLAN-axe-perf.md](builder/PLAN-axe-perf.md), is **complete**. Its conclusion: one vendored source patch, `SOURCE_PATCHES['plain-color-fields']` in `axe-scan.mjs` (see above), is worth **26 %** across a realistic page set and **30 %** on large pages. It is gated audit-by-audit by [check_a11y_fingerprint.mjs](scripts/check_a11y_fingerprint.mjs) and verified value-by-value by [check_axe_patch_equiv.mjs](scripts/check_axe_patch_equiv.mjs) -- both are needed, because the fingerprint gate compares `incomplete` as a rule-id set and would not notice a colour error that shifted ratios without flipping a pass/fail. The patch asserts an exact occurrence count at each substitution point, so an axe-core bump fails loudly rather than silently reverting. Before touching the scan: - **Audit cost is super-linear in page size** -- `k = 2.73` on real pages, almost entirely inside `color-contrast`. The four largest pages in the site cost ~5.9x an average sample page each. -- **`SAMPLE_PAGES` used to contain only small pages** (2,175--2,694 elements, against a site maximum of 5,231) and none of the site's tables, images, disclosure widgets or video cards. Widening it -- see [Choosing the sample](#choosing-the-sample) -- found six violation classes on 54 pages, and cost 2.96x the audit time for 2.2x the pages, because cost tracks element count rather than page count. - **Every axe-core upgrade needs the fingerprint gate run across it** -- axe ships new and revised WCAG rules between minors, and the source patches are pinned to the bundle's current text. -- **`ab-axe.mjs` cannot resolve an effect under ~20 %** -- its tracer overhead plus GC timing swamps the signal, and it has now mis-read a real ~10 % effect as noise twice (`plain-color-fields`, then `config-only`). Use [perf/ab-axe-pages.mjs](perf/ab-axe-pages.mjs) to decide anything modest, and run `--scheme production` as an A/A control before believing a small delta. `config-only` measures **-10.7 %** there against a **-0.4 %** A/A, and is declined anyway: `noHtml` and `selectors: false` remove the `html` and `target` fields that say which element failed. +- **`ab-axe.mjs` cannot resolve an effect under ~20 %** -- its tracer overhead plus GC timing swamps the signal, and it has mis-read a real ~10 % effect as noise twice (`plain-color-fields`, then `config-only`). Use [perf/ab-axe-pages.mjs](perf/ab-axe-pages.mjs) to decide anything modest, and run `--scheme production` as an A/A control before believing a small delta. `config-only` measures **-10.7 %** there against a **-0.4 %** A/A, and is declined anyway: `noHtml` and `selectors: false` remove the `html` and `target` fields that say which element failed. diff --git a/WIP.Assert.md b/WIP.Assert.md index 46cb799b2..057cd654a 100644 --- a/WIP.Assert.md +++ b/WIP.Assert.md @@ -32,6 +32,6 @@ The 15 members per module (identical signatures across all three): | `SequenceEquals(Expected, Actual, [FailMessage])` | element-by-element comparison of two sequences / arrays | | `NotSequenceEquals(Expected, Actual, [FailMessage])` | inverse of `SequenceEquals` | -Surface each member as if it were an ordinary `Sub` (e.g. `Sub AreEqual(Expected, Actual, [Message])`); the source-side `Lib ""` / `Alias "#N"` / `PreserveSig` / `DeclareWide` decoration is internal pseudo-DLL plumbing and is **not** surfaced. Call out `[DebugOnly(True)]` (assertions compile out of release builds) and `[MustBeQualified(True)]` (callers must write the module name, e.g. `Strict.IsTrue(x)`). +Show each member as an ordinary `Sub` (e.g. `Sub AreEqual(Expected, Actual, [Message])`). The source-side `Lib ""` / `Alias "#N"` / `PreserveSig` / `DeclareWide` decoration is internal pseudo-DLL plumbing and is **not** shown. Call out `[DebugOnly(True)]` (assertions compile out of release builds) and `[MustBeQualified(True)]` (callers must write the module name, e.g. `Strict.IsTrue(x)`). -Layout: one page per module, listing all 15 members inline under `## ` headings (deep-linkable as `…/Strict#areequal`). Replicating 15 × 3 = 45 near-duplicate pages would add noise without value. +Layout: one page per module, listing all 15 members inline under `## ` headings (deep-linkable as `…/Strict#areequal`). Forty-five near-duplicate pages (15 × 3) would add noise without value. diff --git a/WIP.Authoring.md b/WIP.Authoring.md index 5c2f15732..ba18cc303 100644 --- a/WIP.Authoring.md +++ b/WIP.Authoring.md @@ -1,7 +1,7 @@ # twinBASIC Documentation --- Authoring a Page -The page-writing contract, split out of [WIP.md](WIP.md) so a session that is -not writing documentation does not carry it. **Read this before writing or +The page-writing contract, kept apart from [WIP.md](WIP.md) so a session that is +not writing documentation need not load it. **Read this before writing or editing any page under `docs/`.** WIP.md keeps the placement rules, the plain-English prose rules and the Don'ts; everything about the shape of a page is here. @@ -70,7 +70,7 @@ This example... Formatting conventions: -- Heading levels: the page title is `#` (a *chapter* -- a page may legitimately have more than one). Top-level sections (Example, See Also, ...) are `##`; subsections `###`. Do **not** skip `##` with the old `# Title` -> `### Example` "house style". That pattern (h1 straight to h3) exists on many older pages only to keep GitHub's raw-markdown view at a modest heading size; it is a heading-order defect on the built site. `headingLevelNormalizePlugin` in [builder/render.mjs](builder/render.mjs) repairs those legacy pages at build time (it raises `h3`->`h2` on any page that uses h1 and h3 but no h2), so existing pages are not being churned all at once -- but new content must use correct levels, and pages that already mix `##` and `###` are left untouched. A mixed page that skips a level is a real defect the normalizer will not save you from: five were found and fixed this way, and axe's `heading-order` rule now guards the sample against more. +- Heading levels: the page title is `#` (a *chapter* -- a page may legitimately have more than one). Top-level sections (Example, See Also, ...) are `##`; subsections `###`. Do **not** skip `##` with the old `# Title` -> `### Example` "house style". That pattern (h1 straight to h3) exists on many older pages only to keep GitHub's raw-markdown view at a modest heading size; it is a heading-order defect on the built site. `headingLevelNormalizePlugin` in [builder/render.mjs](builder/render.mjs) repairs those legacy pages at build time (it raises `h3`->`h2` on any page that uses h1 and h3 but no h2), so existing pages need no mass edit -- but new content must use correct levels, and pages that already mix `##` and `###` are left untouched. A mixed page that skips a level is a real defect the normalizer does not repair; axe's `heading-order` rule guards the sample against it. - `**...**` for keywords/literal tokens; `*...*` for placeholders/arguments. - Code blocks use ` ```tb ` (highlighted via Shiki using the vendored `builder/twinbasic.tmLanguage.json` grammar). - Parameter lists use the deflist `term` + `: definition` indentation pattern (NOT the MS-style markdown table). @@ -86,8 +86,8 @@ Three keys on a package's `index.md` are **provenance for the authoring pass**. them reaches the HTML --- `grep -r indexed_from docs/_site` returns nothing, because nothing in `template.mjs` iterates frontmatter generically; every consumer reads a named field. The build reads one of them, `exclude_from_docs`, for one purpose: the symbol index's count of -public symbols no page documents leaves those names out (`reportableGaps` in -[builder/symbols.mjs](builder/symbols.mjs), and `--symbol-gaps` writes the list). Leave all +public symbols that no page documents leaves those names out (`reportableGaps` in +[builder/symbols.mjs](builder/symbols.mjs); `--symbol-gaps` writes the list). Leave all three in place. | Key | Means | @@ -102,8 +102,7 @@ a documentation gap.** Without them a re-index cannot tell "not yet written" fro "deliberately omitted", which is the whole point of recording them. The six packages carrying `indexed_from` are `AppGlobalClassObject`, `CustomControls`, -`TwinBasicAssertions`, `WinEventLogLib`, `WinNamedPipesLib` and `WinServicesLib`, all set by -the 2026-06-04 pass: `9f406eb`, `05ed96b`, `77c6a03`, `4cd4f4b`, `3429b97`, `6a68815`. +`TwinBasicAssertions`, `WinEventLogLib`, `WinNamedPipesLib` and `WinServicesLib`. `exclude_from_docs` is on three of them (`InternalStuff`; `EventLogHelperPrivate` + `EventLogAPIs`; `ServicesConstantsPublic`). @@ -115,9 +114,9 @@ check was run against a build that no longer matches the package. > yet**, covering packages beyond the six above. If you are about to document a package > from scratch, ask first --- it may already be written and waiting to merge. -One other inert key exists, and is a different thing: `has_children` on four pages is a -just-the-docs leftover. tbdocs derives the nav tree itself and never reads it. Harmless, but -it is theme residue rather than something anyone is meant to maintain. +One other inert key is a different thing: `has_children` on four pages is a +just-the-docs leftover. tbdocs derives the nav tree itself and never reads it. Nobody +needs to maintain it. ### `symbols:` --- what a page documents, for the symbol index @@ -302,10 +301,10 @@ claim that a permalink is referenced from source, not an obligation to update on other changes. A reviewer looking at a `Description` string and at a reference page for the same symbol will reach for that connection; there isn't one yet. -Related: **`[Documentation(...)]` is not a twinBASIC attribute.** It does not exist. It was -asserted in eight places, including a `tb` code sample teaching it, as the in-source consumer -of the `/tB/` URL contract. The real attribute is `[Description(...)]`, which is not that. -The IDE help system remains a genuine consumer of the permalinks. +Related: **`[Documentation(...)]` is not a twinBASIC attribute.** It does not exist. Never +present it, in prose or in a `tb` sample, as the in-source consumer of the `/tB/` URL +contract. The real attribute is `[Description(...)]`, which is not that. The IDE help +system remains a genuine consumer of the permalinks. ## Per-symbol workflow @@ -322,7 +321,7 @@ The IDE help system remains a genuine consumer of the permalinks. - WinNativeCommonCtls → single-file (`DTPicker`, `MonthView`, `ProgressBar`, `Slider`, `UpDown`) or folder-style (`ImageList/`, `ListView/`, `TreeView/`) when the control has sub-object companions. Module-level enums under `Enumerations/`; per-control nested enums fold onto the declaring control's page. - Pick `` from VBA's grouping (Information, Interaction, Strings, FileSystem, DateTime, Math, Financial, Conversion, ...) and the existing folders under `Reference//`. 2. **Flag tB deviations** with a `> [!NOTE]` callout (see next section). -3. **Update the parent index** — turn an unlinked bullet into a link with a short blurb. Match the existing style of the page. If a new package is being added, also add a bullet to `docs/Reference/Built-In.md` (or `Default.md`) and bump the count on `docs/Reference/Packages.md` and `docs/Reference/index.md`. `Packages.md` itself only links to those two landing pages -- it does not enumerate packages, which is how `AppGlobalClassObject` went unlisted for months. +3. **Update the parent index** — turn an unlinked bullet into a link with a short blurb. Match the existing style of the page. If a new package is being added, also add a bullet to `docs/Reference/Built-In.md` (or `Default.md`) and bump the count on `docs/Reference/Packages.md` and `docs/Reference/index.md`. `Packages.md` itself only links to those two landing pages -- it does not enumerate packages. 4. **Add the page** to `Reference/Statements.md` or `Reference/Procedures and Functions.md` if it's a statement or callable and not already listed there. 5. **Run the [site integrity check](WIP.md#site-integrity-check)** after the batch and before committing. @@ -342,12 +341,11 @@ When in doubt about a tB-specific behavior, check `docs/Features/` and `docs/Ref **twinBASIC is in a long beta, so a superseded name is not history --- it is noise.** Document the current API and nothing else. A reader cannot use a name that no longer -exists, cannot search for it, and has to carry two names in their head to read one snippet. +exists, cannot search for it, and would have to keep two names in mind to read one snippet. -Two CustomControls tutorials showed `Public Sub New(Serializer As SerializationInfo)` and -`Deserialize()`, each with a NOTE underneath explaining that the framework actually names -them `SerializeInfo` and `RuntimeUISrzDeserialize()`. Both pages now show the current names -and carry no note; the snippets are shorter and the reader has one name per thing. +Show the current name and no note about the old one. The CustomControls tutorials show +`SerializeInfo` and `RuntimeUISrzDeserialize()`, not the superseded `SerializationInfo` +and `Deserialize()` with a NOTE explaining the difference. Version-skew notes earn their place when a reader may be *on* the older version --- a behaviour that changed in a shipped BETA, where the page says which build changed it (see @@ -368,22 +366,21 @@ reader's query `check_page_baseline` can never prefix-match. Measured: | `check_page_baseline` | **0** | | `check-page-baseline` | 420, as noise --- hyphens split, so it becomes three common words | -`build_dot_metrics` was 0 as well. An evaluator issued seven queries including the gate's -own filename and verbatim prose from its section, and never once saw it --- **the string a -refused developer actually has in hand is the one that found nothing.** +**The string a developer actually has in hand --- the bare script name --- is the one +that finds nothing.** -**The fix was to drop the directory prefix from the headings**, so the token becomes +**So write each script's heading without the directory prefix**, so the token is `check_page_baseline.mjs` and the trailing wildcard reaches it. Nothing is lost from the -page: each entry's anchor is pinned with `{: #... }` so no URL moved, and the synopsis +page: each entry's anchor is pinned with `{: #... }` so no URL moves, and the synopsis block on the next line still shows the full `node scripts/.mjs`. Every script name -returns hits now, and the index grew by nothing. +returns hits this way, and the index does not grow. -**Two other fixes were measured and rejected, and both are worth not re-proposing.** -Raising `search.heading_level` to 3 --- so each `###` becomes its own entry, which is what -would give a script section its own title boost and its own anchor in the results --- +**Two other fixes were measured and rejected; do not re-propose them.** +Raising `search.heading_level` to 3 --- so each `###` becomes its own entry, which would +give a script section its own title boost and its own anchor in the results --- **doubles the index, 3,742 entries to 7,524, and does not fix it**: the titles still carry -the `scripts/` prefix, so the bare query still misses. That is a 3.4 MB payload every page -already downloads, doubled, for nothing. Widening the tokeniser's separator to include +the `scripts/` prefix, so the bare query still misses. That doubles a 3.4 MB payload every +page already downloads, for nothing. Widening the tokeniser's separator to include `/`, `_` and `.` is the other obvious proposal; the `check-page-baseline` row above is what it would do to *every* path token on a site whose subject matter is `Debug.Print` and `_App`. diff --git a/WIP.Build.md b/WIP.Build.md index 9e9a3446f..79dcf1ca5 100644 --- a/WIP.Build.md +++ b/WIP.Build.md @@ -7,16 +7,20 @@ where their internals are](WIP.md#the-gates-and-where-their-internals-are). **Read this before adding a gate, changing one, or adding any rewrite over markdown source or rendered HTML.** Each gate here guards a failure that every -other gate passes, and the recurring shape is a check that quietly stops -checking --- which reports exactly what a healthy tree reports. +other gate passes. The recurring shape is a check that quietly stops checking, +which reports exactly what a healthy tree reports. ## The pipeline -**Before adding a fan-out to the task graph, read [why a dep count of zero does not mean the submits have run](builder/PLAN-sab-pull-scheduler.md#a-dep-count-of-zero-does-not-mean-the-submits-have-run).** A worker posts its result and *then* decrements its successors' dependency counts in shared memory, so a barrier's count can reach zero while results are still queued and the `submit()` calls that merge them into build state have not run --- the shared counter orders the work, not the state. A dynamic barrier must therefore list every chunk task in its `expected`, even when its own `execute()` ignores the inputs; that list is the only thing the scheduler checks before it lets the barrier proceed. A barrier without the list drops pages from `search-data.json` on some builds, because the index is built by flattening a `new Array(N)` and `Array.prototype.flat()` skips holes without reporting anything: a race and a silent skip combine into one invisible failure. Every skip on the chunk-merge path therefore refuses to continue when a piece is missing --- see [where the completeness checks are](builder/PLAN-sab-pull-scheduler.md#where-the-completeness-checks-are). On this path, "the piece is missing" is a bug, not a case to handle. +**Before adding a fan-out to the task graph, read [why a dep count of zero does not mean the submits have run](builder/PLAN-sab-pull-scheduler.md#a-dep-count-of-zero-does-not-mean-the-submits-have-run).** A worker posts its result and *then* decrements its successors' dependency counts in shared memory. A barrier's count can therefore reach zero while results are still queued and the `submit()` calls that merge them into build state have not run: the shared counter orders the work, not the state. + +A dynamic barrier must list every chunk task in its `expected`, even when its own `execute()` ignores the inputs. That list is the only thing the scheduler checks before it lets the barrier proceed. Without it, some builds drop pages from `search-data.json`: the index is built by flattening a `new Array(N)`, and `Array.prototype.flat()` skips holes without reporting anything. A race and a silent skip combine into one invisible failure. + +So every skip on the chunk-merge path refuses to continue when a piece is missing --- see [where the completeness checks are](builder/PLAN-sab-pull-scheduler.md#where-the-completeness-checks-are). On this path, "the piece is missing" is a bug, not a case to handle. **A sort key on this path must be total.** `discover()` fills `pages` from inside a `Promise.all`, so a page is pushed when its `readFile` resolves, not in -`allFiles` order; `pages.sort(byName)` is stable and Jekyll's key is the +`allFiles` order. `pages.sort(byName)` is stable and Jekyll's key is the *basename*, so a tie would keep that I/O completion order --- and over a hundred folder-style classes are all named `index.md`. Tied pages would reorder the chunking and make `search-data.json` differ between two builds of one commit. @@ -24,65 +28,66 @@ chunking and make `search-data.json` differ between two builds of one commit. except for `BuildInfo.html` and `gantt.svg`, which record build timings and cannot be. -**`scripts/compare_trees.mjs` is the check that relies on it.** It builds a -commit and the working tree from two git worktrees and compares all three trees -byte for byte, replacing those two regions and the PDF title page's build line, -which also differs when the sides are different commits or were built on -different days. Run it after any change to `builder/` that should leave the -output alone; a change meant to alter the output is checked the same way, and -what it reports should be the intended differences and nothing else. Build the -working tree from a checkout, never in place: under `core.autocrlf` a file a -tool has rewritten holds LF where a fresh checkout writes CRLF, so every file -the build copies verbatim would differ. +**`scripts/compare_trees.mjs` relies on that.** It builds a commit and the +working tree from two git worktrees and compares all three trees byte for byte. +It replaces those two regions and the PDF title page's build line, which also +differs when the sides are different commits or were built on different days. +Run it after any change to `builder/` that should leave the output alone. A +change meant to alter the output is checked the same way: the report should show +the intended differences and nothing else. Build the working tree from a +checkout, never in place: under `core.autocrlf` a file a tool has rewritten holds +LF where a fresh checkout writes CRLF, so every file the build copies verbatim +would differ. ### A hung build times out and says where it hung Readers get this at [When a build stops instead of failing](docs/Documentation/Building.md), with a `--stall-timeout` row in `Tools.md`'s flag table and an entry in `Builder.md`'s failure-mode list. Keep -those in step with any change here: a feature nobody can find is worth what an -absent one is worth. +those in step with any change here. **A task that a worker claims and never finishes wedges the whole graph in silence.** Its successors' dep counts never drop, `_remaining` never reaches zero, the scheduler's promise never settles, and the process sits there with its -last log line on screen --- no error, no exit code, nothing to grep. Nothing in -the SAB protocol can notice, because the scheduler is waiting on a message that -is not coming. +last log line on screen. Nothing in the SAB protocol can notice, because the +scheduler is waiting on a message that is not coming. -`Scheduler` watches for that: if no task completes for `--stall-timeout` +`Scheduler` watches for that. If no task completes for `--stall-timeout` seconds (default **120**, `0` disables), it prints what was outstanding and -fails the build. The default is deliberately generous --- the longest single task -here is worker cold boot at ~1.6 s, so a loaded CI box may be an order of -magnitude slower than the dev box without being called stalled. +fails the build. The default is generous: the longest single task is worker cold +boot at ~1.6 s, and a loaded CI box may be an order of magnitude slower than the +dev box without being stalled. -The report splits the outstanding tasks three ways, because listing them -together buries the two names that matter under a dozen that do not: +The report splits the outstanding tasks three ways, because one list buries the +two names that matter under a dozen that do not: - **Claimed by a worker that never returned** --- the cause. For a `render:i` or `flush:i` chunk it also prints the chunk's source pages, via an optional `describe()` on the task def that nothing but this report reads. "render:33 never returned" is not actionable; the six paths under it are, because the fault is nearly always one page's content. -- **Runnable, but nothing picked it up** --- including the `F_PIN_TO_PRED` case, - which is worth spelling out: a pinned `flush:i` can only run on the lane its - `render:i` ran on, so when that lane is the wedged one the task is runnable and - permanently unrunnable at once. Unlabelled it reads as a second, unrelated - fault. +- **Runnable, but nothing picked it up** --- including the `F_PIN_TO_PRED` case. + A pinned `flush:i` can only run on the lane its `render:i` ran on. When that + lane is the wedged one, the task is runnable and permanently unrunnable at + once; unlabelled, it reads as a second, unrelated fault. - **Blocked on a predecessor** --- the consequence, with the missing input names. -Two details worth knowing. `Worker.terminate()` kills a thread spinning -inside a regex, so the abort ends the process rather than adding a second hang. -And under `--serve` the pool -outlives a rebuild, so a wedged worker would poison every later build (the -per-worker tasks wait on every lane); the stall error carries a `stalled` flag -and `serve.mjs` replaces the whole pool when it sees one. Replacing just the -wedged lane would mean identifying it, and the SAB records the lane a task -*completed* on, not the one that claimed it. +Two details: -Folding `check.bat`'s gates into that same graph is designed in [builder/PLAN-checks.md](builder/PLAN-checks.md). Phase A, the link checker, is **implemented**: extraction runs inside `flush`, where both trees' final HTML is already in worker memory, so the build never writes ~270 MB out only to read it back and re-parse it. The `pick_a11y_sample.mjs --check` census and the axe scan's orchestration are follow-ons, seeded with measurements and open questions but not yet designed. +- `Worker.terminate()` kills a thread spinning inside a regex, so the abort ends + the process rather than adding a second hang. +- Under `--serve` the pool outlives a rebuild, so a wedged worker would poison + every later build (the per-worker tasks wait on every lane). The stall error + carries a `stalled` flag and `serve.mjs` replaces the whole pool when it sees + one. Replacing just the wedged lane would need to identify it, and the SAB + records the lane a task *completed* on, not the one that claimed it. -Historical engineering notes from the Jekyll era --- the original build pipeline, the HTML-compress plugin, the per-phase optimisation passes that preceded the JS port, the migration notes, and the Phase 11 parity-update retrospective --- live in [WIP.OldJekyll.md](WIP.OldJekyll.md). +[builder/PLAN-checks.md](builder/PLAN-checks.md) designs folding `check.bat`'s +gates into the task graph. The link checker is in: extraction runs inside +`flush`, where both trees' final HTML is already in worker memory, so the build +never writes ~270 MB out only to read it back and re-parse it. The +`pick_a11y_sample.mjs --check` census and the axe scan's orchestration are not +yet designed; the plan holds measurements and open questions for them. ## Tooling policy @@ -103,12 +108,13 @@ Two `.py` files stay, and neither is an oversight: of the same standalone tool. Porting it would delete a deliberate offering. - **`scripts/build_fonts.py`** stays because the JavaScript build of HarfBuzz it would use produces wrong CFF2 metrics --- a one-line build-configuration defect in harfbuzzjs, - documented with the evidence in [WIP.Fonts.md](WIP.Fonts.md). + documented with the evidence in [WIP.Fonts.md](WIP.Fonts.md), which also holds the full + account of the blocked JavaScript port. One `.ps1` exists for a third kind of reason. **`scripts/lib/tb-launch.ps1`** is Win32 calls --- `CreateDesktop`, `CreateProcess` with `STARTUPINFO.lpDesktop`, and the job object the IDE runs in (`CreateJobObject`, `AssignProcessToJobObject`) --- which Node cannot make -without a native FFI addon, and adding one for a handful of calls would mean +without a native FFI addon. Adding one for a few calls would mean `npm install` no longer suffices to run the tooling. It is also not a script anyone runs: `scripts/lib/tb-ide.mjs` reads the text and passes it through `-EncodedCommand`, so it never meets the execution policy. See [Compiling a twinBASIC project without the IDE in front of @@ -120,10 +126,6 @@ for, and neither adds a file: `scripts/tbrun.mjs` takes a process snapshot with keys through .NET, because `reg.exe` mangles names outside the console code page. See [What a run leaves in the registry](WIP.Harness.md#what-a-run-leaves-in-the-registry-and-putting-it-back). -The full account of the JavaScript port of `build_fonts.py` --- what works, the harfbuzzjs -build defect that blocks it, the evidence, the root cause in `hb-config.hh`, and what the -port must check for when it happens --- is in [WIP.Fonts.md](WIP.Fonts.md). - `lib/` — modules that every other tooling folder may import, and that import none of them. `builder/` may not import `scripts/`, so code that both need lives here; [lib/README.md](lib/README.md) states the rule, and `biome.jsonc` refuses an import that @@ -133,10 +135,10 @@ breaks either one. `eval/` — use-case evaluation of the developer documentation. `build_corpus.mjs` mirrors the repository with every non-prose file stubbed unreadable, so "documentation only" is a property -of the tree rather than an instruction; `site_search.mjs` replays the site's real lunr index +of the tree rather than an instruction. `site_search.mjs` replays the site's real lunr index and query logic, because search and navigation fail on different pages. `usecases.md` is the catalogue, `protocol.md` is what an evaluator is given. **This asks whether the docs *work*, -which is orthogonal to whether they are accurate** — most findings so far involve sentences +which is orthogonal to whether they are accurate** — most findings involve sentences that are individually true. Mine this file for cases: it is substantially a catalogue of "this shipped broken and nobody noticed", and each entry is a use case waiting to be written. @@ -146,34 +148,40 @@ that are individually true. Mine this file for cases: it is substantially a cata **The split is by what a gate interrogates, not by what it happens to open.** `check_axe_patch_equiv.mjs` loads a built page, but only because its probe needs some document to run inside --- what it tests is the axe source patch, and it would be worth running against an empty `docs/`. That is the test: a new gate belongs in `test.bat` if it would still mean something with no documentation in the tree. -Older notes under `builder/PLAN-*.md` still place `check_publish_policy.mjs` and `check_axe_patch_equiv.mjs` in `check.bat`; both are in `test.bat`, and those notes are historical. +Older notes under `builder/PLAN-*.md` still place `check_publish_policy.mjs` and `check_axe_patch_equiv.mjs` in `check.bat`. Both are in `test.bat`. **Both CI workflows run every one of these scripts as its own step, unconditionally** --- CI never invokes the `.bat` files. So the split changes what a *local* content edit has to pay for and nothing about what reaches `staging`; a tooling regression cannot get in by someone skipping `test.bat`. -**The link and integrity check runs inside the build.** `build.bat` passes `--check-audit-index`, which implies `--check`, and the check walks the HTML on the worker lanes that produced it -- both trees' final strings are already decoded and in memory at `flush()`, so the ~270 MB the two trees weigh is never written out only to be read back. It also audits the tree index the build derives from its own records against what landed on disk -- the one direction the two-checker comparison structurally cannot see, since a spurious entry makes the oracle answer "exists" for a path that 404s in production. It catches broken intra-site links, missing pages, malformed `redirect_from` entries (the most common breakage when adding new pages or moving content between sections), duplicate ids, remote ``, badly nested tags, sitemap and search-index gaps, canonical mismatches, and (via a forbidden-prefix rule on the offline tree) any extracted link that still points at the live docs site after the offlinify rewrite. A clean `build.bat && check.bat` is the bar for "ready to commit". +**The link and integrity check runs inside the build.** `build.bat` passes `--check-audit-index`, which implies `--check`. The check walks the HTML on the worker lanes that produced it: both trees' final strings are already decoded and in memory at `flush()`, so the ~270 MB the two trees weigh is never written out only to be read back. It also audits the tree index the build derives from its own records against what landed on disk --- the one direction the two-checker comparison structurally cannot see, since a spurious entry makes the oracle answer "exists" for a path that 404s in production. -A failing check never aborts the build: a broken link still produces a site you want on disk to inspect. It sets the exit code to 1 instead, whether the check found link failures, integrity failures or both (the summary lines say which), and keeps 2 for a build that could not do its job: the same scheme `check_links.mjs` uses. A 2 means a refused command line, a stall or a crash. +It catches broken intra-site links, missing pages, malformed `redirect_from` entries (the most common breakage when adding new pages or moving content between sections), duplicate ids, remote ``, badly nested tags, sitemap and search-index gaps, canonical mismatches, and (via a forbidden-prefix rule on the offline tree) any extracted link that still points at the live docs site after the offlinify rewrite. A clean `build.bat && check.bat` is the bar for "ready to commit". -The remote-asset rule fails the run on any `` resolving off-box (`http://`, `https://`, or protocol-relative `//host`). In the build it is unconditional -- `checkRemoteAssets: true` on both trees in `builder/check.mjs`'s `TREES` -- and is *not* reachable by a flag: `tbdocs` rejects `--check-remote-assets` as an unknown argument. That name belongs to the standalone `scripts/check_links.mjs`, where it is opt-in. The PDF pass over `book.html` is informational, so enforcement comes from the `_site/` pass -- every page in the book is also in `_site/`, making it a superset. The check is deliberately scoped to `` only; `