diff --git a/.gitattributes b/.gitattributes index 4b96a41bd..9da6f3746 100644 --- a/.gitattributes +++ b/.gitattributes @@ -17,6 +17,11 @@ # cannot be run. wisdom/extract/workflow.mjs text eol=lf +# The package indexer's snapshots are checked out byte for byte: a sync compares the +# api.json it writes (LF) with the committed one, and a store seeded from the snapshots +# has to start from the bytes a sync writes, or its first diff is every line. +indexer/snapshots/** -text + # The help add-in's resources are checked out byte for byte: every build compares its # copy of the symbol index with the LF file it writes (builder/addin-index.mjs). add-in/Resources/** -text diff --git a/.github/actions/run-gates/action.yml b/.github/actions/run-gates/action.yml index 209791d56..290e63a41 100644 --- a/.github/actions/run-gates/action.yml +++ b/.github/actions/run-gates/action.yml @@ -148,6 +148,15 @@ runs: - name: Unit-test the help add-in's project file (test/addin-project.test.mjs) shell: bash run: node --test test/addin-project.test.mjs + # Unit tests for the package indexer, indexer/lib/: the VB6 preprocessing, + # the files of a package, the api.json built from them (sorted by code + # unit, nested types by their path, values and fields in declaration order, + # the same bytes from the same files) and the differ that compares two of + # them. Reads indexer/sample.twinpack; no network, no git, no install. No + # browser, no built tree, well under a second. + - name: Unit-test the package indexer (test/indexer.test.mjs) + shell: bash + run: node --test test/indexer.test.mjs # A regex that backtracks exponentially does not fail a build, it stops # one: the corpus passes until a page happens to contain the trigger, and # then a render worker sits inside String.replace forever. Nothing else diff --git a/WIP.Authoring.md b/WIP.Authoring.md index e043bff6e..66e54343f 100644 --- a/WIP.Authoring.md +++ b/WIP.Authoring.md @@ -83,17 +83,18 @@ The attribution rule is **per-page**, determined by content provenance — not b ### Authoring-only frontmatter -Three keys on a package's `index.md` are **provenance for the authoring pass**. None of +Four keys on a package's `index.md` are **provenance for the authoring pass**. None of them reaches the HTML --- `grep -r indexed_from docs/_site` returns nothing, because nothing in `template.mjs` iterates frontmatter generically; every consumer reads a named field. The build reads one of them, `exclude_from_docs`, for one purpose: the symbol index's count of public symbols that no page documents leaves those names out (`reportableGaps` in [builder/symbols.mjs](builder/symbols.mjs); `--symbol-gaps` writes the list). Leave all -three in place. +four in place. | Key | Means | |-----|-------| -| `indexed_from` | The twinBASIC build whose package download the documentation was indexed against. Currently `beta-x-0983` on all six. | +| `package_name` | The packages in the indexer's store that the section documents, by their names there, when the folder's name is not theirs: `WebView2Package` for `WebView2`, `CustomControls, CustomControlsPackage` for `CustomControls`, `cefPackage145` for `CEF`. Absent, the folder's name is the package's. `indexer/lib/doc-pages.mjs` reads it. | +| `indexed_from` | The twinBASIC build whose package download the documentation was indexed against, as its release tag. Currently `beta-x-0997` on all thirteen, the build whose `indexer/snapshots/` the pages are compared with. | | `exclude_from_docs` | Modules or classes in that package deliberately left undocumented --- internals, private helpers, API shims. | | `exclude_kinds` | Member *kinds* deliberately left undocumented. Only `WinEventLogLib` uses it, for `Declare`. | @@ -102,14 +103,16 @@ newer build, and anything new that is not named in `exclude_from_docs` or `exclu a documentation gap.** Without them a re-index cannot tell "not yet written" from "deliberately omitted", which is the whole point of recording them. -The six packages carrying `indexed_from` are `AppGlobalClassObject`, `CustomControls`, -`TwinBasicAssertions`, `WinEventLogLib`, `WinNamedPipesLib` and `WinServicesLib`. -`exclude_from_docs` is on three of them (`InternalStuff`; `EventLogHelperPrivate` + -`EventLogAPIs`; `ServicesConstantsPublic`). +All thirteen package landing pages carry `indexed_from`. `exclude_from_docs` is on three +of them (`InternalStuff`; `EventLogHelperPrivate` + `EventLogAPIs`; +`ServicesConstantsPublic`). The three CEF builds are one section with one snapshot; why, and +how the indexer ties a section to its snapshots: [indexer/README.md](indexer/README.md#documented-packages). **Bump `indexed_from` whenever a package is re-indexed against a newer build**, in the same -commit as the content it adds. A stale value is worse than none: it asserts a completeness -check was run against a build that no longer matches the package. +commit as the content it adds, and write the new snapshots into `indexer/snapshots/` in it +(`node indexer/twin-index.mjs sync`), with `builder/package-api.json` for a built-in or +default package (`node indexer/twin-index.mjs derive`). A stale value is worse than none: it asserts a +completeness check was run against a build that no longer matches the package. One other inert key is a different thing: `has_children` on four pages is a just-the-docs leftover. tbdocs derives the nav tree itself and never reads it. Nobody @@ -322,6 +325,22 @@ system remains a genuine consumer of the permalinks. 4. **Add the page** to `Reference/Statements.md` or `Reference/Procedures and Functions.md` if it's a statement or callable and not already listed there. 5. **Run the [site integrity check](WIP.md#site-integrity-check)** after the batch and before committing. +**Finding the symbols to write.** After a `sync`, `node indexer/twin-index.mjs analyze --package ` +lists what is public in the package and the section's pages lack, in +`indexer/.packages/report.json`. A section with `indexed_from` is compared with the snapshot of +that build: symbols **added** (create a page), **changed** (update it; the report gives the old +and the new signature) and **removed** (flagged: reword or remove the page by hand). A section +without it is audited: each public symbol is matched by name with the `title:` of the section's +pages, so a member documented inline on its container's page is listed as missing, and a name +that several types share counts as documented once any page has it. A symbol is a type or a +name a type declares, so a property's Get, Let and Set and the overloads of a name are one. +Anything deliberately undocumented belongs in the landing page's `exclude_from_docs` or +`exclude_kinds` (see [Authoring-only frontmatter](#authoring-only-frontmatter)), not left in +the list. When the batch is done, bump `indexed_from` and commit the new snapshots and +`derive`'s `builder/package-api.json` with it; the +rules and the reasons are in [WIP.md](WIP.md#the-package-indexer) and +[WIP.Indexer.md](WIP.Indexer.md). + ## twinBASIC deviations from VBA to flag Add a `> [!NOTE]` callout or rewrite the affected section when source diverges. Known cases: diff --git a/WIP.Build.md b/WIP.Build.md index 7302c2543..e8b23fe3b 100644 --- a/WIP.Build.md +++ b/WIP.Build.md @@ -93,7 +93,7 @@ yet designed; the plan holds measurements and open questions for them. ### Tooling is JavaScript, and the three remaining `.py` files each have a reason -Everything under `scripts/`, `builder/`, `lib/`, `book/`, `eval/` and `wisdom/` is Node.js. +Everything under `scripts/`, `builder/`, `lib/`, `book/`, `eval/`, `wisdom/` and `indexer/` is Node.js. One trap: **a tool that rewrites a file must preserve its line endings byte-exactly.** Python's `Path.read_text` / `write_text` round-trip applies universal-newline translation, rewriting any LF file it touches to CRLF on Windows @@ -196,6 +196,8 @@ project file](#the-help-add-ins-project-file)). `wisdom/` — Discord knowledge-harvesting tool (three-phase: export → process → extract). Plans in `wisdom/PLAN-{1,2,3}.md`; implementation under `wisdom/`. Uses only Node.js built-in APIs. Running it is [WIP.Wisdom.md](WIP.Wisdom.md). +`indexer/` — package indexer: reads the twinBASIC packages (the built-in and default ones from the local install, or with `--release` from the GitHub release; the community ones from TWINSERV), parses their `.twin` sources and writes an `api.json` snapshot per package into a local git store. The build does not use it. Uses only Node.js built-in APIs. Its layout and commands are in `indexer/README.md`, and why it is built as it is in [WIP.Indexer.md](WIP.Indexer.md). It is under the lint, format and regex-safety gates like the other tooling folders; `sample.twinpack` and the committed snapshots are data, not scripts; the sync state lives in the gitignored store, not in the repository. + `eval/` — use-case evaluation of the developer documentation. `build_corpus.mjs` mirrors the repository with every non-prose file stubbed unreadable, so "documentation only" is a property of the tree rather than an instruction. `site_search.mjs` replays the site's real lunr index @@ -679,9 +681,8 @@ Three decisions about the build side, each with the alternative it rules out: with no page and put the docs' own layout (Array filed under Information, the Styles classes documented though declared `Private`) in the wrong place. - **The snapshot is committed, like `inter-metrics.json`, because making it needs a - twinBASIC install.** `scripts/build_package_api.mjs` exports the packages through - `scripts/lib/tb-packages.mjs` (shared with `census_attributes.mjs`), scans them with - `scripts/lib/twin-api.mjs`, and writes ~255 KB. `package-api.json` is a build *input*, so `check_tree_fresh.mjs` + twinBASIC install.** `node indexer/twin-index.mjs derive` reads the packages' `.twinproj` + files as `sync` does, scans them with `scripts/lib/twin-api.mjs`, and writes ~255 KB. `package-api.json` is a build *input*, so `check_tree_fresh.mjs` watches it; `symbol-baseline.json` is an output and is in its `IGNORED_FILES`. - **The heading scan is `indexOf`, not a regex.** `/]*)>([\s\S]*?)<\/h\1>/` is cubic in the regex gate's census, and it runs over every reference page; the scan @@ -707,8 +708,18 @@ out `-text`: a CRLF checkout of the copy would differ from the LF the build writ build. `scripts/check_symbol_index.mjs` is the gate on all three pieces, in `test.bat` and both -CI workflows: forty-six probes on fixtures, no tree, no install. The scanner's probes are -the traps the packages' `.twin` sources contain; the derivation's are each a rule the real +CI workflows: over a hundred probes on fixtures, no tree, no install. The scanner's probes are +the traps the packages' `.twin` sources contain, the community packages' included, and what +a declaration's record says (signature, attributes with their arguments, description, return +type, accessor, the project name). A block of them holds the forms an audit of `parseTwin` +found it losing without a word (lists after `Inherits` and `Implements`, a qualified CoClass +interface, `[Hidden(False)]`, modifier-named and initialised fields, brackets in a Type, a +result type before `Overrides`, an unmodified `Declare`, and the `#If` branches as +alternatives), each with the compiler's answer in its comment, checked with `tbrun --tests` +on BETA 1005. A second block holds the second step's: declarations joined by colons, +identifiers with characters past ASCII (the compiler takes every one as part of a name, a +no-break space included, so `\b` is no word boundary for them), a type-declaration +character on a name, and the condition each `#If` branch gives its members. The derivation's are each a rule the real site exercises only once or twice (`symbols:`, a section heading named like a member, the ellipsis the typographer puts in a Core H1). @@ -926,7 +937,7 @@ polynomial. Every one of the constructions classifies `safe`. ### The regex-safety gate [scripts/check_regex_safety.mjs](scripts/check_regex_safety.mjs) parses every -`.mjs` under `builder/`, `scripts/`, `lib/`, `book/`, `eval/` and `wisdom/` with +`.mjs` under `builder/`, `scripts/`, `lib/`, `book/`, `eval/`, `wisdom/` and `indexer/` with acorn, takes the regex literals *and* every `new RegExp(...)` whose arguments the source decides, and refuses any that can backtrack exponentially. In `test.bat` and both CI workflows; ~10 s, no browser, no built tree. Exit 0 no exponential diff --git a/WIP.Harness.md b/WIP.Harness.md index 63036f2bf..c8f0c8ba6 100644 --- a/WIP.Harness.md +++ b/WIP.Harness.md @@ -33,7 +33,7 @@ makes it the strongest available evidence for what is legal syntax. The tooling unpacks and packs in process, through `unpackProject` and `packTree` in [scripts/lib/tb-project.mjs](scripts/lib/tb-project.mjs), which call impexp's -`exportProject` and `importProject`: the census and `build_package_api`, `tbbuild --build`, +`exportProject` and `importProject`: the census, `tbbuild --build`, `tbrun`, `check_examples`, `sweep_attributes`, `bug_repro`'s `pack`, and the add-in and IDE lanes. The compiler executable's `export` verb writes the same tree: on BETA 995 the two unpack the install's sixteen packages and thirty-two samples to identical trees, 1,633 files @@ -139,7 +139,13 @@ comment in the file: - An **escaped identifier is spelled like an attribute**: `[_HiddenModule].Foo`, `[_MAX] = 0`. The tail after the `]` is what separates them. An Enum member may *be* one --- `Report.twin` declares `[ ]`, `[A4 Portrait]`, `[Letter Landscape]` as member - names. + names --- and **a bracket group alone on its line in an Enum body is always a member, + whatever is inside the brackets** (BETA 1005): `[Hidden]` is the member `Hidden`, + `E.[Hidden]` and `E.Hidden` are 0, `[Description("x")]` is a member of that name, and + `[NoSuchAttribute]` too. Code after the group (`[Hidden] Red = 5`) is refused, TB5182 at + `Red`. So the census records no attribute for such a group (`scripts/lib/census-scan.mjs`, + probes in `check_twin_parsers.mjs`); it once did for the 35 `[_]` members VBCCR's + `LongPtr` Enum declares in the samples. - **A comment can sit anywhere**: inline `/* voffset &H00A8*/ Property Get X()` before a declaration (DAO.twin), a trailing `' NOTE: ...` after a `]`, a whole `'` line *between* two attribute groups (VBA/Strings.twin; 142 sites), or a `#If` between an attribute and @@ -195,13 +201,15 @@ other 193 are refused at every site. tool that takes a hyphenated option once with each of them before trusting it. - **A control must be refused for a site to mean anything.** An unknown name is built at every site, and a site whose control *compiles* is voided. An Enum body accepts any own-line - `[...]` --- `[ClassId("guid")]` and `[Hidden(True)]` included, neither of which can be a - member name --- so what it does with the line is unchecked, and nothing accepted there is - evidence. Inline, `[Name] X = 1`, the compiler refuses every attribute, `[Hidden]` included, + `[...]` --- `[ClassId("guid")]` and `[Hidden(True)]` included --- because the line is a + member, not an attribute (BETA 1005: `E.[Description("x")]` compiles and is 0, so such a + group is a legal member name), and nothing accepted there is evidence. Inline, + `[Name] X = 1`, the line is the member `[Name]` followed by text the parser has no + handler for (TB5182, at `X`), so the compiler refuses every attribute, `[Hidden]` included, though the page documents it on an Enum member. An Enum member target is therefore reported - as one the sweep cannot test, which is true. (The mechanism is not established; only the - behaviour was measured.) A local variable is different: an own-line `[Name]` in a Sub is a - *call* statement, so that variant is dropped and only the inline one is kept. + as one the sweep cannot test, which is true. A local variable is different: an own-line + `[Name]` in a Sub is a *call* statement, so that variant is dropped and only the inline one + is kept. - **A probe that draws what the control draws is a refusal, whatever the code.** The signature compared has the attribute's name (whole word, any case) and the probe's own generated names (`S000062`, `S000062_U`) taken out, or the control's message never equals the probe's. diff --git a/WIP.HelpAddin.md b/WIP.HelpAddin.md index 9119e07e4..962722cee 100644 --- a/WIP.HelpAddin.md +++ b/WIP.HelpAddin.md @@ -729,10 +729,9 @@ WinNativeCommonCtls' 738, 294 --- inherited members that pages name in a sentenc documents `=`, `<>` and four more. A page's first heading `# \ and \= operators` must be written `\\=`, since `\=` is a markdown escape. - **From the packages:** [builder/package-api.json](builder/package-api.json), which - [scripts/build_package_api.mjs](scripts/build_package_api.mjs) writes from the packages of - an install, through the export of [scripts/lib/tb-packages.mjs](scripts/lib/tb-packages.mjs) - (shared with `census_attributes.mjs`) and a declaration scanner, - [scripts/lib/twin-api.mjs](scripts/lib/twin-api.mjs). It is committed, like + `node indexer/twin-index.mjs derive` ([indexer/lib/package-api.mjs](indexer/lib/package-api.mjs)) + writes from the packages of an install, read and parsed as the indexer's `sync` reads them, + through the declaration scanner [scripts/lib/twin-api.mjs](scripts/lib/twin-api.mjs). It is committed, like `inter-metrics.json`, because CI has no install. 1,379 types in 16 exports, 255 KB. It supplies each symbol's package, container and kind, lists the public symbols that have no page (documentation coverage as a side effect), and supplies each class's default @@ -758,8 +757,9 @@ WinNativeCommonCtls' 738, 294 --- inherited members that pages name in a sentenc to `VBA.Collection` and `tbIDE.IToolWindowsV1` to `tbIDE.ToolWindows`, 37 in all, which is P5's hover answer turned into a lookup. A lookup that takes the compiler's word has to map `_Collection` to `Collection`. -- **Regenerate** `package-api.json` with `node scripts/build_package_api.mjs` when the - reference is re-indexed against a newer build, and commit it with the pages. +- **Regenerate** `package-api.json` with `node indexer/twin-index.mjs derive` when the + reference is re-indexed against a newer build, and commit it with the pages and the + indexer's snapshots. **The gates.** `builder/symbol-baseline.json` lists every published URL, and a build that loses one fails --- the case it exists for is a reworded member heading, which moves an diff --git a/WIP.Indexer.md b/WIP.Indexer.md new file mode 100644 index 000000000..65d8409bf --- /dev/null +++ b/WIP.Indexer.md @@ -0,0 +1,222 @@ +# twinBASIC Documentation --- The Package Indexer + +`indexer/twin-index.mjs` turns the sources of the twinBASIC packages into `api.json` +snapshots and lists the documentation work a new build leaves. **What to do** is in +[WIP.md](WIP.md#the-package-indexer); how to run it and the format of `api.json` are in +[indexer/README.md](indexer/README.md); the published account is the +[Tools-Build entry](docs/Documentation/Tools-Build.md#twin-index). This file says **why it is +built the way it is**, and keeps the decisions and open items that still decide something. +History is in git. + +## The job + +After a new twinBASIC build, find what the reference has not yet said about it, without +anyone reading diffs of `.twin` files. Two commands, one after the other: + +- `sync` reads the packages (the built-in and default ones from the local install, the + contributed ones from TWINSERV), indexes each into an `api.json`, keeps sources and index + in the store, and copies the index of each documented package to the committed snapshots. +- `analyze` compares the store's `api.json` with the snapshot committed at `HEAD` and writes + `indexer/.packages/report.json`: symbols to create, to update, to flag for removal. + +`analyze` ends at the report. The pages are written by the per-symbol workflow +([WIP.Authoring.md](WIP.Authoring.md#per-symbol-workflow)), by hand or with the +`document-symbol` skill. + +## Where the state lives + +**The store, `indexer/.packages/`, is gitignored; the snapshots, `indexer/snapshots/`, are +committed.** The store is a git repository of its own, seeded with the committed snapshots as +its baseline commit, so that its history shows what changed in each package's API between +syncs. It keeps sources byte for byte (`core.autocrlf` off) and opens long paths +(`core.longpaths` on): some source names are over a hundred characters, more than Windows +opens from a deeply nested clone otherwise. + +**`analyze` reads its baseline from `HEAD`** (`git show HEAD:indexer/snapshots/...`), never +from the working tree. A `sync` against a newer build rewrites the working-tree snapshots, and +the baseline has to stay what the pages say until the pages and the new snapshots are +committed together. It reads with a buffer of 1 GB because VB's snapshot is larger than the +1 MB `execFile` default, which once made VB fall back to an audit. + +**The manifests live in the store, not in the repository.** They were committed once, and two +things went wrong. After a pull, the manifest said the new build had been read while the store, +which is not committed, held old sources or none, so `sync` printed "Up to date" over stale +data. And a `sync` limited to some packages (`--package`) moved the whole manifest to the new +build, so the packages it skipped looked current and were not. Now the manifest says only what +was last fetched, a clone has none, and its first `sync` reads every package. Whether a package +is current is judged from what the store holds: its sources, an `api.json` in the format this +indexer writes, of the version the source has now, written by the scanner this checkout has. + +**The fingerprint.** An `api.json` is a function of the sources *and of the code that reads +them*. The store records, beside each `api.json`, a SHA-256 over `scripts/lib/twin-api.mjs`, +`scripts/lib/twin-declarations.mjs`, `indexer/lib/api-model.mjs` and +`indexer/lib/vb6-preprocess.mjs` (line ends made alike, so a CRLF and an LF checkout agree). +Without it a scanner fix left every store's `api.json` "current" and `analyze` compared output +of the old scanner. With it, `sync` indexes the package again from the stored sources, with no +download and under the version it had. That is why the store keeps each package's `Settings` +file (the project name). The list is every module `api-model.mjs` imports, and the unit test +fails when an import is added and the list is not; `package-files.mjs` is not on it, because +its output is what the store keeps. + +**No user name in a committed file.** The manifest records the source (`install` or +`release`) and the BETA number, never the install's path, which holds a user name. Snapshot +paths are relative to `Sources/`. + +## One scanner + +**`parseTwin` in `scripts/lib/twin-api.mjs` is the only scanner**; the indexer has no lexer or +extractor of its own. It had them once, and they misread what the shared scanner reads right: +a field named for a keyword opened a container nothing closed, a `Public CoClass` was missed, +so were Delegates and a `/*Private*/ Module`, and the values of a nested Enum and the fields of +a nested Type were dropped. On the same input `parseTwin` found a superset. Two scanners also +drift: the symbol index (`builder/package-api.json`) and the indexer would disagree about +one package, and each fix would be made twice. The scanner is pinned by +`scripts/check_symbol_index.mjs`, `scripts/check_twin_parsers.mjs` and +`test/indexer.test.mjs`; its header comment lists the ways to misread this corpus. + +What the compiler answered, each asked of BETA 1005 and settled; the scanner follows it, and +the probes in `check_symbol_index.mjs` and the comments in `twin-api.mjs` hold the cases: + +- **An unmodified `Declare` is Public in a Module** (another module calls it) **and must be + Private in a Class** (TB5227), so it is never a Private member: in a Class it is an error. +- **`Dim x = 5`, `Public n& = 0` and a bare `Public r` are legal** without `As`, in a Module + and a Class: a Variant with the warning TB0003, unless a type character gives the type. A + Type takes no initialiser (TB5182) but a bare name. +- **`Abstract` and `Virtual` are not modifiers.** `Public Abstract Sub Y()` is a Variant + variable named `Abstract` followed by a Sub the parser refuses (TB5182). Thirteen more words + (`Shared`, `Partial`, `Overrides`, `Overloads`, `MustOverride`, `MustInherit`, + `NotInheritable`, `Iterator`, `Async`, `Naked`, `Unsafe`, `Extern`, `Inline`) are refused the + same way, and stay on the shared list until a source is found that needs them gone. + `Overrides` is a clause after a result type, never a modifier. +- **A generic constraint of any form is refused** (TB5069); the scanner reads past one, taking + the name before the parenthesis. +- **A bracket group alone on a line is a member in an Enum**, the attribute list of the next + field in a Type. In an Enum it is a value whatever it holds (`[_]`, `[Flat]`, `[A4 + Portrait]`, `[Hidden]`, `[Description("x")]`), and an attribute in front of a value on the + same line is refused (TB5182), so an Enum value cannot carry an attribute. In a Type or Union + `[Class] As Long` is the attribute `Class` and a field named `As`, which is refused (TB5182). +- **A colon joins statements, and each is a declaration of its own**: `Private x As Long: + Private y As Long`, `Sub P(): End Sub: Sub Q(): End Sub`, `Class C: End Class`, `Enum E: A: B: + End Enum`. A colon inside a string, brackets, parentheses or `:=` joins nothing. +- **Every character past ASCII is an identifier character**, the no-break space and emoji + included, so `\b` is no word boundary: `Public` followed by an accented letter is a name, not + the modifier. +- **A type-declaration character ends an unbracketed name and gives its type** on a function, + Property Get, delegate, Declare, variable, field and constant; a Sub, Property Let or Set and + Event refuse one (TB5204). + +Branches of an `#If` are read as alternatives that each start from the block structure the +`#If` found, nothing is evaluated, and branches that leave different blocks open are listed in +`problems`. + +## api.json format 3 and the `#If` merge + +`format` changes whenever the shape does, and `sync` indexes a package again when the store +holds another number. Format 3 merges the declarations in the branches of an `#If` +(`#If VBA7 Then ... As LongPtr #Else ... As Long`) into **one member with a `signatures` list**, +a condition on each. Before, the branches were separate members with no condition, so an +`#If` pair looked like overloads and a changed condition changed nothing the differ could see. +The differ reads the merged form, with the condition in front of each signature, so that a +changed condition, a new branch or an edited signature each modifies the one symbol of that +name and kind. The rules for what counts as an alternative (same block, name, kind, accessor +and Declare-ness, more than one condition; two declarations under one condition are +overloads) are in the README. + +**A change to the fingerprinted code or to `format` means regenerating the committed snapshots +from the build they record (`indexed_from`), in the same commit as the change.** If it does +not, `analyze` compares the new scanner's output with the old scanner's and reports the +scanner's changes as changes to the API. The BETA 997 snapshots were regenerated this way when +the scanner changed. It needs that build's install, `sync --ide `: +`--release` downloads only the newest release, and a plain `sync` reads the newest install, +which would write the snapshots of another build. + +## Decisions of the owner + +- **Snapshots are committed only for the documented packages, to start with**: 14 files, 3.2 MB + (about 350 KB compressed). The contributed packages and the extra CEF builds are indexed into + the store and left out of the repository. A package joins when its section does; `sync` + copies a snapshot for a documented package it indexed in that run. +- **They are taken from the build the reference was last indexed against, BETA 997** + (`beta-x-0997`), not from a newer one, so that the pages and the baseline agree and + `analyze` of a newer build shows the real distance. +- **The twinBASIC "install" is the release zip unzipped.** Its `packages//package.twinproj` + files are the zip's, so either gives the same sources and the same `api.json`, byte for byte. + The local install is therefore the default source and `--release` exists only for a machine + without one. Nothing in the tool starts the IDE. +- **`analyze` covers the documented packages by default**; `--all` takes everything in the + store, and `--package` and `--packages` name any. A package no section documents is audited + against no pages, so every public symbol of it is work: the report would otherwise be mostly + the contributed packages and the two spare CEF builds. +- **The group of a package is a constant, not something the install says.** `DEFAULT_PACKAGES` + in `twin-index.mjs` (`VBA`, `VBRUN`, `VB`: the three every project loads) are `default`; + everything else in the install is `built-in`, and everything from TWINSERV `contributed`. A + package the IDE starts to ship is `built-in` until someone adds it to the constant. The group + decides the store folder, the snapshot folder and the folder of the reference. +- **CEF is one snapshot, `cefPackage145`.** `cefPackage49`, `cefPackage109` and `cefPackage145` + are one source tree compiled against three CEF runtimes, and their `api.json` files differ + only in the `package` key (checked on BETA 997 and 1005). `analyze` compares a package with + its own snapshot, so naming all three would triple every CEF task and keep two copies of + 460 KB. If a build ever declares an API of its own, it needs its name in `package_name:` and + a snapshot. +- **CustomControls maps to two packages**, `CustomControls, CustomControlsPackage`. + `package_name:` replaces the folder-name default, so the folder's own name is listed when it + is a package too. The project name inside `api.json` is not always the package's: + `TwinBasicAssertions` is the project `Assert`, `AppGlobalClassObject` is + `AppGlobalClassProject`, and the three CEF builds are `cefPackage`. +- **A contributed package's `indexed_from` is the four-part TWINSERV version** + (`1.3.0.0`), because that is the `version` of its snapshot; a three-part one never matches + and `analyze` falls back to an audit. + +## `derive`: the symbol index's packages + +`builder/package-api.json` is the input of the help add-in's symbol index +([WIP.HelpAddin.md](WIP.HelpAddin.md#stage-3-the-symbol-index-generated-by-the-docs-build)). +`twin-index derive` writes it (`lib/package-api.mjs`), and `scripts/build_package_api.mjs`, +which wrote it before, is gone. That tool exported every package through the compiler +harness's impexp into a cache and read the exported folders; `derive` reads the `.twinproj` +files with the reader `sync` uses and parses them with `parsePackageFiles`, the parse +`buildApi` makes, so the two outputs come from one reading. Its output was byte-identical to +the old tool's at BETA 997, and it takes half a second with no cache. + +- **It reads the packages, not the snapshots or the store.** An `api.json` sorts types and + members by name and drops line numbers, and `apiSnapshot` needs declaration order: the + first declaration of a name gives a member its kind, and the file lists types in that + order. The snapshots also hold only the documented packages, and the store only what the + last `sync` read. So `derive` needs no `sync` first, and takes `--ide` and `--release` as + `sync` does. +- **It is keyed by project name**, `cefPackage` and `Assert`, where the snapshots are keyed + by package; each project lists its packages as `exports`. The three CEF builds must + declare one API, or `derive` exits 2. +- **It stays committed.** It is a build input `check_tree_fresh.mjs` watches, and CI has no + packages to derive it from. `derive --check` is outside every gate for the same reason. +- **A scanner change can change it**, as it can the snapshots. The `[ ]` Enum member (a + bracket group alone on a line is a member) gave VB's `ReportBaseCtl.ReportSizeConstants` + its 17 values at BETA 997, the first named `" "`, which the committed file lacked until it + was derived again. + +## Open items + +- **`diffApi` reports `attributes` as changed as well as `signature` when only a property's + accessors changed.** A symbol's attributes are compared as a list with one entry for each + declaration of the name, and an added Let adds an entry. The report's `reason` is still + right (`signature_changed`); `changes` lists both. +- **`#If` alternatives of Enum values and Type fields are not merged**, because their order is + part of what they mean, and **alternative types carry no condition**: two `Class` headers in + the branches of one `#If` are two types of one path, told apart as `#2` in file order, so a + changed condition cannot be seen. +- **The audit matches by name.** `auditCoverage` looks a symbol's name up among the titles of + every page in the section. A member documented inline on its container's page (the + `Assert` functions on `Exact.md`) shows as undocumented, and a name shared by several types + shows as documented once any page carries it. Update mode does not audit, but finds a + changed symbol's `doc_page` the same way. +- **Adopting a package whose store is already current writes no snapshot.** By reading `runSync`: + the snapshots are copied for the packages the run indexed, and a package that is up to date + is not among them. Documenting a package that `sync` has already read needs its store folder + removed first, or a build newer than the store's. +- **`indexer/TWINSERV-protocol.md` stays where it is, as reference beside the code.** It + describes a third party's HTTP API, which is neither a reason nor a decision, and folding 239 + lines of it into this note would bury both. Its one reader is `lib/twinserv-client.mjs`, which + calls only `query` and `download`; the rest (login, publish, version check, the IDE-to-compiler + commands) is there for whoever writes a publisher tool. It was reverse-engineered from + BETA 983 (June 2026) and keeps no copy of the IDE files it came from. Check it against the + current IDE before relying on anything but those two endpoints. diff --git a/WIP.md b/WIP.md index a9f87e341..ffdea2313 100644 --- a/WIP.md +++ b/WIP.md @@ -21,6 +21,7 @@ one covering what you are about to change. | run or change Wisdom, the Discord harvester | [WIP.Wisdom.md](WIP.Wisdom.md) --- the three-phase pipeline and the Phase 3 extract flow | | retake the documentation's IDE screenshots, or change the tool that takes them | [WIP.Screenshots.md](WIP.Screenshots.md) --- `shoot_docs.mjs`: the setups, cut-outs, the annotation overlay and the owner's decisions | | work on the IDE help add-in, or on testing IDE add-ins by machine | [WIP.HelpAddin.md](WIP.HelpAddin.md) --- the plan, the IDE facts it rests on, and the probes still open | +| find the documentation work a new twinBASIC build leaves, or change the package indexer (`indexer/`) | [WIP.Indexer.md](WIP.Indexer.md) --- the store and the snapshots, the one shared scanner, `api.json` format 3, the owner's decisions and the open items | | run a use-case evaluation round | [eval/README.md](eval/README.md) --- start every evaluator with `eval/run_case.mjs`, **never as a subagent**: a subagent inherits this session's `CLAUDE.md`, and with it this file, which is the answer key the corpus withholds | The rule that decides where a new note belongs: **this file says what to do, a @@ -365,7 +366,7 @@ is a build artifact. - **Edit the `.dot`, never the `.svg`**, and never set a diagram's `font-family` anywhere else. Graphviz sizes every box to the text *it* measured, and its WASM build has no font machinery at all --- it falls back to Times for any face it does not know. A face the layout never saw leaves labels painted outside their boxes. - `node scripts/check_dot_fit.mjs` is the gate on exactly that, and runs inside `check.bat`. Run it after touching any diagram. -- **A diagram whose boxes link** (a `URL` on a node or cluster, relative to the page) declares its nodes in reading order and sets `outputorder=nodesfirst`: that order is the Tab order. `builder/inline-diagram.mjs` makes it a group of links when it is inlined; `Tools-Groups.dot` is the example. +- **A diagram whose boxes link** (a `URL` on a node or cluster, relative to the page's rendered URL, which the browser resolves it against: a markdown link on the same page is first looked up against the source file's folder, so the two can be written differently, as on `Documentation/index.md`) declares its nodes in reading order and sets `outputorder=nodesfirst`: that order is the Tab order. Each linked box has a `tooltip`, a sentence or two on what it links to; without one the hover text is the raw label. `builder/inline-diagram.mjs` makes it a group of links when it is inlined; `Tools-Groups.dot` is the example. - **Scale a diagram whole, or not at all.** Stretching an SVG up inflates its labels; shrinking the text alone unmoors the labels from their boxes. - **A `@font-face` added to [docs/_sass/custom/_fonts.scss](docs/_sass/custom/_fonts.scss) needs its stack added to [docs/_sass/modules-dark.scss](docs/_sass/modules-dark.scss) as well**, and must stay inside the `emit-font-faces` mixin. The dark compilation re-emits its whole payload under two selectors at raised specificity: a face declared there would be invalid, and a stack left out there applies in light mode and silently does not in dark. - **Run the full a11y sweep after any change that moves type metrics.** An inline element's measured height is its font's content area, so `target-size` results move with the face. The fourteen-page sample can stay clean while the full sweep regresses. @@ -382,7 +383,7 @@ must never do: [WIP.Typography.md](WIP.Typography.md). ### Tooling is JavaScript -Everything under `scripts/`, `builder/`, `lib/`, `book/`, `eval/` and `wisdom/` is +Everything under `scripts/`, `builder/`, `lib/`, `book/`, `eval/`, `wisdom/` and `indexer/` is Node.js, and a new tool joins them there. Four files and one folder are not, each for a stated reason rather than by oversight: `scripts/impexp.py` is a published download offered to readers rather than tooling, `scripts/build_fonts.py` stays Python @@ -399,6 +400,39 @@ Node. After a change to its `src/`, pack `imagestrip.twinproj` again with `impex it with `IMAGESTRIP_EXE` set to its exe: no gate can build it, so nothing else will. The full accounting, and what the two ports gained, is in [WIP.Build.md](WIP.Build.md). +### The package indexer + +`indexer/twin-index.mjs` is how a session finds the documentation work a twinBASIC build +leaves. `sync` indexes every package's sources into an `api.json`; `analyze` compares the +current ones with the snapshots committed in `indexer/snapshots/` (those of the build each +section's `indexed_from` names) and lists the public symbols to create, update and remove, in +`indexer/.packages/report.json`; `derive` writes `builder/package-api.json`, the package half +of the build's symbol index. It needs a twinBASIC install (or `--release`, for a machine +without one), is outside every gate and CI, and is documented in +[Tools-Build](docs/Documentation/Tools-Build.md#twin-index) and +[indexer/README.md](indexer/README.md); [WIP.Indexer.md](WIP.Indexer.md) says why it is built +as it is. The rules: + +- **Run `analyze` to find what a page set lacks**, before and after a batch of pages; see the + [per-symbol workflow](WIP.Authoring.md#per-symbol-workflow). +- **Bump a section's `indexed_from` when its package is re-indexed**, and commit the pages, the + new `indexed_from`, the new snapshots and `derive`'s `builder/package-api.json` together. + `analyze` compares with the snapshot at `HEAD` and only when its `version` is the + `indexed_from` tag; a `sync` against a newer build rewrites the working-tree snapshots, and + committing them on their own, or the bump without them, drops the next `analyze` into audit + mode. +- **A change to `scripts/lib/twin-api.mjs`, `scripts/lib/twin-declarations.mjs`, + `indexer/lib/api-model.mjs` or `indexer/lib/vb6-preprocess.mjs`, or to `format` in + `api.json`, regenerates the committed snapshots in the same commit**, with `sync --ide` + naming the install of the build they record. Otherwise `analyze` reports the scanner's + changes as changes to the API. Run `derive --check --ide` on the same install too: a + scanner change can change `builder/package-api.json`. +- **There is one scanner, `parseTwin`.** A misread declaration is fixed in + `scripts/lib/twin-api.mjs` and pinned in `scripts/check_symbol_index.mjs`, never worked + around in `indexer/`. +- **A user name never reaches a committed file.** The manifests keep the BETA number, never an + install's path; keep a path out of a snapshot, a note and a commit message too. + ### The published docs assume manual work **Nothing under `docs/Documentation/` should require Claude, an agent or a skill to follow.** @@ -465,7 +499,7 @@ Why the report separates the wedged task from the merely blocked ones, and why - `build.bat` — runs `node builder\tbdocs.mjs --src docs --check-audit-index` (which implies `--check`) and produces three trees in one pass: the online copy at `_site/`, a `file://`-browsable copy at `_site-offline/`, and the sparse pagedjs source at `_site-pdf/`. The offline pass adds ~700 ms and the PDF pass adds ~150 ms on top of the ~2 s online build. Toggle `also_build_offline` / `also_build_pdf` in `_config.yml` (or pass `--no-offline` / `--no-pdf`) to skip a sibling output. `--check` adds ~1.7 s and runs the link + integrity check over the HTML while it is still in worker memory; `build.bat --no-check` gets a plain build. A build also writes the help add-in's archive, `add-in/Resources/HELP/site.zip` (see the help-archive bullet below); `--no-help-archive` skips it, and CI never passes it. - `serve.bat` — runs `tbdocs --serve`: initial build, then a long-lived process with watcher, debounced rebuilds, and SSE-driven browser auto-reload. Writes to `docs/_serve/` (disjoint from `build.bat`'s `_site*/`) and skips the offline + PDF passes — so a one-off `build.bat` for the PDF or offline mirror doesn't disturb the live preview. Ctrl+C to stop. - `check.bat` — the gates that read the built site: a freshness check that refuses a stale tree (`scripts/check_tree_fresh.mjs`), the DOT diagram fit check (`scripts/check_dot_fit.mjs`), the a11y sample-coverage check (`scripts/pick_a11y_sample.mjs --check`), then the accessibility check (`scripts/check_a11y.mjs`). The link + integrity check runs in `build.bat`. ~37 s. -- `test.bat` — the tests the *toolchain* has to pass: the publish-allowlist self-test (`scripts/check_publish_policy.mjs`), the gate-list check (`scripts/check_gate_lists.mjs`), the CI-workflow roster check (`scripts/check_ci_workflows.mjs`), the lint gate (`scripts/check_lint.mjs`), the site-search unit tests (`node --test test/search.test.mjs`), the markdown-plugin unit tests (`node --test test/render.test.mjs`), the date-formatter unit tests (`node --test test/strftime.test.mjs`), the bug-reproducer picture tests (`node --test test/png.test.mjs`), the embedded-image tests (`node --test test/compact-image.test.mjs`), the SVG screenshots' one-line test (`node --test test/svgshot.test.mjs`), `check_examples.mjs`'s probes (`node --test test/example-batches.test.mjs`), the port-claim unit tests (`node --test test/ports.test.mjs`), the add-in project-file tests (`node --test test/addin-project.test.mjs`), the regex-safety gate (`scripts/check_regex_safety.mjs`), the code-region gate (`scripts/check_code_regions.mjs`), the page-count drift-guard probes (`scripts/check_page_baseline.mjs`), the book-coverage probes (`scripts/check_book_coverage.mjs`), the symbol-index probes (`scripts/check_symbol_index.mjs`), the twinBASIC-scanner probes (`scripts/check_twin_parsers.mjs`), the attribute-sweep probes (`scripts/check_attribute_sweep.mjs`), the command-line probes and cases (`scripts/check_cli.mjs`), the pdf-lib shim comparison (`scripts/check_pdf_shims_equiv.mjs`), the impexp parity check (`scripts/check_impexp_parity.mjs`), and the axe source-patch verification (`scripts/check_axe_patch_equiv.mjs`). ~23 s, of which the regex-safety gate is ~10 s and the impexp check ~4 s. See [What belongs in test.bat rather than check.bat](WIP.Build.md#what-belongs-in-testbat-rather-than-checkbat). +- `test.bat` — the tests the *toolchain* has to pass: the publish-allowlist self-test (`scripts/check_publish_policy.mjs`), the gate-list check (`scripts/check_gate_lists.mjs`), the CI-workflow roster check (`scripts/check_ci_workflows.mjs`), the lint gate (`scripts/check_lint.mjs`), the site-search unit tests (`node --test test/search.test.mjs`), the markdown-plugin unit tests (`node --test test/render.test.mjs`), the date-formatter unit tests (`node --test test/strftime.test.mjs`), the bug-reproducer picture tests (`node --test test/png.test.mjs`), the embedded-image tests (`node --test test/compact-image.test.mjs`), the SVG screenshots' one-line test (`node --test test/svgshot.test.mjs`), `check_examples.mjs`'s probes (`node --test test/example-batches.test.mjs`), the port-claim unit tests (`node --test test/ports.test.mjs`), the add-in project-file tests (`node --test test/addin-project.test.mjs`), the package-indexer tests (`node --test test/indexer.test.mjs`), the regex-safety gate (`scripts/check_regex_safety.mjs`), the code-region gate (`scripts/check_code_regions.mjs`), the page-count drift-guard probes (`scripts/check_page_baseline.mjs`), the book-coverage probes (`scripts/check_book_coverage.mjs`), the symbol-index probes (`scripts/check_symbol_index.mjs`), the twinBASIC-scanner probes (`scripts/check_twin_parsers.mjs`), the attribute-sweep probes (`scripts/check_attribute_sweep.mjs`), the command-line probes and cases (`scripts/check_cli.mjs`), the pdf-lib shim comparison (`scripts/check_pdf_shims_equiv.mjs`), the impexp parity check (`scripts/check_impexp_parity.mjs`), and the axe source-patch verification (`scripts/check_axe_patch_equiv.mjs`). ~23 s, of which the regex-safety gate is ~10 s and the impexp check ~4 s. See [What belongs in test.bat rather than check.bat](WIP.Build.md#what-belongs-in-testbat-rather-than-checkbat). - `book.bat` — renders the PDF from `docs\_site-pdf\book.html` via `node book\render-book.mjs` into `docs\_pdf\twinBASIC Book.pdf`. Run `build.bat` first to populate `_site-pdf/`; `book.bat` refuses a tree older than its sources rather than rendering the previous book (see [The book refuses a stale source tree](WIP.Build.md#the-book-refuses-a-stale-source-tree)). - `examples.bat` — compiles the documentation's own twinBASIC code samples, every `tb` fence marked `check_build`, and reports the ones the compiler refuses against the line in the page they came from; a statement sample also marked `check_run` is built and run, and what it prints is checked against the page. Needs a twinBASIC install and Windows, so it is outside every gate and outside CI; ~110 s over the 1,197 samples marked (33 of them `check_run`). `--build` also builds each project that compiles clean, and `--llvm` builds it with LLVM (see [WIP.ExamplesBuild.md](WIP.ExamplesBuild.md)). Two modes need no compiler at all: `--census` classifies every fence and says how many classifiable ones are still unmarked, and `--report ` groups a saved `--propose --json` survey by diagnostic, section and unresolved name. `--propose` itself does compile. See [Compiling the reference's own code samples](#compiling-the-references-own-code-samples) and [WIP.ExamplesBuild.md](WIP.ExamplesBuild.md). @@ -478,7 +512,7 @@ Why the report separates the wedged task from the merely blocked ones, and why - `node scripts/bug_repro.mjs` --- the reproducer projects under `bugs//` for the entries of [BUGS-TO-REPORT.md](BUGS-TO-REPORT.md): `new`, `pack` (impexp, then the zip, written in Node), `compile`, `build` and `run` through `tbbuild` and `tbrun`, `vb6` (builds the optional `vb6/` VB6 project beside the twinBASIC one in a temp copy, runs its exe on a private desktop, and prints its `out.txt`; a project with a form or a user control is built without Unattended Execution, which VB6 refuses for it; `pack` zips its sources into `-vb6.zip`; VB6 is started only by `scripts/lib/vb6.mjs`, never from a shell; a graphical defect's `images` key makes `run` and `vb6` keep what the `PngDump` modules of `test/repro-templates/png/` draw as `images/-tb.png` and `-vb6.png`, with a `-compare.png` of both, tested by `test/png.test.mjs`), `file` (moves a filed entry and its reproducer to `bugs/filed//`; `file --marked` does every marked entry), and `verify`, which reads each `bugs/*/repro.json` and `bugs/filed/*/repro.json` and says whether the bug still reproduces on the newest beta; a `lane` reproducer names the `ide-test` or `addin-test` lane and tests that assert its bug, which `verify` runs last through that runner, a `probe` one a script under `scripts/`, and a `test` one its own `[TestCase]` Subs and compile cases through `tbrun --tests`, judged by the `PASS` lines of what is fixed and the `FAIL` lines of the bug (`new --template tests` starts one). **No reproducer should need a person at the IDE** (owner, 2026-10-05): a bug the IDE has to be operated for gets a lane test, and `manual` is left for what cannot be automated at all. It has no wrapper. **`verify` is run by a person only**, never by a gate or CI: it needs a twinBASIC install, like `examples.bat`. Default port 9440, and `--jobs N` takes N ports from there. Its command-line cases are in `scripts/lib/cli-cases.mjs`, and need no IDE. - `node scripts/vb6run.mjs ` and `node scripts/vb6run.mjs --docs [--only ]` --- build and run VB6 code, so that what a sample prints in twinBASIC can be compared with VB6: a stand-alone file of statements or a `.bas` module, or `--docs`, which builds the documentation's `check_run` fences in VB6 and says for each whether it prints what its page says (`same`, `differs`, `not VB6`, `error`, `refused`; a `projname=` group is built as a project of its own, its `slot=file` fences translated into VB6 `.cls` and `.bas` components). It has no wrapper and is outside every gate and outside CI: it needs VB6 (`--vb6`, `VB6_EXE`, or the standard install folder). **Never start `VB6.EXE` from a shell**: in Git Bash `/make` is rewritten as a path and VB6 answers with a modal box on the desktop; the tool spawns it from Node with an argument array, and a sample that calls `MsgBox` or `InputBox` or contains `End` is refused. Its `Debug.Print` rewrite and its file translation have probes in `test/example-batches.test.mjs`, and its command-line cases are in `scripts/lib/cli-cases.mjs`; both need no VB6. What it relies on is in [WIP.ExamplesBuild.md](WIP.ExamplesBuild.md). -Three generators sit outside that loop and produce committed artifacts rather than build output — none runs during a build, and none is needed for one. `python scripts/build_fonts.py` rebuilds the subset webfaces under `docs/assets/fonts/` and needs a network connection; `node scripts/build_dot_metrics.mjs` regenerates `builder/inter-metrics.json` from those webfaces and needs only a browser. See [Typography](#typography). `node scripts/build_package_api.mjs` regenerates `builder/package-api.json`, the packages' declared API that the build's symbol index (`tB/symbols.json`, for the IDE help add-in) is annotated from; it needs a twinBASIC install, so **run it when the reference is re-indexed against a newer build** and commit it with the pages. See [WIP.HelpAddin.md, Stage 3](WIP.HelpAddin.md#stage-3-the-symbol-index-generated-by-the-docs-build). +Three generators sit outside that loop and produce committed artifacts rather than build output — none runs during a build, and none is needed for one. `python scripts/build_fonts.py` rebuilds the subset webfaces under `docs/assets/fonts/` and needs a network connection; `node scripts/build_dot_metrics.mjs` regenerates `builder/inter-metrics.json` from those webfaces and needs only a browser. See [Typography](#typography). `node indexer/twin-index.mjs derive` regenerates `builder/package-api.json`, the packages' declared API that the build's symbol index (`tB/symbols.json`, for the IDE help add-in) is annotated from; it needs a twinBASIC install (or `--release`), so **run it when the reference is re-indexed against a newer build** and commit it with the pages and the snapshots. See [WIP.HelpAddin.md, Stage 3](WIP.HelpAddin.md#stage-3-the-symbol-index-generated-by-the-docs-build). ## Site integrity check @@ -490,7 +524,7 @@ build.bat && check.bat On the dev box that is ~4 s of build against ~37 s of check, of which the axe scan is ~20 s. [builder/PLAN-checks.md](builder/PLAN-checks.md) records how the link checker sits in the build's task graph and its cost; the axe follow-ons are designed there but not implemented. -**If the change touched `builder/`, `scripts/`, `lib/`, `book/`, `eval/`, `wisdom/`, `test/`, `add-in/`, the site's scripts in `docs/assets/js/`, a wrapper or a workflow, run `test.bat` as well** --- another ~23 s. Twenty-one of its twenty-four gates cannot be affected by an edit under `docs/` at all. **Three can.** `check_lint.mjs` lints the site's two scripts in `docs/assets/js/` along with the tooling. `check_gate_lists.mjs` reads `README.md` and every page under `docs/Documentation/`, so an edit to any developer page that states a gate count can fail it. **`check_code_regions.mjs` has two halves, and a content edit reaches only one.** Its corpus sweep reads `DOCS_DIR`, which is `/docs`, and tokenises every markdown file in it, so a page that provokes a rewrite into *altering* a code region fails it --- that half is content-dependent. Its fixed probes are not: they run against their own sources whatever the tree holds, and they cover the **mirror** fault, where a rewrite silently stops firing. The sweep cannot see that fault, because text the rewrite skipped is masked and restored unchanged and every region still matches. So run `test.bat` after adding an unusual code construct --- a fence whose contents include a fence marker, a 4-space indented block, an admonition wrapping a fence --- and read the built page as well, because for the mirror fault the gate is asserting that the mask and the admonition scan still work rather than checking your page: +**If the change touched `builder/`, `scripts/`, `lib/`, `book/`, `eval/`, `wisdom/`, `indexer/`, `test/`, `add-in/`, the site's scripts in `docs/assets/js/`, a wrapper or a workflow, run `test.bat` as well** --- another ~23 s. Twenty-two of its twenty-five gates cannot be affected by an edit under `docs/` at all. **Three can.** `check_lint.mjs` lints the site's two scripts in `docs/assets/js/` along with the tooling. `check_gate_lists.mjs` reads `README.md` and every page under `docs/Documentation/`, so an edit to any developer page that states a gate count can fail it. **`check_code_regions.mjs` has two halves, and a content edit reaches only one.** Its corpus sweep reads `DOCS_DIR`, which is `/docs`, and tokenises every markdown file in it, so a page that provokes a rewrite into *altering* a code region fails it --- that half is content-dependent. Its fixed probes are not: they run against their own sources whatever the tree holds, and they cover the **mirror** fault, where a rewrite silently stops firing. The sweep cannot see that fault, because text the rewrite skipped is masked and restored unchanged and every region still matches. So run `test.bat` after adding an unusual code construct --- a fence whose contents include a fence marker, a 4-space indented block, an admonition wrapping a fence --- and read the built page as well, because for the mirror fault the gate is asserting that the mask and the admonition scan still work rather than checking your page: ```sh build.bat && check.bat && test.bat @@ -550,6 +584,7 @@ wrapper: | `test.bat` | `test/example-batches.test.mjs` | `check_examples.mjs`'s probes, from `scripts/lib/example-batches.mjs`: batching, crash isolation through a fake lane, the canaries, the fence classifier and `check_run`'s refusals, expected output, dispatcher and marker reading, with no IDE. Run by `node --test` | | `test.bat` | `test/ports.test.mjs` | `scripts/lib/tb-ports.mjs` claims ports as the lane runners need: two claims at once are disjoint, a port in use and a live lock are passed over, a stale lock is taken over, a lock goes when its process exits. Run by `node --test` | | `test.bat` | `test/addin-project.test.mjs` | `builder/addin-project.mjs` packs the help add-in's project file from exactly the files git tracks in `add-in/`, as the working folder holds them, never `Resources/HELP/`, with the index it is given, the same bytes from a CRLF and an LF checkout, into every tree it is given. Needs git. Run by `node --test` | +| `test.bat` | `test/indexer.test.mjs` | the package indexer (`indexer/lib/`) keeps its promise that a snapshot changes only when the API does: VB6 files named by `VB_Name`, else by their file; a package's files in code-unit order with the `Packages` folder left out; `api.json` types sorted by path with `in` for nested ones, members by name, Enum values and Type fields in declaration order, a type character stripped from an unbracketed name only, the same bytes from the same files in any order; `diffApi` and `auditCoverage` on small hand-made inputs. Reads `indexer/sample.twinpack`; no network, git or install. Run by `node --test` | | `test.bat` | `check_pdf_shims_equiv` | the book's pdf-lib shims write what stock pdf-lib writes: one document written by the gate and one built with `PDFDocument.create` are each saved both ways in child processes, and each pair of files is compared object by object with streams inflated and their cross-reference entries checked; every shim must run, and the members of pdf-lib the shims patch must be those its `PATCHES` lists, each run in one document or the other unless marked there as not reached; each of them must also be in the target table its shim passes to `checkTargets`, which `book/lib/shim-targets.mjs` records for the side to ask | | `test.bat` | `check_impexp_parity` | `impexp.mjs` and `impexp.py` pass the same built-in tests, and one sequence of commands through each gives the exit code each command is there for, the same output and the same files. Without Python it reports itself skipped and passes, but fails when `CI=true` | | `test.bat` | `check_publish_policy`, `check_gate_lists`, `check_page_baseline`, `check_book_coverage`, `check_axe_patch_equiv` | the gates on the gates | diff --git a/biome.jsonc b/biome.jsonc index 731f7f8bd..c41a991bd 100644 --- a/biome.jsonc +++ b/biome.jsonc @@ -16,6 +16,7 @@ "book/lib/progress-handler.js", "eval/**/*.mjs", "wisdom/**/*.mjs", + "indexer/**/*.mjs", "test/**/*.mjs", "docs/assets/js/*.js", // Vendored code, and the two pagedjs-cli ports the review treats as @@ -130,9 +131,10 @@ "**/book/**", "**/eval/**", "**/wisdom/**", + "**/indexer/**", "**/test/**" ], - "message": "lib/ must not import builder/, scripts/, book/, eval/, wisdom/ or test/." + "message": "lib/ must not import builder/, scripts/, book/, eval/, wisdom/, indexer/ or test/." } ] } diff --git a/builder/package-api.json b/builder/package-api.json index af09bd18f..d15393116 100644 --- a/builder/package-api.json +++ b/builder/package-api.json @@ -324,7 +324,7 @@ {"name":"ReportBaseCtl","kind":"class","public":false,"inherits":["VB.BaseForm","VB.GraphicsBase"],"members":{"PaperColor":"field","InternalControlSections":"field","TopMost":"field","Opacity":"field","TransparencyKey":"field","HelpContextID":"field","RightToLeft":"field","Activate":"event","Deactivate":"event","Initialize":"event","Load":"event","Click":"event","DblClick":"event","GotFocus":"event","KeyDown":"event","KeyPress":"event","KeyUp":"event","LostFocus":"event","MouseDown":"event","MouseMove":"event","MouseUp":"event","BeforePaintSection":"event","Resize":"event","Terminate":"event","Unload":"event","QueryUnload":"event","DPIChange":"event","Recordset":"property","ZoomStep":"field","Page":"field","Pages":"field","RecordNum":"field","ZoomPercent":"field","ZoomAutoFit":"field","PixelsReportWidth":"field","PixelsReportHeight":"field","PixelsLeftMargin":"field","PixelsRightMargin":"field","PixelsTopMargin":"field","PixelsBottomMargin":"field","DpiScaleFactorX":"property","DpiScaleFactorY":"property","Count":"property","Icon":"property","Move":"sub","SetFocus":"sub","InternalEnumerator":"function","hDC":"property","ChangePageSize":"property","PrintReport":"sub","MinWidth":"property","MinHeight":"property","MaxWidth":"property","MaxHeight":"property","Width":"property","Height":"property"}}, {"name":"ZoomAutoFitConstants","kind":"enum","in":"ReportBaseCtl","values":["vbZoomAutoFit","vbZoomAutoFitOnce","vbZoomAutoFitNever"]}, {"name":"Expression","kind":"type","in":"ReportBaseCtl","public":false}, - {"name":"ReportSizeConstants","kind":"enum","in":"ReportBaseCtl","values":[]}, + {"name":"ReportSizeConstants","kind":"enum","in":"ReportBaseCtl","values":[" ","A4 Portrait","A4 Landscape","A3 Portrait","A3 Landscape","A5 Portrait","A5 Landscape","Letter Portrait","Letter Landscape","Tabloid Portrait","Tabloid Landscape","Legal Portrait","Legal Landscape","Statement Portrait","Statement Landscape","Executive Portrait","Executive Landscape"]}, {"name":"PageRangeConstants","kind":"enum","in":"ReportBaseCtl","values":["rptRangeAllPages","rptRangeFromTo"]}, {"name":"Report","kind":"class","control":true,"inherits":["VB.ReportBaseCtl"]}, {"name":"ShapeBaseCtl","kind":"class","public":false,"inherits":["VB.BaseControlWindowlessNoDrag"],"members":{"BackColor":"field","BackStyle":"field","BorderColor":"field","BorderStyle":"field","BorderWidth":"field","DrawMode":"field","FillColor":"field","FillColorAlt":"field","FillStyle":"field","Shape":"field","RoundedCornerSize":"field","VariationA":"field","VariationB":"field","VariationC":"field","Angle":"field","Initialize":"event","New":"sub","Parent":"property","_Default":"property"}}, diff --git a/builder/symbols.mjs b/builder/symbols.mjs index 1b653759e..0f7efa2d5 100644 --- a/builder/symbols.mjs +++ b/builder/symbols.mjs @@ -9,7 +9,7 @@ // single-file ones do not, and VBA's package page is `/tB/Packages/VBA` with // neither -- or that plus the id the build gave a heading, read from the // rendered HTML and never computed a second time. What the pages cannot say -// comes from builder/package-api.json, which scripts/build_package_api.mjs +// comes from builder/package-api.json, which `indexer/twin-index.mjs derive` // writes from the packages an install ships: the kind of a member that has a // page of its own, the values of an enumeration, which type declares a member // a page documents, and the interface a CoClass's members are declared on. A diff --git a/builder/tbdocs.mjs b/builder/tbdocs.mjs index 550e3fe74..a59e8f125 100644 --- a/builder/tbdocs.mjs +++ b/builder/tbdocs.mjs @@ -969,7 +969,7 @@ const TASKS = { if (err.code !== "ENOENT") throw err; throw new Error( "builder/package-api.json is missing, and the symbol index needs it. " + - "Restore it from git, or regenerate it with node scripts/build_package_api.mjs", + "Restore it from git, or regenerate it with node indexer/twin-index.mjs derive", ); } const pages = symbolPages(state.pages); diff --git a/docs/Documentation/Authoring.md b/docs/Documentation/Authoring.md index 8f675ffc6..7119e30eb 100644 --- a/docs/Documentation/Authoring.md +++ b/docs/Documentation/Authoring.md @@ -8,9 +8,9 @@ permalink: /Documentation/Development/Authoring # Authoring Pages {: .no_toc } -The starting point for writing or editing a documentation page. It covers the anatomy of a page --- frontmatter, headings, formatting, prose style, attribution, and links --- so a first contribution matches the rest of the site without a second editing pass. For the build-and-preview workflow that turns markdown into HTML, see [Building and Deployment](Building); for the stable URL contract a page's `permalink` must honour, see [Permanent Links](Permanent-Links). +The starting point for writing or editing a documentation page: frontmatter, headings, formatting, prose style, attribution and links. To build and preview, see [Building and Deployment](Building). For the URL contract a page's `permalink` must honour, see [Permanent Links](Permanent-Links). -The surest guide is an existing page of the same kind. Open the nearest neighbour to what you are writing --- a sibling in the same sidebar folder --- and match its shape. The rules below are the conventions those pages already follow. Two worked reference pages to imitate: [`Const.md`](https://github.com/twinbasic/documentation/blob/main/docs/Reference/Core/Const.md) and [`Dim.md`](https://github.com/twinbasic/documentation/blob/main/docs/Reference/Core/Dim.md). +The best guide is an existing page of the same kind. Open a sibling in the same sidebar folder and match its shape. Two worked reference pages to imitate: [`Const.md`](https://github.com/twinbasic/documentation/blob/main/docs/Reference/Core/Const.md) and [`Dim.md`](https://github.com/twinbasic/documentation/blob/main/docs/Reference/Core/Dim.md). * TOC goes here {:toc} @@ -24,20 +24,17 @@ The documentation is grouped by area, each a folder under `docs/`: - **Features** --- articles on individual twinBASIC language features. - **LLVM** --- compiling with the LLVM back end: turning it on, its options, and its current limitations. - **IDE** --- the IDE user guide. -- **Documentation** --- this section: the toolchain reference and the authoring guide you are reading now. +- **Documentation** --- this section: the toolchain reference and this authoring guide. -Where a page sits in the source tree matters less than its `permalink`, which fixes the page's URL independently of the folder the file lives in. Get the permalink right (next section) and the file can move without breaking a single link. +A page's `permalink` fixes its URL independently of the folder the file lives in. With the right permalink (next section) the file can move without breaking a link. ## What may live in docs/ Anything under `docs/` that is not a page is copied into the published site unchanged, at a URL matching its path. That is how images, fonts and stylesheets -get published, and it used to be how a stray file got published too --- a `.bak` -left beside a page, a `.twin` sample, a scratch `.md`, a `Thumbs.db` the shell -wrote without being asked. All of them went to a public URL on a green build. - -The build now refuses anything that is not a publishable type, names the file, -and stops before writing: +get published. The build refuses anything that is not a publishable type (a +`.bak` beside a page, a `.twin` sample, a scratch `.md`, a `Thumbs.db`), names +the file, and stops before writing: ``` 1 file would be published from docs but is not a publishable type: @@ -49,76 +46,56 @@ and stops before writing: The types a source file may use: `.html`, `.png`, `.jpg`, `.jpeg`, `.gif`, `.svg`, `.woff2`, `.css`, `.js`, `.txt`, and `CNAME` at the site root. Pages -(`.md` with frontmatter) are rendered rather than copied, so they are not -affected. - -Three things follow. +(`.md` with frontmatter) are rendered rather than copied. **A file you do not want published does not belong under `docs/`,** unless a -pattern in `_config.yml`'s `exclude:` already covers it. Source that sits beside -the pages it produces is excluded that way, at any depth in the tree: `.scss` -stylesheets, `.dot` diagram sources, and the Affinity `.af` screenshot sources -under `_Images/`. So is everything underscore-prefixed at the root of `docs/` ---- `_config.yml` and `_book.yml`, the `_sass/` stylesheet tree, and the build's -own output trees (`_site/`, `_site-offline/`, `_serve/`, `_pdf/` and their -variants). Scratch notes, editor backups and working documents have no such -exemption --- keep them outside the tree. - -That root-level underscore rule is two patterns --- `_*` for the root's own -files, `_*/**` for the contents of its folders --- and where it stops matters to -anyone documenting a COM interface. Both are matched against a -file's path relative to `docs/` under fast-glob semantics, where `*` does not -cross a directory separator, so `_*/**` reaches into a folder only when the -underscore sits at the top level. A `docs/_Config/` is therefore dropped, and -every page in it disappears without a message: excluded files are not missing -files, and the build cannot report on a page it never saw. A -`docs/Reference/Built-In/SomePackage/_Config/` is untouched, and so is an -underscore-prefixed *page* at any depth, such as +pattern in `_config.yml`'s `exclude:` covers it. These are excluded at any depth: +`.scss` stylesheets, `.dot` diagram sources, and the Affinity `.af` screenshot +sources under `_Images/`. So is everything underscore-prefixed at the root of +`docs/` --- `_config.yml` and `_book.yml`, the `_sass/` tree, and the build's +output trees (`_site/`, `_site-offline/`, `_serve/`, `_pdf/` and their +variants). Keep scratch notes, editor backups and working documents outside the +tree. + +The root-level underscore rule is two patterns --- `_*` for the root's own +files, `_*/**` for the contents of its folders. `*` does not cross a directory +separator, so `_*/**` reaches into a folder only when the underscore sits at the +top level. A `docs/_Config/` is therefore dropped, and every page in it +disappears without a message. A `docs/Reference/Built-In/SomePackage/_Config/` +is untouched, and so is an underscore-prefixed *page* at any depth, such as `Reference/Built-In/CustomControls/Framework/_CustomControlContext.md`. -The rule is scoped that narrowly because a wider one did real damage. It was -once `**/_*/**`, which matches an underscore folder anywhere in the tree, and it -swallowed all 37 pages under -`Reference/Built-In/AppGlobalClassObject/_App/` --- the twinBASIC interface is -named `_App`, following the COM convention for a hidden interface, and the -folder is named after the interface. Nothing reported the loss. An interface or -class whose name begins with an underscore is documented exactly like any other, -wherever it sits in the tree; the one placement to avoid is an underscore-named -folder directly under `docs/`. - -**A `.md` the build reports as unpublishable is a file with no frontmatter block -at all.** Either it genuinely has none --- a scratch note, a README --- or -something precedes the opening `---`, which has to be the first line. A blank -line before it is enough. A file in that state is not a broken page: without the -refusal it would be copied out and served as raw markdown, frontmatter keys and -all, which is how one package index page was published for months. - -Two neighbouring faults are *not* this, and knowing which you have saves a -search: - -- **A UTF-8 BOM is handled.** Windows editors add one without asking, and it - used to be exactly this failure --- it sits in front of the `---` and the - parser then reports no frontmatter. The build strips it before parsing, so a - BOM'd page renders normally and never reaches this message. -- **Malformed YAML inside the block reports itself.** It does not fall through - to this check; it aborts with `Failed to parse frontmatter in ` and the - parser's own line and column. - -**If a file genuinely belongs on the published site,** there are two ways to -allow it, and picking the wrong one quietly undoes the allowlist. The question is -whether you are adding a *type* or a *file*. +The rule is narrow because a wider one, `**/_*/**`, once swallowed all 37 pages +under `Reference/Built-In/AppGlobalClassObject/_App/` (the interface is named +`_App`, following the COM convention for a hidden interface). Document an +interface or class whose name begins with an underscore like any other; avoid +only an underscore-named folder directly under `docs/`. + +**A `.md` the build reports as unpublishable has no frontmatter block.** Either +it has none --- a scratch note, a README --- or something precedes the opening +`---`, which must be the first line. A blank line before it is enough. Without +the refusal, the file would be served as raw markdown, frontmatter keys and all. + +Two neighbouring faults are *not* this: + +- **A UTF-8 BOM is handled.** Windows editors add one without asking. The build + strips it before parsing, so a BOM'd page renders normally. +- **Malformed YAML inside the block reports itself.** The build aborts with + `Failed to parse frontmatter in ` and the parser's own line and column. + +**If a file belongs on the published site,** there are two ways to allow it, and +the wrong one quietly undoes the allowlist. Decide whether you are adding a +*type* or a *file*. A **type** is an extension the site will go on using --- a new image format, a new -font format --- where the next file of that kind should publish without anyone -having to think about it. Add the extension to `SOURCE_EXTENSIONS` in +font format --- so the next file of that kind publishes without further thought. +Add the extension to `SOURCE_EXTENSIONS` in [`builder/publish-policy.mjs`](https://github.com/twinbasic/documentation/blob/main/builder/publish-policy.mjs) -in the same commit as the file. That is a deliberate one-line edit, and it is the -point of the rule: the decision gets made once, by the person who knows they are -making it. +in the same commit as the file. A **file** is one particular download or sample whose extension you would not -want blessed everywhere --- a `.py` script, a `.json` data file, a `.zip`. -Declare it in `_config.yml`'s `bundle_extra`, which spells out a source and a +want allowed everywhere --- a `.py` script, a `.json` data file, a `.zip`. +Declare it in `_config.yml`'s `bundle_extra`, which gives a source and a published path and exempts that exact path rather than the extension: ```yaml @@ -129,47 +106,39 @@ bundle_extra: `src` is resolved against `docs/`, so it may point outside the tree --- the two real entries publish the [Import/Export Tool](../../Features/Packages/Import-Export-Tool) -from `scripts/`, where it is maintained and run, instead of requiring a second -copy under `docs/` that would drift from it. `dest` is the path in the built -site, and it is what the page links to. A `src` that does not exist aborts the -build, so a typo there is not silent either. - -Between the two, **prefer `bundle_extra` unless you really are adding a type.** -Widening `SOURCE_EXTENSIONS` to `.json` so that one download can ship makes every -stray `.json` under `docs/` publishable again --- a `docs/secrets.json` would -reach a public URL, which is the whole class of accident the allowlist exists to -refuse, reintroduced by the person fixing a build failure. +from `scripts/`, where it is maintained, with no second copy under `docs/`. +`dest` is the path in the built site, and the page links to it. A `src` that +does not exist aborts the build. + +**Prefer `bundle_extra` unless you are adding a type.** Widening +`SOURCE_EXTENSIONS` to `.json` for one download makes every stray `.json` under +`docs/` publishable again --- a `docs/secrets.json` would reach a public URL. `scripts/check_publish_policy.mjs` asserts that a `bundle_extra` exemption stays pinned to its declared path and never leaks to the extension. -The other half of this --- how the refusal is enforced --- is on the build page. -[What the build refuses to publish](Building#what-the-build-refuses-to-publish) -covers the two inventories the policy sweeps (the source files on disk, before -anything is written, and what each output tree is about to receive), why a -finding there aborts the build outright where a broken link only sets an exit -code, and how that self-test proves the allowlist can still refuse something -rather than having been widened until it refuses nothing. +How the refusal is enforced is in +[What the build refuses to publish](Building#what-the-build-refuses-to-publish). ## Frontmatter and permalinks Every page opens with a YAML frontmatter block. The keys that matter: - **`title`** --- the page's name in the sidebar, breadcrumb, and search results. -- **`parent`** (and sometimes **`grand_parent`**) --- the title of the nav parent. The build aborts if this does not resolve to exactly one page, so a typo is caught at build time rather than silently dropping the page from the sidebar or attaching it under the wrong branch. -- **`permalink`** --- **required on every page.** The page's stable URL, and the contract the IDE help system relies on. The build aborts if it is missing rather than deriving one from the file path: this tree deliberately does not mirror its URLs (`Reference/Built-In/CEF/` publishes at `/tB/Packages/CEF/`), so a derived URL would be wrong, and wrong silently. Reference pages follow a fixed scheme by section: a core keyword is `/tB/Core/`, a library symbol is `/tB/Packages//...`, and so on. [Permanent Links](Permanent-Links) documents the full scheme. +- **`parent`** (and sometimes **`grand_parent`**) --- the title of the nav parent. The build aborts if this does not resolve to exactly one page, so a typo cannot drop the page from the sidebar or attach it to the wrong branch. +- **`permalink`** --- **required on every page.** The page's stable URL, and the contract the IDE help system relies on. The build aborts if it is missing: this tree does not mirror its URLs (`Reference/Built-In/CEF/` publishes at `/tB/Packages/CEF/`), so a URL derived from the file path would be wrong. A core keyword is `/tB/Core/`, a library symbol is `/tB/Packages//...`, and so on; [Permanent Links](Permanent-Links) documents the full scheme. - **`nav_order`** --- optional integer that orders the page among its siblings in the sidebar. - **`has_toc`** --- set to `false` to suppress the automatic list of child pages the template appends to a page that has any; see [Folder-style pages](#folder-style-pages-and-has_toc). Every page that sets it sets it to `false`. -- **`redirect_from`** --- optional. List any earlier URL the page has moved from, so existing links keep resolving. Each entry becomes a small stub page at that URL, so two rules apply and the build aborts naming both files if either is broken: a `redirect_from` entry may not point at a URL some page already publishes at, and no two pages may claim the same one. +- **`redirect_from`** --- optional. List any earlier URL the page has moved from, so existing links keep resolving. Each entry becomes a small stub page at that URL. The build aborts, naming both files, if an entry points at a URL some page already publishes at, or if two pages claim the same one. - **`vba_attribution`** --- set to `true` only on pages adapted from the VBA-Docs source; see [Attribution](#attribution). -- **`nav_exclude`**, **`sitemap: false`**, **`search_exclude: true`** --- optional opt-outs, each from exactly one thing: the sidebar, `sitemap.xml`, and the search index. They are independent; a page that should be unlisted everywhere sets all three. The build's own link check honours the last two, so a page that opts out is not then reported as missing from the index it opted out of. +- **`nav_exclude`**, **`sitemap: false`**, **`search_exclude: true`** --- optional opt-outs from the sidebar, `sitemap.xml` and the search index. They are independent; a page that should be unlisted everywhere sets all three. The link check honours the last two, so an opted-out page is not reported as missing from the index. - **`symbols`** --- optional, and rare. The names a page documents, when its title and first heading cannot say; see [Naming what a page documents](#symbols). - **`index`**, **`index_also`** --- optional. Terms a reader searches for that should find this page, as a book's index lists them; see [Index entries for the site search](#index-entries-for-the-site-search). -A package's `index.md` may also carry **`indexed_from`**, **`exclude_from_docs`** and **`exclude_kinds`**. Those are provenance for the authoring pass: they record which twinBASIC build the package download was indexed against, and what was deliberately left undocumented, so a later re-index can tell a genuine gap from a deliberate omission. The build reads `exclude_from_docs` for one thing only, to leave those names out of the [symbol index](Building#the-symbol-index)'s count of undocumented symbols; none of the three reaches the HTML. **Leave them in place**, and bump `indexed_from` in the same commit if you re-index a package against a newer build. +A package's `index.md` may also carry **`package_name`**, **`indexed_from`**, **`exclude_from_docs`** and **`exclude_kinds`**. They record which packages the section documents, which twinBASIC build the package download was indexed against and what was excluded from the docs, so a later re-index can tell a gap from an exclusion. `package_name` names the packages as the [package indexer](Tools-Build#twin-index) names them, and is needed only where the section's folder is named otherwise. The build uses `exclude_from_docs` only to leave those names out of the [symbol index](Building#the-symbol-index)'s count of undocumented symbols; none of the four reaches the HTML. **Leave them in place**, and bump `indexed_from` in the same commit if you re-index a package against a newer build. -Any key the build does not recognise is simply inert --- nothing iterates frontmatter generically, so an unknown key is never emitted into the page. A key whose *value* is not valid YAML is a different matter: the build aborts with `Failed to parse frontmatter in `, quoting the YAML parser's own line and column. +An unrecognised key is inert and never emitted into the page. A key whose *value* is not valid YAML aborts the build with `Failed to parse frontmatter in `, quoting the parser's own line and column. -A value that is valid YAML can still mean less than it shows. YAML reads a `#` after a space, and everything after it, as a comment, so `title: Input #` titles the page *Input*. The build warns about any value left unquoted that ends in `#`, naming the file and line, and goes on building. Quote the value --- `title: "Input #"` --- to keep the `#`. Quoting also silences the warning where YAML would have kept the `#` anyway, as in `C#`. +A value that is valid YAML can still mean less than it shows. YAML reads a `#` after a space, and everything after it, as a comment, so `title: Input #` titles the page *Input*. The build warns about any unquoted value that ends in `#`, naming the file and line. Quote the value --- `title: "Input #"` --- to keep the `#`. Quoting also silences the warning where YAML would have kept the `#` anyway, as in `C#`. A minimal reference-page header: @@ -191,46 +160,42 @@ holds `index.md` and `EnvironmentOptions.md`, and publishes at `/tB/Packages/CEF/CefBrowser/` and `/tB/Packages/CEF/CefBrowser/EnvironmentOptions`. {{tbdocs:folderStyleIndexes}} pages under `Reference/` are `index.md` files of this shape, {{tbdocs:folderStyleSlashPermalinks}} of them declaring a permalink -that ends in a slash, and the layout is written down nowhere but in them. +that ends in a slash. -**The filename carries no meaning; the permalink does all the work.** `index.md` -is not special to the build --- nothing in `discover` looks at basenames except to -sort by them --- and the source folder is not the URL folder either: that CEF page -lives at `Reference/Built-In/CEF/CefBrowser/` and publishes under -`/tB/Packages/CEF/`. What decides the shape of the output is one character: +**The filename carries no meaning; the permalink decides the output.** `index.md` +is not special to the build, and the source folder is not the URL folder: that +CEF page lives at `Reference/Built-In/CEF/CefBrowser/` and publishes under +`/tB/Packages/CEF/`. One character decides the shape of the output: permalink: /tB/Packages/CEF/CefBrowser/ -> tB/Packages/CEF/CefBrowser/index.html permalink: /tB/Packages/CEF/CefBrowser -> tB/Packages/CEF/CefBrowser.html A trailing slash makes the page an `index.html` inside a folder of that name; -without one it is a file beside its siblings. That is not cosmetic, because -relative links resolve against the rendered URL: the trailing-slash form has one -more URL segment than the other, so every `../` count on the page and every link -written *to* it changes with the slash. Pick the form first and do not change it -afterwards. +without one it is a file beside its siblings. Relative links resolve against the +rendered URL, and the trailing-slash form has one more URL segment, so every +`../` count on the page and every link written *to* it changes with the slash. +Pick the form first and do not change it afterwards. The frontmatter follows the same nesting. The index declares `parent: Package`; each sibling declares `parent: ` and `grand_parent: Package`, so the sidebar nests one level deeper. A sibling addresses its own index as `[CefBrowser](.)`, and a section of it as `[Create](.#create)`. -**Set `has_toc: false` on every page in the group.** It is a real key the template -reads, and it suppresses `renderChildrenNav` --- the automatic "Table of contents" -heading and list of child pages that the template otherwise appends to a parent -page, after its own content. On a reference index that list repeats the members -the page already introduces in prose, so every one of the 238 pages that sets the -key sets it to `false`; it is never `true` anywhere on the site. The whole -`AppGlobalClassObject` package is the precedent to copy --- its `index.md` and all -37 pages under `_App/` carry it. +**Set `has_toc: false` on every page in the group.** It suppresses the automatic +"Table of contents" heading and list of child pages that the template otherwise +appends to a parent page. On a reference index that list repeats the members the +page already introduces, so every one of the 238 pages that sets the key sets it +to `false`. The `AppGlobalClassObject` package is the example to copy --- its +`index.md` and all 37 pages under `_App/` carry it. -Note what it does not do. `has_toc` has nothing to do with the in-page table of -contents, which is the separate `* TOC goes here` / `{:toc}` marker, or with -`{: .no_toc }` on the page title, which keeps the title out of that list. A page -can use all three, and most reference index pages do. +`has_toc` is unrelated to the in-page table of contents, which is the +`* TOC goes here` / `{:toc}` marker, and to `{: .no_toc }` on the page title, +which keeps the title out of that list. A page can use all three, and most +reference index pages do. ## Page skeleton -A reference page follows a predictable shape: the title, a one-line summary, the syntax, the parameters, any remarks, then an example and the related links. The parameters use a definition list --- a term line, then a `:` line indented beneath it: +A reference page has the title, a one-line summary, the syntax, the parameters, any remarks, then an example and the related links. Parameters use a definition list --- a term line, then a `:` line beneath it: # MyFunction {: .no_toc } @@ -259,56 +224,56 @@ A reference page follows a predictable shape: the title, a one-line summary, the ## Heading levels -The page title is a single `#` (a *chapter* --- a page may legitimately have more than one). Top-level sections such as **Example** and **See Also** are `##`; subsections are `###`. +The page title is a single `#` (a *chapter*; a page may have more than one). Top-level sections such as **Example** and **See Also** are `##`; subsections are `###`. -Do **not** jump from `#` straight to `###`. That old "house style" --- an h1 followed directly by an h3 --- is a heading-order defect on the built site, where headings must not skip a level. Older pages that still do it are repaired at build time by the `headingLevelNormalizePlugin`, but new content must use the correct levels from the start. See [Extending the Builder](Extending) for where that plugin sits in the render stage. +Do **not** jump from `#` straight to `###`. An h1 followed directly by an h3 is a heading-order defect on the built site, where headings must not skip a level. The `headingLevelNormalizePlugin` repairs older pages that still do it at build time, but new content must use the correct levels. See [Extending the Builder](Extending) for where that plugin sits in the render stage. -The repair is a re-levelling of the whole page, not a patch to one heading. The plugin raises **every** heading of level 3 or deeper until each one sits exactly one level below the heading it belongs under, closing every gap in a single pass: `#` / `###` renders as h1 / h2, and `#` / `###` / `#####` renders as h1 / h2 / h3. Only the h1 chapters are left as they are, since a page may legitimately have several. +The repair re-levels the whole page. The plugin raises **every** heading of level 3 or deeper until each sits exactly one level below its parent: `#` / `###` renders as h1 / h2, and `#` / `###` / `#####` renders as h1 / h2 / h3. Only the h1 chapters are left as they are. -**Only `#`, `##` and `###` headings get an entry of their own in the site search.** The search index cuts each page at its h1, h2 and h3 headings and gives each piece one entry, titled with its heading. A `####` or deeper heading gets no entry: its text is folded into the entry of the nearest heading above it. So does a heading with a generic name, such as **Example** or **See Also**, whatever its level: it names nothing a reader would search for, so its text joins the section before it. The list is `search.fold_headings` in `docs/_config.yml`. The index is built from the rendered page, after the normalizer has run, so it sees the levels the page renders with. +**Only `#`, `##` and `###` headings get an entry of their own in the site search.** The search index cuts each page at its h1, h2 and h3 headings and gives each piece one entry, titled with its heading. A `####` or deeper heading gets no entry: its text joins the nearest heading above it. So does a heading with a generic name, such as **Example** or **See Also**, at any level. The list is `search.fold_headings` in `docs/_config.yml`. The index is built from the rendered page, so it sees the levels after the repair. ### Editing a page that still uses the old style -The repair is conditional, and the condition is easy to break without noticing. **The plugin runs only on a page that uses `#` and `###` and no `##` anywhere** --- a single `##` and it does not run at all. More than four hundred pages on this site are currently in that state, so on most of them, adding one `##` section disarms the normalizer for the whole page: every `###` that was already there stops being repaired and becomes a live heading-order defect, in the same edit that added a correctly-levelled section. +**The plugin runs only on a page that uses `#` and `###` and no `##` anywhere.** More than four hundred pages are in that state. Adding one `##` section to such a page disarms the normalizer: every `###` already there stops being repaired and becomes a heading-order defect, in the same edit that added a correctly-levelled section. -Nothing tells you. The build still succeeds --- heading order is not one of the things it checks --- and `check.bat`'s accessibility scan audits fourteen sample pages out of roughly 1,160, so unless you happened to edit one of those fourteen, the defect ships. +Nothing warns you. The build succeeds, because it does not check heading order, and `check.bat`'s accessibility scan audits fourteen sample pages out of roughly 1,160. -So when you add a section to a page whose headings start at `###`, pick one of these and finish it: +When you add a section to a page whose headings start at `###`, pick one of these and finish it: -- **Leave the page in the old style** and write your new section as `###` too. The normalizer goes on repairing the whole page, your addition included. This is the smaller and safer edit, and the right one when you are adding a section to a page you are otherwise not touching. -- **Convert the page, in the same commit.** Renumber every heading so it sits one level below its parent --- which is what the normalizer was computing for you --- then add your section at the level it belongs. Done correctly the rendered HTML does not change at all, which is what makes the conversion safe to make on its own and easy to review: the diff is only `#` characters. +- **Leave the page in the old style** and write your new section as `###` too. The normalizer keeps repairing the whole page. This is the smaller edit, and the right one when you are not otherwise touching the page. +- **Convert the page, in the same commit.** Renumber every heading so it sits one level below its parent, then add your section at the level it belongs. The rendered HTML does not change, so the diff is only `#` characters and easy to review. -What you must not do is mix the two and leave it there. +Do not mix the two. ## Renaming a heading without breaking its links -A heading's text is its anchor. Nothing in the source declares the `id` --- `headerIdPlugin` derives it from the words, by lowercasing them, dropping every character that is not a letter, digit, hyphen or space, and turning each remaining space into a hyphen. `## Heading levels` becomes `id="heading-levels"`, and `#heading-levels` is what every link into that section uses. Reword the heading and the id changes with it. The old link does not fail visibly: a fragment that matches no element leaves the reader at the top of the page --- the right page, no error --- with nothing to say the section they asked for was ever there. +A heading's text is its anchor. `headerIdPlugin` derives the `id` from the words: it lowercases them, drops every character that is not a letter, digit, hyphen or space, and turns each remaining space into a hyphen. `## Heading levels` becomes `id="heading-levels"`, and every link into that section uses `#heading-levels`. Reword the heading and the id changes with it. The old link fails silently: a fragment that matches no element leaves the reader at the top of the right page, with no error. -Two details of the slug surprise people. Runs of hyphens are not collapsed and the ends are not trimmed, so `## AppObject (optional Bool)` --- two spaces, a parenthesised type --- produces `appobject--optional-bool`, not `appobject-optional-bool`. And repeated heading text is disambiguated by position: the first keeps the bare slug, the second gets `-1` appended, the third `-2`. Adding a second `## Example` *above* an existing one renames the existing one's anchor without touching a character of its text. +Two details of the slug surprise people. Runs of hyphens are not collapsed and the ends are not trimmed, so `## AppObject (optional Bool)` --- two spaces, a parenthesised type --- produces `appobject--optional-bool`, not `appobject-optional-bool`. And repeated heading text is disambiguated by position: the first keeps the bare slug, the second gets `-1` appended, the third `-2`. Adding a second `## Example` *above* an existing one renames the existing one's anchor. **Pin the id when the wording might change, or when anything outside the page links to it.** Write the attribute on the line directly below the heading, as `Reference/Attributes.md` does for all 55 of its attribute headings: ## AppObject (optional Bool) {: #appobject } -`headerIdPlugin` skips a heading that already carries an `id`, so the pinned value wins and the heading can then be reworded freely. Two things about the syntax are load-bearing. The delimiters are `{:` and `}`, not the bare braces the upstream plugin defaults to. And the blank line decides the direction: an attribute list on the line *immediately* after a block attaches backward to that block, while one separated by a blank line attaches forward to the next one. Leave a blank line in and the id silently lands on the following paragraph. +`headerIdPlugin` skips a heading that already carries an `id`, so the heading can then be reworded freely. Two details of the syntax matter. The delimiters are `{:` and `}`, not bare braces. And a blank line changes the direction: an attribute list on the line *immediately* after a block attaches to that block, while one after a blank line attaches to the next block. Leave a blank line in and the id lands on the following paragraph. -The same attribute works inline, attached to a span instead of a block, and that is where most of its use is --- `| **vbPRCMColor**{: #vbPRCMColor } | 2 | ...` gives one enumeration member in a table its own anchor. Across `docs/` 1,342 attribute lists set an id, in 179 files. Only 159 stand alone on a line of their own, 146 of them pinning a heading; the other 1,183 are inline. +The same attribute works inline, attached to a span --- `| **vbPRCMColor**{: #vbPRCMColor } | 2 | ...` gives one enumeration member in a table its own anchor. Across `docs/`, 1,342 attribute lists set an id, in 179 files. Only 159 stand alone on a line of their own, 146 of them pinning a heading; the other 1,183 are inline. -**`redirect_from` cannot rescue a renamed anchor,** and it is the first thing most people reach for. It emits a whole-page stub --- one destination URL written four times, as the canonical link, a `location=` assignment, a meta refresh and a visible fallback link --- and nothing in it maps one fragment to another. It moves a page. It cannot move a section. +**`redirect_from` cannot rescue a renamed anchor.** It emits a whole-page stub (one destination URL written four times, as the canonical link, a `location=` assignment, a meta refresh and a visible fallback link) and nothing maps one fragment to another. It moves a page, not a section. -To find what a rename would break, start with `grep -rn "#old-anchor" docs`, then let the build settle it. `build.bat` resolves every fragment in the tree against the ids that actually reached the HTML and reports `fragment #old-anchor not found` for each link that misses. It is the oracle rather than grep because almost every id is generated: it exists in the built page and in no source file, so there is nothing for grep to match on the receiving end. +To find what a rename would break, run `grep -rn "#old-anchor" docs`, then let the build settle it. `build.bat` resolves every fragment in the tree against the ids that reached the HTML and reports `fragment #old-anchor not found` for each link that misses. Grep alone is not enough because almost every id is generated and exists in no source file. -That check sees only links made from inside this repository, and the `/tB/` anchors have a consumer outside it: the IDE help add-in, through the [symbol index](Building#the-symbol-index). Every member heading on a reference page, and every attribute heading on `/tB/Core/Attributes`, is an entry in that index, and **a build that loses one of its URLs fails**, naming it --- so a reworded member heading is caught whether or not anything in the site links to it. The fix is the one above: keep the new wording and pin the old id. +That check sees only links made from inside this repository. The `/tB/` anchors have a consumer outside it: the IDE help add-in, through the [symbol index](Building#the-symbol-index). Every member heading on a reference page, and every attribute heading on `/tB/Core/Attributes`, is an entry in that index, and **a build that loses one of its URLs fails**, naming it. A reworded member heading is caught even if nothing in the site links to it. The fix is the one above: keep the new wording and pin the old id. -Every heading on the attributes page also carries a pinned `{: #... }` id, and that is not decoration. Three of them once had none and resolved on the default slug alone --- which ties the published URL to the heading text, so appending a parenthesised type to one, to match its neighbours, would have broken it silently. Pin the id when you add the heading. +Every heading on the attributes page carries a pinned `{: #... }` id. Three once had none and resolved on the default slug alone, so appending a parenthesised type to one, to match its neighbours, would have broken it silently. Pin the id when you add the heading. ## Naming what a page documents {: #symbols } -The [symbol index](Building#the-symbol-index) learns what a reference page documents from the page itself: its title, less a trailing word such as `Module` or `class`, and the comma-separated names of its first heading --- `# Left, LeftB` documents both. For a Core page, the words of the title are the statement and its keywords: `Do...Loop` is the **Do** statement, and **Loop** is a keyword that leads to it. +The [symbol index](Building#the-symbol-index) reads what a reference page documents from the page itself: its title, less a trailing word such as `Module` or `class`, and the comma-separated names of its first heading --- `# Left, LeftB` documents both. On a Core page the words of the title are the statement and its keywords: `Do...Loop` is the **Do** statement, and **Loop** is a keyword that leads to it. -A page whose title cannot name its subject says so with `symbols:`, which replaces those names: +A page whose title cannot name its subject lists the names in `symbols:`, which replaces those names: ```yaml --- @@ -319,11 +284,11 @@ symbols: [_HiddenModule] --- ``` -On a statement page the first name is the statement and the rest are its keywords; on an operator page each is an operator of its own --- the comparison operators page lists `["=", "<>", "<", "<=", ">", ">="]`, quoted, because a bare `>` is YAML syntax. Only two pages need it today. The build tells you when a third does: a page in a package folder that gives the index no entry at all is named after the build's summary. +On a statement page the first name is the statement and the rest are its keywords; on an operator page each is an operator of its own --- the comparison operators page lists `["=", "<>", "<", "<=", ">", ">="]`, quoted, because a bare `>` is YAML syntax. Only two pages need it today. A page in a package folder that gives the index no entry at all is named after the build's summary. ## Index entries for the site search -The search finds a page by the words in it. That fails for a term the right page never uses: the `#If` / `#Const` page is where a reader searching for *conditional compilation* should land, and its text never says those words. An index entry names such a term, the way a book's index does, and a search for the whole term then finds that page first. +The search finds a page by the words in it. That fails for a term the right page never uses: a reader searching for *conditional compilation* should land on the `#If` / `#Const` page, whose text never says those words. An index entry names such a term, as a book's index does, and a search for the whole term then finds that page first. A page takes entries in its frontmatter, as one term or a list: @@ -341,24 +306,21 @@ A section takes them on its heading, with terms separated by `;`, on the line di ## Object {: index="late binding" } -**`index`** is the main entry: the one place a reader searching for the term wants first. **`index_also`** is a strong second answer, which ranks right behind it --- `/Reference/Compiler-Constants` takes `index_also: conditional compilation`. A term has one main entry across the whole site, and the build stops, naming every page that claims it, if two do. Any number of places may take it as `index_also`. +**`index`** is the main entry: the page a reader searching for the term wants first. **`index_also`** is a strong second answer that ranks right behind it --- `/Reference/Compiler-Constants` takes `index_also: conditional compilation`. A term has one main entry across the site; the build stops, naming every page that claims it, if two do. Any number of pages may take it as `index_also`. -A term matches only a search that names all of it, alone or among other words: *late binding* and *late binding in VBA* find the entry, *binding* on its own does not. Capitals, hyphens and word endings do not matter. The entries never appear on the page; the build takes them off the heading before it writes the HTML, and fails on `index` written on anything but a heading. +A term matches a search that names all of it, alone or among other words: *late binding* and *late binding in VBA* find the entry, *binding* on its own does not. Capitals, hyphens and word endings do not matter. The entries never appear on the page. The build fails on `index` written on anything but a heading. -Mark a term only where the page's own words cannot find it, and name the page a reader wants for that term, not a summary of the page. The entries in use are few on purpose: before adding one, search for the term on a built site and see where it lands. +Mark a term only where the page's own words cannot find it, and name the page a reader wants for that term. Few entries are in use; before adding one, search for the term on a built site and see where it lands. ## Counts the build fills in {: #counts } -A figure written by hand is correct the day it is written and attached to -nothing that would notice when it stops being correct. A wrong count breaks no -link, fails no gate, and reads exactly like a right one. So the numbers the -build already knows are written as names: +A count written by hand goes stale without any gate noticing. Write the numbers +the build already knows as names: {{tbdocs:pages}} pages under `Reference/` are `index.md` files of this shape -renders as the current figure. The available names, each derived on every build -from the same data the pages themselves come from: +renders as the current figure. The names are derived on every build: | name | is | |---|---| @@ -375,61 +337,50 @@ from the same data the pages themselves come from: | `enumerations` | entries in the alphabetical index of `Reference/Enumerations.md` | | `redirectStubs` | whole-page stubs emitted for `redirect_from:` entries | -The three package names are separate because the words for them collide. The +The package names are separate because the words for them collide. The [welcome page](../../) heads a section *Built-in packages* and lists all {{tbdocs:packages}} under it, while [Packages](../../tB/Packages/) reserves that -word for the {{tbdocs:builtInPackages}} a project references on demand. Neither -is wrong, and no reader can tell the two apart from one page alone, so a -sentence meaning one of them should not be written with a digit. Pick the name -for the set your sentence means and it cannot drift into the other set's number. +word for the {{tbdocs:builtInPackages}} a project references on demand. Both are +right, so a sentence meaning one of them should use the name for that set, not a +digit. -**A name is a derivation, never a constant.** A registry holding `pages: 908` -would not have removed the stale figure, only moved it from a page a -contributor reads into a module nobody opens. If a number cannot be derived it -does not get a name --- an assertion that needs a person to check it belongs in -prose, where a reader can see that it is a claim. +**A name is always derived from the build.** A number that cannot be derived +does not get a name; an assertion that needs a person to check it belongs in +prose. -Two consequences worth knowing before using one: +Two consequences: - **A misspelled name fails the build**, before any page renders, naming the - file, the line and the nearest match. It cannot be allowed to pass: an - unrecognised placeholder would otherwise be published to readers verbatim, - which is the failure the whole mechanism exists to prevent. -- **Code is immune, and needs no escaping.** A placeholder inside backticks or - a fence is left alone, because the substitution runs over markdown's inline - text and code is a different kind of token entirely. A page that needs to - *show* the syntax puts it in backticks, which is what such a page does - anyway. The one placement that does not work is inside a raw HTML block; the - build fails on that too rather than publishing it. + file, the line and the nearest match. +- **Code needs no escaping.** A placeholder inside backticks or a fence is left + alone. To *show* the syntax, put it in backticks. A placeholder inside a raw + HTML block fails the build. Numbers already written as words stay as words, because a substitution always yields digits. [Callouts](#callouts) below opens *Three severities, used -distinctly*, and should keep the word: it counts the three bullets directly -beneath it, which is a fact about that sentence rather than about the site and -could not be derived from build state at all. +distinctly* and keeps the word: it counts the three bullets beneath it, a fact +about that sentence, not about the site. ## Formatting conventions - **Bold** (`**...**`) for keywords and literal tokens the reader would type verbatim; *italic* (`*...*`) for placeholders and argument names. -- twinBASIC code goes in a ` ```tb ` fenced block --- Shiki highlights it with the vendored twinBASIC grammar, and `twinbasic`, `vb` and `vba` select the same grammar. The other highlighted fence languages are `js`, `yaml`, `json`, `c`, `html`, `xml`, `sql` and `batch`. Anything else renders as unhighlighted plain text: the build does not fail, but it prints `highlight: unknown fence language ""` naming the language and the list to add it to, so check the build output rather than the page. +- twinBASIC code goes in a ` ```tb ` fenced block. Shiki highlights it with the vendored twinBASIC grammar; `twinbasic`, `vb` and `vba` select the same grammar. The other highlighted fence languages are `js`, `yaml`, `json`, `c`, `html`, `xml`, `sql` and `batch`. Anything else renders as plain text, and the build prints `highlight: unknown fence language ""` without failing, so check the build output rather than the page. - Parameter lists use the definition-list pattern (a term line, then a `: definition` line beneath it), not a markdown table. -- **An inline code span cannot start or end with a single space.** CommonMark removes one space from each end of a span whose content is not all spaces, so `` ` 1 2 ` `` renders as `1 2`. That matters whenever the padding *is* the value being shown --- what `Debug.Print` emits into its 14-column print zones, a fixed-width return such as [Partition](../tB/Modules/Interaction/Partition)'s, anything a reader might count characters in. Write two spaces at each end to get one, and confirm against the built HTML rather than the preview: five claims on two pages were quietly de-padded this way, each of them a documented output value with its leading and trailing spaces missing. -- For dashes, write `--` in the source (it renders as an en-dash) or `---` (an em-dash). Never paste a literal `–` or `—`. Nothing in the build rejects one: the typographer converts the ASCII forms and passes a literal character straight through, so a stray dash ships silently and only the source becomes inconsistent. `scripts/convert_em_dash_separators.mjs` is the normaliser, and it is run by hand. +- **An inline code span cannot start or end with a single space.** CommonMark removes one space from each end of a span whose content is not all spaces, so `` ` 1 2 ` `` renders as `1 2`. That matters when the padding *is* the value shown --- what `Debug.Print` emits into its 14-column print zones, a fixed-width return such as [Partition](../tB/Modules/Interaction/Partition)'s. Write two spaces at each end to get one, and confirm against the built HTML rather than the preview. +- For dashes, write `--` in the source (it renders as an en-dash) or `---` (an em-dash). Never paste a literal `–` or `—`. The build does not reject one, so a stray dash ships silently. `scripts/convert_em_dash_separators.mjs` is the normaliser, and it is run by hand. ### A code sample that holds a fence marker {: #fence-in-fence } -A sample that builds a Markdown string has ` ``` ` inside it, and the fence -around the sample has to survive that. Which form to use depends on where the -marker sits. +A sample that builds a Markdown string has ` ``` ` inside it. The form of the +fence around the sample depends on where the marker sits. **A marker in the middle of a line needs nothing special.** A fence closes only on a line that is nothing but backticks, so an ordinary three-backtick fence holds a sample whose markers are inside string literals. The [`Description`](../../tB/Core/Attributes#description) entry on the attribute -reference is the shipped precedent: its example assembles a Markdown description -from twinBASIC literals, two of which are fence markers, inside a plain ` ```tb ` -fence. +reference is an example: it assembles a Markdown description from twinBASIC +literals, two of which are fence markers, inside a plain ` ```tb ` fence. ```tb [Description("### Example" & vbCrLf & _ @@ -448,57 +399,43 @@ with four backticks and close with four: ```` Nothing else changes. A fence closes only on a run at least as long as the one -that opened it, and both pre-render passes that have to find fences get them from -the site's own parser, through `lib/markdown.mjs` --- `maskCode`, which hides code -from the rewrites, and `rewriteAdmonitions`, which is the one that matters here -because it runs *outside* the mask by design. The language tag reaching the -highlighter is the same string either way --- a four-backtick `tb` fence is highlighted exactly as a -three-backtick one is. No page in `docs/` uses one yet, so there is no example to -copy; it is still the right form. - -**Prefer backticks to a tilde fence.** Both work: `rewriteAdmonitions` used to -recognise backtick fences only, so a tilde fence holding a ` ``` ` line was not -hidden from it and every `> [!NOTE]` after it rendered as literal text in a plain -blockquote. That is fixed, and `check_code_regions.mjs` carries a probe for it. -Backticks remain the house form because every fence in `docs/` is one, and a -longer backtick run is the one construct here with a shipped precedent. - -That failure is not hypothetical. The attribute reference once shipped all six of -its admonitions as the literal text `[!NOTE]`, because the same pairing closed -that page's opening fence on a marker in the middle of a line and every pairing -after it was off by one. The mid-line case is fixed --- the rewrite now asks the -site's parser where the fences are --- and [`check_code_regions.mjs`](Tools-Toolchain-Tests#check-code-regions), which -`test.bat` runs, is the only gate that sees this class of fault at all. +that opened it, and the language tag reaching the highlighter is the same string +either way. No page in `docs/` uses one yet. + +**Prefer backticks to a tilde fence.** Both work, but every fence in `docs/` is +a backtick one. A fault in how the admonition rewrite finds fences shows every +`> [!NOTE]` after it as literal text; the attribute reference once shipped all +six of its admonitions that way. +[`check_code_regions.mjs`](Tools-Toolchain-Tests#check-code-regions), which +`test.bat` runs, is the only gate that sees this class of fault. ### Typography -The site ships its own fonts, so a page renders the same on every platform: **Inter** for text, **Cascadia Mono** for code, and **Source Serif 4** for body text in the PDF book. Nothing is needed from you to use them --- they apply automatically --- but two things follow from it. +The site ships its own fonts, so a page renders the same on every platform: **Inter** for text, **Cascadia Mono** for code, and **Source Serif 4** for body text in the PDF book. They apply automatically. Two things follow. -**Characters outside the shipped subsets fall back to a system font**, which is visible as a glyph in the wrong face. The subsets cover Latin (including Latin Extended-A and -B), Greek, punctuation, arrows, mathematical operators, currency, fractions, geometric shapes, dingbats, and --- for code --- box drawing. Emoji are deliberately excluded and come from the platform's own emoji font. Two specifics worth knowing: write the check mark as **U+2713** (✓), not **U+2714** (✔), which no text face in the stack carries; and prefer `:` over **U+22EE** (⋮) inside a monospaced ASCII diagram, where a fallback glyph of a different width shears the box borders. +**Characters outside the shipped subsets fall back to a system font**, visible as a glyph in the wrong face. The subsets cover Latin (including Latin Extended-A and -B), Greek, punctuation, arrows, mathematical operators, currency, fractions, geometric shapes, dingbats, and --- for code --- box drawing. Emoji are excluded and come from the platform's own emoji font. Write the check mark as **U+2713** (✓), not **U+2714** (✔), which no text face in the stack carries. Prefer `:` over **U+22EE** (⋮) inside a monospaced ASCII diagram, where a fallback glyph of a different width shears the box borders. -Those two example glyphs are, deliberately, the only characters on the whole site that fall back --- so the ✔ and ⋮ above are being drawn by your system's symbol font right now, next to text that is not. That difference in weight and shape is the effect this section is about. +Those two example glyphs are the only characters on the site that fall back, so the ✔ and ⋮ above are drawn by your system's symbol font, next to text that is not. That difference in weight and shape is the effect to avoid. -Diagram exports carry the font with them. The Download / Copy SVG and PNG buttons of each diagram embed the typeface into the exported file, because an exported SVG has no access to the site's stylesheet and would otherwise render in whatever the viewer has installed. All four buttons work on every diagram. +Diagram exports carry the font with them. The Download / Copy SVG and PNG buttons of each diagram embed the typeface into the exported file, because an exported SVG cannot use the site's stylesheet. All four buttons work on every diagram. -**Do not hand-edit a diagram's `.svg`.** It is a build artifact: the `.dot` beside it is the source, and the next build overwrites your edit. Changing the face is the edit that looks most harmless and is not --- Graphviz sizes each box to the text it measured, so a diagram whose labels are painted in a font the layout never saw has text hanging outside its boxes. `check.bat` fails on that; see [Diagrams](#diagrams) below. +**Do not hand-edit a diagram's `.svg`.** It is a build artifact: the `.dot` beside it is the source, and the next build overwrites your edit. Changing the font is especially harmful --- Graphviz sizes each box to the text it measured, so labels in a font the layout never saw hang outside their boxes. `check.bat` fails on that; see [Diagrams](#diagrams) below. ## Adding a CSS rule that works in both themes {: #css-rules } -A style rule for something new on the site goes in `docs/_sass/custom/custom.scss`. Do not put it in the vendored theme under `builder/vendor/just-the-docs/`, and do not start a stylesheet of your own, which no page would load: everything under `docs/_sass/` is compiled into `just-the-docs-combined.css`, which every page does load. `serve.bat` rebuilds it each time you save. +A style rule for something new goes in `docs/_sass/custom/custom.scss`. Do not put it in the vendored theme under `builder/vendor/just-the-docs/`, and do not start a stylesheet of your own, which no page would load. Everything under `docs/_sass/` is compiled into `just-the-docs-combined.css`, which every page loads. `serve.bat` rebuilds it each time you save. -**The dark theme is a second copy of the whole theme, not a set of CSS variables.** The build compiles the theme twice and emits the dark copy inside a theme selector such as `html[data-theme="dark"]`, so every theme rule is more specific in dark mode than it is in light. A rule you write with a single class can therefore beat the theme in light mode and lose to it in dark, with no error: `.reversefootnote` did exactly that. Prefix the selector with `.main-content` --- `.main-content .reversefootnote` --- and it wins in both. [The specificity trap](Builder#the-specificity-trap) covers the cases where that is not enough. +**The dark theme is a second copy of the whole theme, not a set of CSS variables.** The build compiles the theme twice and emits the dark copy inside a theme selector such as `html[data-theme="dark"]`, so every theme rule is more specific in dark mode. A rule with a single class can therefore beat the theme in light mode and lose to it in dark, with no error: `.reversefootnote` did exactly that. Prefix the selector with `.main-content` --- `.main-content .reversefootnote` --- and it wins in both. [The specificity trap](Builder#the-specificity-trap) covers the cases where that is not enough. -**Check it in both themes.** Run `serve.bat`, open a page that uses the rule, and switch themes with the theme button in the page header rather than with your operating system's setting, because the button is what exercises the `[data-theme]` rules. A rule that fails only in the dark theme is usually cosmetic, and no gate reports it. +**Check it in both themes.** Run `serve.bat`, open a page that uses the rule, and switch themes with the theme button in the page header, not with your operating system's setting: the button is what exercises the `[data-theme]` rules. No gate reports a rule that fails only in dark. -[Project styling](Builder#project-styling) is the full account: which file under `docs/_sass/` holds what, why the theme is compiled twice, and how to verify a style change. +[Project styling](Builder#project-styling) says which file under `docs/_sass/` holds what and how to verify a style change. ## Checking that a sample compiles -Nothing in the ordinary build looks inside a code fence. The link check, the accessibility -scan and the code-region gate all pass over a twinBASIC sample that the compiler would -refuse --- two such samples shipped, one of them a flagship example on a package page, and -every gate was green over both. +The ordinary build does not look inside a code fence. The link check, the accessibility +scan and the code-region gate all pass a twinBASIC sample that the compiler would refuse. A sample can ask to be compiled. Add `check_build` to its fence: @@ -515,9 +452,8 @@ Then run it, which needs a twinBASIC install and Windows: examples.bat --only "^Reference/Core" **Paste the summary line into the pull request description.** `examples.bat` runs in no -CI workflow, so a green pull request says nothing about a sample; the line a run ends with -is the evidence a reviewer has. Run it over the pages you changed, and paste the command -with the line: +CI workflow, so the line a run ends with is the reviewer's only evidence. Run it over the +pages you changed, and paste the command with the line: examples.bat --only "^Tutorials/Testing-with-Assert" check_examples: 7 sample(s), 7 compile, 0 finding(s), 19.6s -- clean @@ -526,48 +462,44 @@ with the line: sample that failed against its page and line, and exits 1. **The marker never reaches the page.** The renderer takes the first word of a fence's info -string as the language and discards the rest, so a marked fence produces byte-identical -HTML to an unmarked one. Nothing in the built site, the search index or the PDF can tell -the difference. +string as the language and discards the rest, so the built site, the search index and the +PDF are identical with or without it. **Mark every `tb` fence: `check_build` for a sample that is a program, `inert=` for one that is not.** A statement run with an elision in it, a signature with no body, a -syntax skeleton with `` --- those are good documentation, and there is nothing -for a compiler to say about them. They are marked all the same, as [Saying that a sample is -not a program](#saying-that-a-sample-is-not-a-program) describes, because an unmarked fence -is never compiled and `examples.bat --census` counts it as still unmarked. Checking is -opt-in for a historical reason: when the harness arrived, roughly a third of the fences -were not programs, and a gate that demanded every fence compile would have needed hundreds -of exceptions on the first day. +syntax skeleton with `` are good documentation, but a compiler has nothing to +say about them. Mark them anyway, as [Saying that a sample is +not a program](#saying-that-a-sample-is-not-a-program) describes: an unmarked fence +is never compiled, and `examples.bat --census` counts it as still unmarked. The tool works out what to build around a sample --- a whole `Class` goes in a file of its own, procedures and declarations go in a generated module, loose statements go in a -generated `Sub`. Three keys override it when it guesses wrong, and one flag asks for more: +generated `Sub`. These keys override it when it guesses wrong: | In the fence | Means | |---|---| | `check_build` | Compile this sample. | | `check_run` | Compile it, build it and run it, then compare what it prints with what the page says it prints, as described below. Only a statement sample (`slot=sub`) can be run. A sample that raises an error, does not return, or prints something else is a finding. A sample that calls `MsgBox` or `InputBox`, or contains an `End` statement, is refused without being built. | | `hidden` | Context for the page's other samples that the reader never sees. Implies `check_build`. See below. | -| `slot=file` / `slot=module` / `slot=sub` / `slot=class` / `slot=method` | What to generate around it, when the inference is wrong. `class` and `method` are the same two shapes inside a `Class` rather than a `Module`, for code-behind. The report always names the slot it used, so a wrong guess reads as a wrong guess. | +| `slot=file` / `slot=module` / `slot=sub` / `slot=class` / `slot=method` | What to generate around it, when the inference is wrong. `class` and `method` are the same two shapes inside a `Class` rather than a `Module`, for code-behind. The report names the slot it used. | | `inherits=` | The sample is code-behind *of* something --- `inherits=Form`, `inherits=MDIForm`. The wrapper becomes a `Class` that inherits it, so `Me.Caption` resolves against the real type. | | `project=` | Which template project to build into. The default follows the page: a page under `Reference/Built-In/` gets the one that references every package. `project=form` is for a `check_run` sample that needs a real form, to draw on or to read a property of: the project has an empty `Form1`, and the checker unloads every form a sample leaves loaded. Every fence of a `projname` group names the same project. | -| `projname=` | Build these samples **as one project**, for a page that presents one program in pieces --- a function in one fence and the tests for it in the next three. Every sample sharing the name is compiled together and nothing else is compiled with them. | +| `projname=` | Build these samples **as one project**, for a page that presents one program in pieces --- a function in one fence and the tests for it in the next three. Samples sharing the name are compiled together and with nothing else. | | `concat_group=` | Join these fences, in page order, into **one** piece of code before building it --- for a single construct shown in parts, such as a `Sub` or a `Class` introduced a section at a time. Implies `check_build`. See below. | -| `expect-error=` | This sample is *meant* not to compile --- it is showing what goes wrong --- and the run fails if it compiles. | +| `expect-error=` | This sample is *meant* not to compile --- it shows what goes wrong --- and the run fails if it compiles. | | `resource=` | **On a fence in any language**, not just ` ```tb `. The fence's contents are written into the generated project at that project-relative path, so the page's samples can be compiled against it. For the compile-time attributes that read a project file --- see below. | -| `inert=` | This fence is **not a program**, and saying so settles it: it is never compiled, never proposed, and counted under its reason instead of sitting in the backlog. See below. | - -A `check_run` sample states what it prints in one of two ways, and the checker reads -the sample's own text, so the page needs nothing extra. A trailing comment on a -`Debug.Print` line gives that line's output. A comment line that is exactly `' Output:` -gives the whole output, one line per comment line below it. A sample that states -nothing is still run, and must return without raising an error. An error the sample -handles itself is not raised, so a sample can show an error with `On Error Resume Next` -and print `Err.Number`. When a sample states -anything, it must print exactly as many lines as it states. Each line is compared with -the spaces at both ends removed, because twinBASIC prints a space before a positive -number. +| `inert=` | This fence is **not a program**: it is never compiled or proposed, and is counted under its reason instead of sitting in the backlog. See below. | + +A `check_run` sample states what it prints in one of two ways, read from the +sample's own text. A trailing comment on a `Debug.Print` line gives that line's +output. A comment line that is exactly `' Output:` gives the whole output, one +line per comment line below it. When a sample states anything, it must print +exactly as many lines as it states. Each line is compared with the spaces at both +ends removed, because twinBASIC prints a space before a positive number. + +A sample that states nothing is still run, and must return without raising an +error. An error the sample handles itself is not raised, so a sample can show an +error with `On Error Resume Next` and print `Err.Number`. ````markdown ```tb check_run @@ -591,41 +523,39 @@ Next ### Saying that a sample is not a program -Plenty of good fences are not programs. A syntax skeleton written with placeholder names -(`Interface name Extends base_interface`) teaches the shape better than any compilable -stand-in would; a fence that continues the previous one, or shows the invalid form beside -the valid one, is doing its job exactly as written. +Many good fences are not programs. A syntax skeleton written with placeholder names +(`Interface name Extends base_interface`) teaches the shape better than a compilable +stand-in; so does a fence that continues the previous one or shows the invalid form beside +the valid one. Mark those `inert=` rather than leaving them unmarked, so nobody triages them twice: | Reason | For | |---|---| | `skeleton` | placeholder identifiers --- `name`, `base_interface`, `` | -| `signature` | a procedure's signature shown on purpose without a body, where a body would mislead --- the members an interface declares, which a reader implements under other names | -| `excerpt` | deliberately continues another fence, or shows part of one | +| `signature` | a procedure's signature shown without a body, where a body would mislead --- the members an interface declares, which a reader implements under other names | +| `excerpt` | continues another fence, or shows part of one | | `pseudo` | prose, a table or a protocol listing set in a code fence | -| `contrast` | shows the invalid form on purpose, beside the valid one | +| `contrast` | shows the invalid form, beside the valid one | | `external` | needs a file or environment the checker cannot provide | | `designer` | needs a real form designer --- a `Handles` clause on designer-declared fields, or a control array | -| `blocked` | correct code that a **product defect** stops compiling *and for which no workaround exists*. Look hard for one first --- a package's private half can be reached with an asterisk [library symbol](../Features/Packages/Library-Symbols), and a page that shows the qualified form plus a note is better than a page that shows code the reader cannot run | +| `blocked` | correct code that a **product defect** stops compiling *and for which no workaround exists*. Look hard for one first --- a package's private half can be reached with an asterisk [library symbol](../Features/Packages/Library-Symbols), and a page that shows the qualified form plus a note beats one that shows code the reader cannot run | -An unrecognised reason is refused, the same as a bad `slot=`, and `inert` together with -`check_build` is refused as a contradiction. The census then reports three numbers rather -than two: how many samples are checked, how many are inert, and how many are **undecided**. -Only the last is a backlog. +An unrecognised reason is refused, as is `inert` together with `check_build`. The +census reports how many samples are checked, how many are inert, and how many are +**undecided**. Only the last is a backlog. **A fence that stops before its closing line is usually one line from compiling.** Before marking it `excerpt` or `signature`, put an elision comment (`' ...`) where the omitted code would go and add the construct's closing line --- `End Function`, `End Interface`. The -reader still sees that something was left out, and the compiler can now check what was -left in. +reader still sees that something was left out, and the compiler can check what was left in. ### A sample that reads a file Some twinBASIC features read a project file *while compiling*. -[`[PopulateFrom]`](../tB/Core/Attributes#populatefrom) is the one to know: it fills an empty -**Enum** with members taken from a JSON resource, so a page documenting it has an enum whose -members exist only if that file does. +[`[PopulateFrom]`](../tB/Core/Attributes#populatefrom) fills an empty **Enum** with +members taken from a JSON resource, so a page documenting it has an enum whose members +exist only if that file does. Mark the JSON block that the page already shows with `resource=`, and the file is staged beside the samples: @@ -636,23 +566,18 @@ beside the samples: ``` ```` -The file travels with its page, exactly as a `hidden` fence does, so every sample on that -page is compiled against it. The block is rendered normally --- the reader is meant to see -the file --- and the path may not climb out of the project, so no `..` and no drive letter. - -What this buys is more than a green tick: the enum members the page's *other* samples use -come from that JSON, so the compiler checks the file's shape and the names it produces -against the code that reads them. +Every sample on that page is compiled against the file. The block is rendered normally, +and the path may not climb out of the project: no `..` and no drive letter. The enum +members the page's *other* samples use come from that JSON, so the compiler checks the +file's shape and the names it produces against the code that reads them. A sample that assumes a control on a form is fine: the templates declare `Text1`, -`ListView1`, `CefBrowser1` and the others, exactly as a reader's own project would, so what -gets checked is the part the sample is actually claiming. +`ListView1`, `CefBrowser1` and the others, as a reader's own project would. -**`Me` and `WithEvents` are read as declarations of intent.** Neither is legal in a standard -module --- the compiler says so in as many words --- so a fence using either is class -code-behind, and the tool wraps it in a `Class` without being told. What it cannot guess is +**`Me` and `WithEvents` mark class code-behind.** Neither is legal in a standard module, +so the tool wraps a fence using either in a `Class` without being told. It cannot guess *which* class, so a sample reaching a member of the thing it is code-behind of --- -`Me.Caption`, `Me.Arrange` --- needs `inherits=` to say. +`Me.Caption`, `Me.Arrange` --- needs `inherits=`. ### A page can carry its own context @@ -661,8 +586,8 @@ nothing**: it is absent from the page, the search index, the offline mirror and It is for the declarations a sample assumes but no reader needs to read --- a class the page describes in prose but never lists, an API `Declare`, a control instance only this -page uses. Putting them in a hidden fence keeps them beside the samples that need them, -in one file, instead of in a template shared by every other page on the site. +page uses. They stay in one file beside the samples that need them, not in a template +shared by every other page. ````markdown ```tb hidden @@ -674,27 +599,26 @@ End Class ``` ```` -Use it for context, not for hiding a sample. A hidden fence is still compiled, so it is -checked like everything else --- but nobody can read it, and a fence nobody can read is -not documentation. +Use it for context, not for hiding a sample. A hidden fence is compiled like any other, +but nobody can read it, and a fence nobody can read is not documentation. -**Decide what to hide by asking what the reader needs.** Before hiding a line, ask whether -a reader would write it, or would need it to know where an object comes from. An event -handler's header is both --- `Request.Headers` means nothing until the reader sees which -event passes `Request` --- and so is the subject of a `With` that a leading-dot line depends -on, or the `WithEvents` field that an event handler's name is built from. Show those. Hide -what the reader brings from elsewhere: their own helper routines, the controls the form -designer declares, and stand-ins for declarations that an external library provides. +**Decide what to hide by asking what the reader needs.** Show a line the reader would +write, or needs to know where an object comes from: an event handler's header +(`Request.Headers` means nothing until the reader sees which event passes `Request`), the +subject of a `With` that a leading-dot line depends on, or the `WithEvents` field that an +event handler's name is built from. Hide what the reader brings from elsewhere: their own +helper routines, the controls the form designer declares, and stand-ins for declarations +that an external library provides. ### One construct across several fences `concat_group=` joins every fence on the page that carries the name, in page order, -into one piece of code before anything else happens. It is for a single construct shown in -parts --- a `Sub` built up a few lines at a time, a `Class` whose members are introduced one -section at a time --- which no fence can compile on its own, because each part is an -unclosed block. It is not `projname`: that compiles its samples as *separate* modules of one -project, while the parts of a `concat_group` become one file, so a `Private` field declared -in the first part is visible in the last. +into one piece of code. It is for a single construct shown in parts --- a `Sub` built up a +few lines at a time, a `Class` whose members are introduced one section at a time --- which +no fence can compile on its own, because each part is an unclosed block. It differs from +`projname`, which compiles its samples as *separate* modules of one project: the parts of a +`concat_group` become one file, so a `Private` field declared in the first part is visible +in the last. Parts can be `hidden`. The [Painting](../Tutorials/CustomControls/Painting) tutorial shows only the `OnPaint` method it is teaching. A hidden fence before it opens the class, @@ -705,84 +629,69 @@ the sample *inside* anything. The joined code is reported as its first visible p error in any part is reported against that part's own line. **A sample that needs another sample needs `projname`.** Samples are packed several to a -generated project, so one can sometimes see another's declarations by luck --- and luck -changes with what else is being checked, which makes a page pass one run and fail the next. -Naming the group says the dependency out loud, and the tool then refuses a group that is -only half marked rather than reporting a missing symbol in the sample that is fine. +generated project, so one can see another's declarations by luck, and a page passes one +run and fails the next. Naming the group states the dependency, and the tool refuses a +group that is only half marked. -**A mistyped marker is caught.** `check_bild` renders identically to no marker at all, so a -sample carrying one would simply never be compiled; the tool reports an unrecognised token -rather than skipping it in silence. +**A mistyped marker is caught.** `check_bild` renders like no marker at all, so the tool +reports an unrecognised token rather than skipping the fence silently. [Code Samples and Attributes](Tools-Samples#check-examples) covers running it --- the census and survey modes, the flags, and what the report means. ## Bullet lists, dashes, and parentheses -Most bullets on this site are a term, a dash, and a description, and which dash -depends on what precedes it. The practice is near-absolute and has never been -written down, so it survives only by imitation: +Most bullets on this site are a term, a dash, and a description. The dash depends +on what precedes it: -- A **bold-term** bullet takes `---`, the em-dash: `- **Reference** --- the language and library reference.` 50 of the 50 such bullets in `docs/Reference/` do this, with no exceptions. -- A **link** bullet takes `--`, the en-dash: `- [Permanent Links](Permanent-Links) -- the stable URL scheme.` 1,570 of 1,570 in `docs/Reference/`, again with none the other way. +- A **bold-term** bullet takes `---`, the em-dash: `- **Reference** --- the language and library reference.` All 50 such bullets in `docs/Reference/` do this. +- A **link** bullet takes `--`, the en-dash: `- [Permanent Links](Permanent-Links) -- the stable URL scheme.` All 1,570 in `docs/Reference/` do this. -The reason is visual rather than grammatical. A bold term is a label with a -definition after it, so the separator has to be strong enough to read as a break; -a link already ends in its own visual boundary, and an em-dash after it crowds -the line. The distinction matters only because it is consistent: a page that -mixes the two looks like two people wrote it. +A bold term is a label with a definition after it, so the separator has to read as +a break. A link already ends in its own visual boundary, and an em-dash after it +crowds the line. Do not mix the two on a page. **The description after the dash continues the sentence and is not capitalised.** Of the bold-term bullets in `docs/Reference/` that begin with a letter, 26 of 26 are lowercase, and in `docs/Documentation/` 16 of 16. Link bullets hold to the same rule where they can: 1,411 begin lowercase, and almost all of the 94 that do not begin with a word that carries its own capital anyway --- `Win32`, `OLE`, -`GDI`, `Variant`, `JavaScript`. The only genuine exceptions are 36 bullets on +`GDI`, `Variant`, `JavaScript`. The only exceptions are 36 bullets on `Reference/Built-In/AppGlobalClassObject/index.md`, which writes each annotation -as a full sentence starting `Returns` or `Gets`. That is one page's local habit, -not a second convention to choose between. - -**Parentheses are for a short label, `---` for an aside that is a clause.** The -split is real practice, not a preference: across the prose of `docs/Reference/`, -78% of parenthesised spans are 25 characters or shorter and the median is 12. What -fills them is a tag the reader takes in without stopping --- `(default)` 152 -times, `(VBRUN)` 86, `(twinBASIC)` 59, `(optional Bool)` 38, and the kind markers -`(Class)`, `(Property)`, `(Sub)`. Once the aside has a verb in it, it is part of -the argument of the sentence and belongs between em-dashes, where the reader -still reads it: +as a full sentence starting `Returns` or `Gets`. + +**Parentheses are for a short label, `---` for an aside that is a clause.** +Across the prose of `docs/Reference/`, 78% of parenthesised spans are 25 +characters or shorter and the median is 12. They hold a tag the reader takes in +without stopping --- `(default)` 152 times, `(VBRUN)` 86, `(twinBASIC)` 59, +`(optional Bool)` 38, and the kind markers `(Class)`, `(Property)`, `(Sub)`. An +aside with a verb in it is part of the argument of the sentence and belongs +between em-dashes: ... or **Any** (twinBASIC; the type is inferred from *expression*) ... equivalent to a separate assignment immediately after the **Dim** --- `Dim i As Long = 1` is the same as `Dim i As Long: i = 1`. -`Reference/Core/Dim.md` currently does both within four lines of each other, -which is what an unstated rule looks like. Apply the length test: a tag of two or -three words goes in parentheses, anything that could stand as its own sentence -takes `---`. +`Reference/Core/Dim.md` currently does both within four lines of each other. Apply +the length test: a tag of two or three words goes in parentheses, anything that +could stand as its own sentence takes `---`. ## Tables A table is plain GitHub-flavoured markdown --- a header row, a delimiter row, then -the body. Nothing else is needed, and in particular **the scroll wrapper is not -yours to write.** The renderer's `table_open` rule wraps every table in -`
` on its way out, and that `tabindex` is -an accessibility fix rather than decoration: the wrapper is `overflow-x: auto`, and -a scroll container a keyboard user cannot focus cannot be scrolled without a -mouse. It is unconditional because whether a given table overflows depends on the -reader's viewport, so there is no answer to give at render time. `th` gets its -`scope="col"` from the same place. If you are writing a markdown-it plugin, do not -add a second wrapper: the outer one would have no `tabindex`, which is the shipped -fix undone. +the body. **Do not write the scroll wrapper.** The renderer's `table_open` rule +wraps every table in `
`. The wrapper is +`overflow-x: auto`, and the `tabindex` lets a keyboard user focus and scroll it. +`th` gets its `scope="col"` from the same place. If you are writing a markdown-it +plugin, do not add a second wrapper: the outer one would have no `tabindex`. **A literal `|` inside a cell must be escaped as `\|`,** or the parser reads it as -the next column boundary and the row silently gains a cell. Backticks are no -protection: the row is split into cells before inline parsing runs, so an -unescaped pipe inside a code span breaks the row exactly as a bare one does. It -comes up in this documentation more than in most, because `|` is twinBASIC's -separator for alternatives in a syntax line and the builder's own reference writes -type unions with it --- `[ ( True \| False ) ]`, `string\|undefined`. It is escaped -in 38 files and explained in none of them. +the next column boundary and the row silently gains a cell. Backticks do not +help: the row is split into cells before inline parsing runs, so an unescaped pipe +inside a code span breaks the row too. It comes up often here, because `|` +separates alternatives in a twinBASIC syntax line and the builder's reference +writes type unions with it --- `[ ( True \| False ) ]`, `string\|undefined`. Per-column alignment goes in the delimiter row, with a colon on the side the content should sit: `:---` left, `---:` right, `:---:` centred, and a bare `---` @@ -794,28 +703,22 @@ the default against 73 explicit left, 24 centred and 4 right. three columns, 16 have four, and five have five. Exactly one has more: `Reference/Default/VBA/Interaction/Partition.md` at seven, and it is the model to copy if you need one that wide. It right-aligns the three numeric argument columns -and left-aligns the four result columns, and it spends a sentence of prose before -the table explaining what the two least obvious column headings mean, which is -what makes seven columns readable at all. +and left-aligns the four result columns, and a sentence of prose before the table +explains the two least obvious column headings. **Every total in this section is a hand count of `docs/`, taken on 2026-09-21.** -No [count name](#counts) covers tables, and a number the build cannot derive does -not get one --- so nothing keeps these current, and each of them is already a -little wrong by the time you read it. The proportions are the durable part, and -the rarities are more durable still: two or three columns stays the norm, four -stays rare, five rarer, and seven stays one page, however many tables get -written. Quote the shape, and re-count before quoting a total. - -Check a wide table in the PDF as well as on the site, because the two behave -differently and only one of them degrades gracefully. `book.mjs` strips the -`table-wrapper` before assembling the book, and `print.css` sets `table { width: -100% }` with no overflow rule anywhere --- so a table too wide for the 170mm text -column has no scroll affordance to fall back on and simply runs into the margin. -Nothing reports it. Run `book.bat` and look at the page. +No [count name](#counts) covers tables, so the totals drift. The proportions last: +two or three columns is the norm, four is rare, five rarer, and seven is one page. +Re-count before quoting a total. + +Check a wide table in the PDF as well as on the site. The PDF has no scroll +wrapper (`book.mjs` strips it) and `print.css` sets `table { width: 100% }`, so a +table too wide for the 170mm text column runs into the margin, and nothing reports +it. Run `book.bat` and look at the page. ## Diagrams -A diagram is a Graphviz `.dot` file. Put it where it belongs: `docs/assets/images/dot/` if more than one page uses it, or in the `Images/` folder beside the page if only one does. The build renders an `.svg` next to it, inlines that into the page, and gives it the zoom and export controls. Both files belong in git. +A diagram is a Graphviz `.dot` file. Put it in `docs/assets/images/dot/` if more than one page uses it, or in the `Images/` folder beside the page if only one does. The build renders an `.svg` next to it, inlines that into the page, and gives it the zoom and export controls. Commit both files. ``` docs/Tutorials/CEF/Images/MonacoArchitecture.dot <-- you write this @@ -823,8 +726,7 @@ docs/Tutorials/CEF/Images/MonacoArchitecture.svg <-- the build writes this ``` **Reference the `.svg`, never the `.dot`**, and use the markdown image form rather -than a raw `` tag --- the same rule, and the same reason, as [Images](#images) -below: +than a raw `` tag, as under [Images](#images) below: ```markdown ![A twinBASIC form holding two CefBrowser controls. The left one hosts the Monaco @@ -832,11 +734,11 @@ editor and posts the edited HTML to a handler in twinBASIC code, which passes it the right control to render as a preview.](Images/MonacoArchitecture.svg) ``` -Name the font as the existing diagrams do --- copy the `node`, `edge` and `graph` blocks from one of them --- and let the build measure it. Copy from a diagram of the same kind: `docs/Tutorials/CEF/Images/MonacoArchitecture.dot` for one belonging to a page, `docs/assets/images/dot/toolchain-overview.dot` for a shared one. They differ in node `margin` --- `0.16,0.09` against `0.12,0.06` --- so a figure quoted for one family is not the other's. You do not need to do anything special for Inter: `builder/dot-metrics.mjs` installs its real widths into Graphviz before layout, and `check.bat` asserts afterwards that nothing overflowed. +Copy the `node`, `edge` and `graph` blocks from an existing diagram of the same kind, which name the font, and let the build measure it: `docs/Tutorials/CEF/Images/MonacoArchitecture.dot` for one belonging to a page, `docs/assets/images/dot/toolchain-overview.dot` for a shared one. They differ in node `margin` --- `0.16,0.09` against `0.12,0.06`. Nothing special is needed for Inter: `builder/dot-metrics.mjs` installs its real widths into Graphviz before layout, and `check.bat` asserts that nothing overflowed. -Two things to leave alone. Node text is deliberately dark against each node's light fill in both themes; only the cluster and edge labels follow the page's text colour, because those are the ones sitting on the page background. And a diagram needs alt text like any other image --- the whole diagram gets one accessible name, so describe what it shows, not what shapes it contains, and never `![Diagram]`. Everything under [Images](#images) about what makes alt text worth having, and about nothing in the build being able to tell you, applies here unchanged. +Leave node text alone: it is dark against each node's light fill in both themes. Only the cluster and edge labels follow the page's text colour, because they sit on the page background. A diagram needs alt text like any other image. The whole diagram gets one accessible name, so describe what it shows, not what shapes it contains, and never `![Diagram]`. What [Images](#images) says about alt text applies here. -A box can link somewhere: give its node, or its cluster, a `URL`, relative to the page that shows the diagram (`URL="Tools-Build#tbdocs"`). The diagram is then a group of links rather than one image, and the build moves each cluster's label just before the first box inside it, so that a reader, and the Tab key, takes each label and then its boxes. Declare the nodes in reading order and set `outputorder=nodesfirst` on the graph, so that the boxes come in that order too. `docs/Documentation/Images/Tools-Groups.dot` does both. The build's link check reports a link that leads nowhere, as it does for any other link. +A box can link somewhere: give its node, or its cluster, a `URL`, relative to the page that shows the diagram (`URL="Tools-Build#tbdocs"`). Give the box a `tooltip` as well: a pointer shows it on hover, and without one Graphviz shows the label, `
` and all. The diagram is then a group of links rather than one image, and the build moves each cluster's label just before the first box inside it, so the Tab key takes each label and then its boxes. Declare the nodes in reading order and set `outputorder=nodesfirst` on the graph, so the boxes come in that order too. `docs/Documentation/Images/Tools-Groups.dot` does both. The link check reports a link that leads nowhere. ## Images @@ -846,48 +748,48 @@ Images live in an `Images/` folder beside the page that uses them, and are refer ![The Package Manager panel, with arrows pointing at the PUBLISH THIS PACKAGE button](Images/packPublishButton.png) ``` -**The text in the square brackets is the image's alt text, and every image needs one.** Write what a reader who cannot see the picture would otherwise miss --- what the screenshot shows and which part of it the surrounding prose is pointing at --- rather than a label for the file. A caption-length phrase is usually right; the file name, the word *screenshot*, and the name of the button you happened to be looking for are not. +**The text in the square brackets is the image's alt text, and every image needs one.** Write what a reader who cannot see the picture would otherwise miss --- what the screenshot shows and which part of it the surrounding prose is pointing at. A caption-length phrase is usually right; the file name, the word *screenshot*, and the name of the button you happened to be looking for are not. -Nothing in the build will correct you, and it is worth knowing exactly why. The accessibility scan's `image-alt` rule asks only whether an accessible name *exists*, never whether it is the right one. `![image](…)` passes it. So does `![img](…)`. So does an empty `![](…)`, because an empty alt is the explicit signal that an image is decorative and a screen reader should skip it --- true of a spacer or a bullet glyph, never true of a screenshot. A green `check.bat` therefore says only that something is present in the brackets. Whether it describes the picture is a judgement no gate here makes, and the scan samples fourteen pages in any case, so most images are never looked at by it at all. +No gate checks the alt text. The accessibility scan's `image-alt` rule asks only whether an accessible name *exists*. `![image](…)` passes it. So does `![img](…)`. So does an empty `![](…)`, because an empty alt marks an image as decorative, which is right for a spacer and wrong for a screenshot. The scan also samples only fourteen pages. Whether the text describes the picture is for you and your reviewer to judge. -One habit to avoid in particular: copy the *shape* of a neighbouring page's image markdown, never its alt text. The example above used to read `![Create Package]`, and that one string travelled from it onto ten further images across the site --- a references dialog, a toolbox, a properties window, an animated GIF --- describing none of them, on pages that passed every check the project has. They have since been rewritten, but nothing except a reader would ever have found them. +Copy the *shape* of a neighbouring page's image markdown, never its alt text. The example above used to read `![Create Package]`, and that string spread onto ten further images --- a references dialog, a toolbox, a properties window, an animated GIF --- describing none of them. -Use the markdown form. A raw `` tag with a page-relative `src` is **not** rewritten when the PDF book is assembled: the book flattens every page into a single document, so a path like `Images/x.png` that resolves correctly on the site resolves against the book root instead and the render aborts with `pdf: missing image`. The markdown form is rewritten to a section-qualified path and works in all three outputs. +Use the markdown form. A raw `` tag with a page-relative `src` is **not** rewritten when the PDF book is assembled. The book flattens every page into one document, so a path like `Images/x.png` resolves against the book root and the render aborts with `pdf: missing image`. The markdown form is rewritten to a section-qualified path and works in all three outputs. -**A finished page never *renders* an image from a remote URL.** Pasting a screenshot into a GitHub issue or pull request produces a `https://github.com/user-attachments/assets/...` link, and pasting that straight into a page is fine --- the build vendors it for you. The next local build downloads the file to `assets/attachments/gh-.` and substitutes that path while rendering, so the published HTML, the offline mirror and the PDF all point at the local copy. Your markdown keeps the remote URL, which is why the downloaded file has to be committed along with your edit: it is the only copy the site has. Any *other* remote host has no such handling --- download it yourself, commit it under the section's `Images/` folder, and reference that path in the page. +**A finished page never *renders* an image from a remote URL.** Pasting a screenshot into a GitHub issue or pull request produces a `https://github.com/user-attachments/assets/...` link, and pasting that into a page is fine. The next local build downloads the file to `assets/attachments/gh-.` and substitutes that path while rendering, so the published HTML, the offline mirror and the PDF all point at the local copy. Your markdown keeps the remote URL, so commit the downloaded file with your edit: it is the only copy the site has. Any *other* remote host has no such handling --- download the image, commit it under the section's `Images/` folder, and reference that path. -There are three reasons, in increasing order of severity: +A remote image does harm in three ways, in increasing order of severity: - **Every page view pays a network round trip.** A GitHub attachment URL answers with a redirect to S3, so a single image costs two requests --- up to about a second on a cold connection. -- **The offline mirror stops being self-contained.** `_site-offline/` is meant to be browsable from a `file://` URL with no network at all. A remote image renders as a broken placeholder there. -- **The PDF book fails to build.** The forked paged.js in `book/lib/` dropped support for loading images asynchronously, so an image that has not finished downloading by the time the page-breaking pass runs raises an error and aborts the whole render. A remote image makes `book.bat` depend on a reachable third-party host; when that host is slow, blocked, or has expired the asset, the build does not degrade to a missing picture --- it stops. +- **The offline mirror stops being self-contained.** `_site-offline/` is meant to be browsable from a `file://` URL with no network. A remote image renders as a broken placeholder there. +- **The PDF book fails to build.** The forked paged.js in `book/lib/` cannot load images asynchronously, so an image still downloading when the page-breaking pass runs aborts the whole render. When the host is slow, blocked, or has expired the asset, `book.bat` stops. [`build.bat`](Building#checking-link-integrity) enforces this. A remote `` --- `http://`, `https://`, or protocol-relative `//host/...` --- is reported as `remote-asset:` and fails the run. ### A figure's buttons -An image that stands alone in its paragraph is shown as a *figure*: the image with a bar of buttons to download it, copy it and zoom it. That holds for a diagram, a picture in both themes and any other PNG, JPEG, GIF or WebP, with no markup of yours. An image inside a sentence, or inside a link, is a plain image and gets no bar, so put an image you want to be zoomable on a line of its own. +An image that stands alone in its paragraph is shown as a *figure*: the image with a bar of buttons to download it, copy it and zoom it. That holds for a diagram, a picture in both themes and any other PNG, JPEG, GIF or WebP. An image inside a sentence, or inside a link, is a plain image with no bar, so put an image you want to be zoomable on a line of its own. -Give a picture its `{:width="..." height="..."}`. The build uses the size to decide where the bar goes: a figure at least as tall as the bar stood on end gets the bar beside it, in the margin of a wide window and in view while the figure scrolls past, and a shorter figure gets the bar above it or at its top right. A picture with no stated size is treated as short: its bar stays above it, however tall it is. The decision is made in the build, so nothing on the page moves after it loads. +Give a picture its `{:width="..." height="..."}`. A figure at least as tall as the bar stood on end gets the bar beside it, in the margin of a wide window and in view while the figure scrolls past. A shorter figure gets the bar above it or at its top right. A picture with no stated size is treated as short, however tall it is. ### A picture in both themes -A screenshot of the IDE shows one of its themes, and the site has two. To give a picture a version for the site's light theme, save it beside the first one under the same name with `.light` before the extension: +A screenshot of the IDE shows one of its themes, and the site has two. To give a picture a version for the light theme, save it beside the first one under the same name with `.light` before the extension: ```text docs/IDE/Images/Watches.png <-- the dark picture: the page names this one docs/IDE/Images/Watches.light.png <-- the light picture: the page never names it ``` -Keep the page as it is, naming `Watches.png` with its `{:width height}`. The build finds the light picture by its name and puts both into the page: a reader sees the light one in the light theme and the dark one in the dark theme, whether the theme was chosen with the toggle or follows the operating system, and the browser downloads only the one it shows. The PDF book has the light picture only. +The page keeps naming `Watches.png` with its `{:width height}`. The build finds the light picture by its name and puts both into the page: a reader sees the light one in the light theme and the dark one in the dark theme, whether the theme was chosen with the toggle or follows the operating system, and the browser downloads only the one it shows. The PDF book has the light picture only. -Three things to get right: +Three rules: -- **Never name the `.light.png` in a page**, in an image, a link or raw HTML. The build stops and says which page did it, because that picture would then show in both themes. -- **Take the two pictures at the same scale.** The width and height in the page are for the dark picture, and the build shows the light one at the same scale: a light picture 8 pixels narrower is shown 4 pixels narrower when the page halves the dark one. The IDE's light theme lays some things out a few pixels smaller, so the two need not be exactly the same size. -- **A `.light.png` needs its dark picture beside it.** One without stops the build, because it would never be shown. +- **Never name the `.light.png` in a page**, in an image, a link or raw HTML. The build stops and names the page, because that picture would then show in both themes. +- **Take the two pictures at the same scale.** The width and height in the page are for the dark picture, and the build shows the light one at the same scale: a light picture 8 pixels narrower is shown 4 pixels narrower when the page halves the dark one. The two need not be exactly the same size, because the IDE's light theme lays some things out a few pixels smaller. +- **A `.light.png` needs its dark picture beside it.** One without stops the build. -The IDE's own screenshots come from `scripts/shoot_docs.mjs`, which takes each one in both themes and writes both files; it keeps no light picture that looks exactly like the dark one. Describe a picture in words that are true in both themes: its alt text and the prose around it are the same for both. "The Run button" is true of both, while "the white Run button" is not. +The IDE's own screenshots come from `scripts/shoot_docs.mjs`, which takes each one in both themes and writes both files; it keeps no light picture that looks exactly like the dark one. The alt text and the prose around a picture serve both themes, so describe it in words true in both: "The Run button", not "the white Run button". ### A picture as SVG @@ -900,11 +802,11 @@ docs/IDE/Images/Watches.light.png docs/IDE/Images/Watches.light.svg <-- shown in the light theme ``` -`scripts/shoot_docs.mjs --svg` draws each SVG from the IDE's page at the moment it takes the PNG. Its text is real text, in the fonts the IDE drew it with, so the picture stays sharp at any zoom, and in the PDF book its text can be selected and searched. The page still names `Watches.png`: the build shows the SVG in its place wherever there is one, in each theme. +`scripts/shoot_docs.mjs --svg` draws each SVG from the IDE's page at the moment it takes the PNG. Its text is real text, in the fonts the IDE drew it with, so the picture stays sharp at any zoom, and in the PDF book its text can be selected and searched. The page still names `Watches.png`, and the build shows the SVG in its place in each theme. -- **Name the PNG, never the SVG.** The PNG stays in the repository. The SVG is checked against it when it is made, and an output that shows PNGs uses it. Which outputs show the SVGs is set by `picture_svg` in `docs/_config.yml`: the website and the PDF book do, and the offline copy and the IDE help add-in's archive show the PNGs. The SVGs would make that archive half as large again. +- **Name the PNG, never the SVG.** The PNG stays in the repository, and an output that shows PNGs uses it. The `picture_svg` setting in `docs/_config.yml` decides which outputs show the SVGs: the website and the PDF book do, and the offline copy and the IDE help add-in's archive show the PNGs (the SVGs would make that archive half as large again). - **To keep one picture's PNG everywhere**, put `png` in its attribute block: `![The Watches panel](Images/Watches.png){:width="190" height="337" png}`. The build takes the word out of the page. -- **Do not edit a picture's SVG.** Take the picture again. `shoot_docs` writes an SVG only when it matches its PNG closely. One that has more than 0.25% of its pixels far from the PNG's is not written, and an older one is removed, so the page shows the PNG instead. +- **Do not edit a picture's SVG.** Take the picture again. `shoot_docs` writes an SVG only when it matches its PNG closely: if more than 0.25% of its pixels are far from the PNG's, it is not written, an older one is removed, and the page shows the PNG. ### Videos @@ -916,23 +818,23 @@ Link the video and mark it `.video`: The build renders that as a poster frame with a play button, linking out to the video page. The link text becomes the image's accessible name, so write the real title. -The poster frame is downloaded once to `assets/thumbnails/yt-.jpg` and committed; the build never re-fetches a thumbnail it already has. Do **not** hotlink `img.youtube.com` --- that reintroduces the third-party request, and [`build.bat`](Building#checking-link-integrity) fails the run for it, exactly as it does for any other remote ``. +The poster frame is downloaded once to `assets/thumbnails/yt-.jpg` and committed; the build never re-fetches it. Do **not** hotlink `img.youtube.com`: [`build.bat`](Building#checking-link-integrity) fails the run, as it does for any remote ``. -This is why the pages do not embed YouTube players. An `