Updated tBRegExp docs and tests (v1.9.0.0)

docs/Reference/Contributed/tbRegExp/PatternSyntax.md

@@ -0,0 +1,98 @@
+---
+title: Pattern syntax
+parent: tbRegExp Package
+permalink: /tB/Packages/Contributed/tbRegExp/PatternSyntax
+has_toc: false
+---
+
+# Pattern syntax
+{: .no_toc }
+
+How a pattern is written for **tbRegExp**, and the few places where that needs saying.
+
+A pattern written for VBScript's **RegExp** needs no change. What follows either adds to that syntax or spells out a rule VBScript follows without documenting it.
+
+## What tbRegExp adds
+
+**tbRegExp** accepts, beyond VBScript's syntax:
+
+- named groups, `(?<name>...)`: [**SubMatches.Item**](SubMatches/Item) returns one by its name, and [**Replace**](RegExp/Replace) inserts one as `$<name>`. A name starts with a letter or an underscore and goes on with letters, digits and underscores; a pattern with any other name is not valid;
+- lookbehind, `(?<=...)` and `(?<!...)`, as well as lookahead;
+- the options `i`, `m` and `s` inside the pattern: `(?i)` for the rest of the pattern, `(?i:...)` for one group.
+
+```tb check_build check_run
+Dim re As New RegExp
+re.Pattern = "(?<year>\d{4})-(?<month>\d{2})"
+Debug.Print re.Execute("2026-10")(0).SubMatches("month")  ' 10
+Debug.Print re.Replace("2026-10", "$<month>/$<year>")     ' 10/2026
+
+re.Pattern = "(?<=\$)\d+"
+Debug.Print re.Execute("costs $123")(0).Value            ' 123
+
+re.Pattern = "(?i:abc)DEF"
+Debug.Print re.Test("ABCDEF")                            ' True
+Debug.Print re.Test("ABCdef")                            ' False
+```
+
+## Escapes
+
+The escapes that name a control character or a character code are read as in VBScript:
+
+- `\c` followed by a letter, in either case, is a control character: `\cJ` and `\CJ` are the line feed. A `\c` followed by anything else makes the pattern invalid;
+- `\0` is the character with code 0, `\0` followed by octal digits, such as `\012`, is that code, and `\8` and `\9` are the digits themselves;
+- `\1` to `\9` and longer numbers are backreferences when the group they name has already been opened, that is, when its `(` stands earlier in the pattern. Otherwise the digits are read as an octal character code, so `\101` is `A` and `(a)\2` is `a` followed by the character with code 2; [**IsValid**](RegExp/IsValid) returns **True** for both;
+- `\f`, `\n`, `\r` and `\t`, inside a character class as well, are the form feed, line feed, carriage return and tab.
+
+<!-- test: PatternSyntaxTests.Test_NamedBackreferenceIsText, PatternSyntaxTests.Test_CapitalAAndZAreLetters -->
+A backslash before a letter that names no escape stands for the letter itself, as in VBScript. The constructs that other engines spell that way are therefore ordinary text here, not errors: `\k<name>` matches `k<name>`, `\A` matches `A`, and `\p{L}` matches `p{L}`, the braces being text as well.
+<!-- /test -->
+
+```tb check_build check_run
+Dim re As New RegExp
+re.Pattern = "\101"
+Debug.Print re.Test("A")                 ' True
+re.Pattern = "\CJ"
+Debug.Print re.Test(vbLf)                ' True
+re.Pattern = "\k<n>"
+Debug.Print re.IsValid                   ' True
+Debug.Print re.Test("k<n>")              ' True
+re.Pattern = "(a)\2"
+Debug.Print re.Test("a" & ChrW(2))      ' True
+```
+
+<!-- test: PatternSyntaxTests.Test_BackreferenceToLaterGroupMatchesNothing -->
+Digits that name a group declared *later* in the pattern are therefore not a backreference at all. `\1(a)` is the character with code 1 followed by `a`, and finds no match in the string `a`, as in VBScript's **RegExp**. A group that is open but has not yet taken part, such as the `\1` in `(\1a)`, is a real backreference and matches no characters.
+<!-- /test -->
+
+## Character classes
+
+`\w` matches only the ASCII letters and digits and `_`, as in VBScript, so `\w+` finds `caf` in `café`.
+
+## Quantifiers and braces
+
+A `{` starts a quantifier when it is followed by `{n}`, `{n,}` or `{n,m}`. A `{` that starts none, such as the one in `a{`, `a{x}` or `a{,2}`, is an ordinary character, as in VBScript; `\{` for the character is accepted too but not needed. A `]` or `}` that closes nothing is an ordinary character as well.
+
+<!-- test: PatternSyntaxTests.IsValid_BraceThatStartsNoQuantifier, PatternSyntaxTests.Test_BraceThatStartsNoQuantifierIsText, PatternSyntaxTests.Test_CommaAndMaximumOnlyIsText -->
+`{,m}` is text and not a quantifier, because a quantifier needs a count before the comma: `a{,2}` finds the text `a{,2}` and not up to two `a`. A `{` that does start a quantifier with counts that cannot be used is still an error, though: `a{2,1}`, whose upper count is below its lower one, makes the pattern invalid.
+<!-- /test -->
+
+```tb check_build check_run
+Dim re As New RegExp
+re.Pattern = "(?<_part2>\d)"
+Debug.Print re.IsValid                   ' True
+re.Pattern = "(?<2part>\d)"
+Debug.Print re.IsValid                   ' False
+
+re.Pattern = "a{x}"
+Debug.Print re.IsValid                   ' True
+Debug.Print re.Execute("a{x}")(0).Value  ' a{x}
+re.Pattern = "a{2,1}"
+Debug.Print re.IsValid                   ' False
+```
+
+## See Also
+
+- [Pattern](RegExp/Pattern) property -- the regular expression
+- [IsValid](RegExp/IsValid) property -- whether a pattern compiles
+- [LastErrorNumber](RegExp/LastErrorNumber) property -- the faults a pattern can have, with an example of each
+- [tbRegExp Package](./) -- overview

docs/Reference/Contributed/tbRegExp/RegExp/Execute.md

@@ -36,8 +36,8 @@ Debug.Print re.Execute("fooBarBaz").Count         ' 2
 Debug.Print re.Execute("fooBarBaz")(1).FirstIndex  ' 6
 ```
 
-<!-- test: SearchGivesUpTests.Execute_CatastrophicPatternKeepsMatchesFound -->
-A search that backtracks without end, such as `(a+)+b` on thirty `a`, gives up instead of hanging. The collection then holds the matches found before it gave up, and [**SearchAborted**](SearchAborted) returns **True**.
+<!-- test: SearchGivesUpTests.Execute_CatastrophicPatternKeepsNothing -->
+A search that backtracks without end, such as `(a+)+b` on thirty `a`, gives up instead of hanging, and **Execute** then raises an error. The matches found before it gave up are not returned: a part of the matches, handed back as though it were all of them, is worse than no answer. See [**SearchStepLimit**](SearchStepLimit).
 <!-- /test -->
 
 Each **Match** gives the matched text ([**Value**](../Match/Value)), the position where it starts ([**FirstIndex**](../Match/FirstIndex), counted from 0), its length, and the text of each group of the pattern ([**SubMatches**](../Match/SubMatches)). A group written `(?<name>...)` can be read by its name.

docs/Reference/Contributed/tbRegExp/RegExp/IsValid.md

@@ -24,22 +24,22 @@ The property compiles the pattern if it has not been compiled since [**Pattern**
 Check **IsValid** before [**Execute**](Execute), [**Test**](Test), [**Replace**](Replace) or [**Split**](Split) when the pattern comes from outside the program, because each of those raises the compile error.
 
 <!-- test: IsValidTests.IsValid_InvalidPatternLeavesNoError -->
-> [!NOTE] 1.8.0.0
-> Although nothing is raised, the compile error of an invalid pattern stays in the **Err** object after the call that compiled the pattern returns. A program that tests **Err** for other errors calls **Err.Clear** first.
+Nothing is raised and nothing is left behind. The **Err** object is clear when the property returns, even for a pattern that failed to compile, so a program that runs `On Error Resume Next` and tests **Err** for its own errors does not have to call **Err.Clear** first. The number and the message are kept in [**LastErrorNumber**](LastErrorNumber) and [**LastError**](LastError).
+<!-- /test -->
 
 <!-- test: IsValidTests.IsValid_LowCountAboveHigh, IsValidTests.LastErrorNumber_LowCountAboveHigh, IsValidTests.Test_LowCountAboveHighRaisesCompileError, IsValidTests.Execute_LowCountAboveHighRaisesCompileError, IsValidTests.Replace_LowCountAboveHighRaisesCompileError, IsValidTests.Split_LowCountAboveHighRaisesCompileError -->
-> [!NOTE] 1.8.0.0
-> A quantifier whose lower count is above its upper count, such as `a{2,1}`, is not found when the pattern is compiled, so **IsValid** returns **True** for it and [**LastErrorNumber**](LastErrorNumber) returns 0. [**Execute**](Execute), [**Test**](Test), [**Replace**](Replace) and [**Split**](Split) then raise error 3000 when a search reaches the quantifier, with the description that Windows gives that number, *The specified print monitor is unknown.*
+A quantifier whose lower count is above its upper count, such as `a{2,1}`, is found when the pattern is compiled, like any other fault: **IsValid** returns **False**, [**LastErrorNumber**](LastErrorNumber) gives the invalid-quantifier number, and [**Execute**](Execute), [**Test**](Test), [**Replace**](Replace) and [**Split**](Split) raise it. A `{` that starts no quantifier at all, such as the one in `a{x}`, is not a fault but an ordinary character.
+<!-- /test -->
 
 ## Example
 
 ```tb check_build check_run
 Dim re As RegExp
 Set re = New RegExp
 re.Pattern = "(\d+"
-Debug.Print re.IsValid                       ' False
-Debug.Print Err.Number = re.LastErrorNumber  ' True
-Err.Clear
+Debug.Print re.IsValid                          ' False
+Debug.Print Err.Number                          ' 0 -- nothing left in Err
+Debug.Print re.LastErrorNumber - vbObjectError  ' 3009
 
 re.Pattern = "(\d+)"
 Debug.Print re.IsValid  ' True

docs/Reference/Contributed/tbRegExp/RegExp/LastError.md

@@ -20,7 +20,7 @@ Returns a **String**: the description of the compile error, or an empty string w
 ## Remarks
 
 <!-- test: IsValidTests.LastError_InvalidPatternLeavesNoError -->
-The property compiles the pattern if it has not been compiled since [**Pattern**](Pattern) or [**IgnoreCase**](IgnoreCase) last changed. The compilation traps its error, so **LastError** raises nothing, but the error stays in the **Err** object (see [**IsValid**](IsValid)). The text is a sentence that names the fault, such as *Unterminated character class: a '[' without a matching ']'.* for the pattern `[abc`. [**LastErrorNumber**](LastErrorNumber) is the number of the error and lists the faults.
+The property compiles the pattern if it has not been compiled since [**Pattern**](Pattern) or [**IgnoreCase**](IgnoreCase) last changed. The compilation traps its error, so **LastError** raises nothing and leaves the **Err** object clear. The text is a sentence that names the fault, such as *Unterminated character class: a '[' without a matching ']'.* for the pattern `[abc`. [**LastErrorNumber**](LastErrorNumber) is the number of the error and lists the faults.
 <!-- /test -->
 
 [**Execute**](Execute), [**Test**](Test), [**Replace**](Replace) and [**Split**](Split) raise an error with this description when the pattern is invalid.

docs/Reference/Contributed/tbRegExp/RegExp/LastErrorNumber.md

@@ -22,7 +22,7 @@ Returns a **Long**: the number of the compile error, or 0 when the pattern is va
 The property compiles the pattern if it has not been compiled since [**Pattern**](Pattern) or [**IgnoreCase**](IgnoreCase) last changed. The compilation traps its error, so **LastErrorNumber** raises nothing. The value is the `Number` of the error that the compilation raised; [**LastError**](LastError) is its description.
 
 <!-- test: IsValidTests.LastErrorNumber_InvalidPatternLeavesNoError -->
-The engine raises its pattern errors with numbers of the form `vbObjectError + 3000 + n`, where *n* identifies the fault, as in the example. The compilation traps the error, so **LastErrorNumber** raises nothing, but the error stays in the **Err** object (see [**IsValid**](IsValid)).
+The engine raises its pattern errors with numbers of the form `vbObjectError + 3000 + n`, where *n* identifies the fault, as in the example. The compilation traps the error, so **LastErrorNumber** raises nothing and leaves the **Err** object clear.
 <!-- /test -->
 
 | *n* | Fault | Example |
@@ -32,15 +32,21 @@ The engine raises its pattern errors with numbers of the form `vbObjectError + 3
 | 3 | A range in a character class that runs backwards | `[z-a]` |
 | 4 | A character class without its `]` | `[abc` |
 | 6 | A group construct `(?<` that is neither a lookbehind nor a valid name | `(?<1a>x)` |
-| 8 | A `{` that does not start a quantifier, or a count that is too large | `a{2`, `\p{L}` |
+| 8 | A quantifier whose upper count is below its lower count, or a count that is too large | `a{2,1}`, `a{99999999999}` |
 | 9 | A group without its `)` | `(\d+` |
 | 11 | A `)` that closes no group | `)` |
 | 12 | A quantifier with nothing to repeat | `*a` |
 | 13 | A group name with a character that a name cannot hold | `(?<a-b>x)` |
 | 14 | An inline option that is not `i`, `m` or `s`, which includes an atomic group and a conditional | `(?x)`, `(?>a)` |
 
+<!-- test: PatternSyntaxTests.IsValid_AtomicGroup, PatternSyntaxTests.IsValid_Conditional -->
+An atomic group `(?>...)` and a conditional `(?(`*n*`)`*yes*`|`*no*`)` fall under *n* = 14. VBScript's **RegExp** does not accept either construct, so a pattern that uses one is invalid in both.
+<!-- /test -->
+
 [**Execute**](Execute), [**Test**](Test), [**Replace**](Replace) and [**Split**](Split) raise an error with this number when the pattern is invalid.
 
+Those four methods can also raise `vbObjectError + 3016`, which is not a compile error and so never appears here: it means a valid pattern ran out of its step budget before the search finished. See [**SearchStepLimit**](SearchStepLimit).
+
 ## Example
 
 ```tb check_build check_run

docs/Reference/Contributed/tbRegExp/RegExp/Pattern.md

@@ -34,7 +34,7 @@ Syntax: *object*.**Pattern** **=** *value*
 
 Setting a different pattern makes it compile again the next time it is needed. Setting the current pattern does nothing. A pattern that does not compile raises no error when it is set: [**IsValid**](IsValid), [**LastError**](LastError) and [**LastErrorNumber**](LastErrorNumber) report the failure, and [**Execute**](Execute), [**Test**](Test), [**Replace**](Replace) and [**Split**](Split) raise it.
 
-A named group is written `(?<name>...)`. Its text is read from the [**SubMatches**](../SubMatches/) of a [**Match**](../Match/) by name, and used in [**Replace**](Replace) as `$<name>`. A name starts with a letter or an underscore and goes on with letters, digits and underscores. The [overview of the package](../#pattern-syntax) lists the other differences from VBScript's pattern syntax.
+A named group is written `(?<name>...)`. Its text is read from the [**SubMatches**](../SubMatches/) of a [**Match**](../Match/) by name, and used in [**Replace**](Replace) as `$<name>`. A name starts with a letter or an underscore and goes on with letters, digits and underscores. [**Pattern syntax**](../PatternSyntax) covers the rest of the pattern syntax.
 
 ## Example
 

docs/Reference/Contributed/tbRegExp/RegExp/Replace.md

@@ -35,13 +35,12 @@ A `$` in *replaceString* starts a substitution item. Any other text is copied as
 | `$'` | The part of *sourceString* after the match. |
 | `$`*n* or `$`*nn* | The text of numbered group *n*, counting from 1. Two digits are read as the number when the pattern has a group with that number, otherwise one. It is an empty string when the group matched nothing. When the pattern has no group *n*, the `$` is copied, and so are the digits: `$0` and `$00` are never items, and `$3` in a pattern of two groups is the text `$3`. |
 | `$<`*name*`>` | The text of the group written `(?<name>...)`. It is an empty string when the group matched nothing or no group has the name. When the pattern has no named group, or the `>` is missing, or the name is empty, the `$` is copied. |
-| `$~` | Nothing. The item is dropped. |
 
 A `$` that starts no item raises no error, so a replacement string can hold a literal `$` as it is, at the end or before any other character. Before a character that would make an item, write `$$`: `$$$&` inserts a `$` and then the matched text, as the last example does.
 
 <!-- test: ReplaceTests.Replace_MatchAndContextItems, ReplaceTests.Replace_NumberedGroupItems, ReplaceTests.Replace_DollarThatStartsNoItemIsCopied, ReplaceTests.Replace_GroupThatTookNoPartIsEmpty, ReplaceTests.Replace_NamedGroupItem, ReplaceTests.Replace_TildeItemIsCopied -->
-> [!NOTE] 1.8.0.0
-> Items are read as in VBScript's **RegExp**, with two differences: `$<`*name*`>` is an addition of this package, and `$~` is dropped, where VBScript copies `$~` into the result.
+> [!NOTE] 1.9.0.0
+> Items are read as in VBScript's **RegExp**, with one difference: `$<`*name*`>` is an addition of this package. A `$~`, which earlier versions dropped, is copied into the result as VBScript copies it.
 
 ## Remarks
 
@@ -57,8 +56,8 @@ re.Pattern = "^"
 Debug.Print Replace(re.Replace("a" & vbLf & "b" & vbLf & "c", "> "), vbLf, "/")  ' > a/> b/> c
 ```
 
-<!-- test: SearchGivesUpTests.Replace_CatastrophicPatternReplacesMatchesFound -->
-A search that backtracks without end, such as `(a+)+b` on thirty `a`, gives up instead of hanging, and **Replace** then replaces only the matches found before it gave up. [**SearchAborted**](SearchAborted) returns **True** in that case.
+<!-- test: SearchGivesUpTests.Replace_CatastrophicPatternKeepsNothing -->
+A search that backtracks without end, such as `(a+)+b` on thirty `a`, gives up instead of hanging, and **Replace** then raises an error. No partly replaced string is returned. See [**SearchStepLimit**](SearchStepLimit).
 <!-- /test -->
 
 An invalid pattern raises the compile error, with the number in [**LastErrorNumber**](LastErrorNumber) and the description in [**LastError**](LastError). Check [**IsValid**](IsValid) first to avoid it.