From 2f2c38fc07a1ec8b753a065904b957eaf57268f2 Mon Sep 17 00:00:00 2001 From: antonsynd <2648730+antonsynd@users.noreply.github.com> Date: Thu, 22 Oct 2026 01:18:24 -0400 Subject: [PATCH 01/13] =?UTF-8?q?fix(core):=20a=20Sharpy=20class=20instanc?= =?UTF-8?q?e=20is=20never=20a=20plain=20CLR=20sequence=20=E2=80=94=20str?= =?UTF-8?q?=20and=20repr=20print=20=20and=20never=20iterat?= =?UTF-8?q?e=20it=20(refs=20#2057)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit R-HO (a). Str(object) and Repr(object) ran the #1453 plain-sequence arm before the R-CF instance fallback, so a user class with __iter__ (emitted as IEnumerable; with __next__ also as its own IEnumerator) was formatted by enumerating it: `['item', 'item']` from str, `[]` from every later site, and a loop afterwards saw 0 items (per the 2026-10-15 cell, an endless __next__ hangs print). Both now run one Sharpy-instance arm, TryFormatSharpyInstance (Str.cs), before the sequence arm — one predicate, two callers. The predicate is the compiler's stamp (new PyFormat.IsSharpyStamped: a SharpyName/SharpyModule/SharpyModuleType attribute on the type or its declaring chain) and not [CompilerGenerated], then the existing TryFormatInstanceFallback. Not the fallback's wider IsSharpyDeclared: its "nested in a static class" rule also matches LINQ's Enumerable+IteratorSelectIterator, and a first cut with it moved bare_clr_sequence_display_1453 (`['c', 'b', 'a']` -> `<__main__.IteratorSelectIterator object>`). The [CompilerGenerated] exclusion keeps a generator's state machine (nested in its stamped declaring type) on the #1453 arm. IsSharpyDeclared is re-expressed as IsSharpyStamped || nested-in-static (same truth table), so the attribute triple has one spelling. Matrix (Core.Tests SharpyInstanceFormattingTests, C# classes mirroring `emit csharp` output): dunders {__iter__+__next__, +__str__, +__repr__, __iter__ as a generator, __len__+__getitem__, none} x site {str, repr, print, f-string hole (PyFormat.Apply ""), list element} = 30 rows, each also asserting a loop afterwards still sees 2 items (or the class is not enumerable). The +__str__ repr and list-element rows assert only the no-iteration half (python: <__main__.Counter object>; still `Counter!` until R-CP). Plus 2 negative controls: a generator function's result and a LINQ select iterator keep `[1, 2]` / `['c', 'b', 'a']`. Run fixture builtins/iterable_instance_str_2057 (.spy + .expected), loop counts 2 and 3, python3 3.12 output modulo the address. Mutations (cp backup, shutil.copy + utime restore, rebuild): - M1 swap the arms back (delete the Sharpy-instance arm in Str and Repr): theory 10 red / 22 green (exactly iter-next x5 + iter-generator x5); fixture iterable_instance_str_2057 red, bare_clr_sequence_display_1453 green. - M2 drop the [CompilerGenerated] exclusion: GeneratorFunctionResult_KeepsThePlainSequenceRendering red, 31 green. - M3 drop the stamp test (wider nested-in-static rule): LinqIteratorNestedInStaticClass_KeepsThePlainSequenceRendering red, 31 green; fixture bare_clr_sequence_display_1453 red. - Restored: theory 32/0/0; the two fixtures 2/0/0. Direction (`sharpyc run`; base = this worktree's binary @ e7e0a0ef7 before editing; after = this commit's tree): - __iter__/__next__ only, sites str/repr/print/f"{c}"/[c] + loop: `['item', 'item']` `[]` `[]` `[]` `[[]]` / 0 -> `<__main__.Counter object>` x4, `[<__main__.Counter object>]` / 2. - __iter__ as a generator: `[1, 2]` x4, `[[1, 2]]` / 2 -> `<__main__.Bag object>` x4, `[<__main__.Bag object>]` / 2. - unchanged: +__str__ (`Counter!`), +__repr__ (`Counter()`), plain and __len__+__getitem__ (`<__main__.X object>`), a generator function (`[1, 2]`), map (``), a generator expression (``), `[ch for ch in reversed("abc")]`. - stdlib, same class (base read under M1, which is the base arm order): sqlite3 cursor after SELECT: print `[[1], [2]]` then fetchall() 0 -> `` then 2 (python: ``, 2); csv.reader: `[['a', 'b'], ['c', 'd']]` -> `` (python `<_csv.reader object at ...>`; the name text is the #2104 Stdlib repr family), loop 2 both. Fixture-hit count: 0 existing .expected moved (one new fixture). Filtered counts, measured on a tree byte-identical to this commit (on base e7e0a0ef7): - Sharpy.Core.Tests (whole): 4021/0/0 - Sharpy.Stdlib.Tests (whole): 4273/0/0 - Compiler.Tests FileBasedIntegrationTests|DifferentialExecution: 3586/0/0 Co-Authored-By: Claude Opus 5.5 (1M context) Claude-Session: https://claude.ai/code/session_0126pNZqmMsSbn4WFFPhpnxW --- .../iterable_instance_str_2057.expected | 10 + .../builtins/iterable_instance_str_2057.spy | 44 +++ .../SharpyInstanceFormattingTests.cs | 252 ++++++++++++++++++ src/Sharpy.Core/PyFormat.cs | 28 +- src/Sharpy.Core/Repr.cs | 9 + src/Sharpy.Core/Str.cs | 42 ++- 6 files changed, 378 insertions(+), 7 deletions(-) create mode 100644 src/Sharpy.Compiler.Tests/Integration/TestFixtures/builtins/iterable_instance_str_2057.expected create mode 100644 src/Sharpy.Compiler.Tests/Integration/TestFixtures/builtins/iterable_instance_str_2057.spy create mode 100644 src/Sharpy.Core.Tests/SharpyInstanceFormattingTests.cs diff --git a/src/Sharpy.Compiler.Tests/Integration/TestFixtures/builtins/iterable_instance_str_2057.expected b/src/Sharpy.Compiler.Tests/Integration/TestFixtures/builtins/iterable_instance_str_2057.expected new file mode 100644 index 000000000..acc13ccea --- /dev/null +++ b/src/Sharpy.Compiler.Tests/Integration/TestFixtures/builtins/iterable_instance_str_2057.expected @@ -0,0 +1,10 @@ +<__main__.Counter object> +<__main__.Counter object> +<__main__.Counter object> +<__main__.Counter object> +[<__main__.Counter object>] +2 +<__main__.Bag object> +<__main__.Bag object> +[<__main__.Bag object>] +3 diff --git a/src/Sharpy.Compiler.Tests/Integration/TestFixtures/builtins/iterable_instance_str_2057.spy b/src/Sharpy.Compiler.Tests/Integration/TestFixtures/builtins/iterable_instance_str_2057.spy new file mode 100644 index 000000000..5dbc201c3 --- /dev/null +++ b/src/Sharpy.Compiler.Tests/Integration/TestFixtures/builtins/iterable_instance_str_2057.spy @@ -0,0 +1,44 @@ +# A Sharpy class instance is never formatted as a sequence (#2057, R-HO): without __str__ or +# __repr__ it prints at every site, and formatting it never iterates it, so a +# loop afterwards still sees every item. + +class Counter: + index: int + + def __init__(self): + self.index = 0 + + def __iter__(self) -> Self: + return self + + def __next__(self) -> str: + if self.index >= 2: + raise StopIteration() + self.index += 1 + return "item" + +class Bag: + def __iter__(self) -> int: + yield 1 + yield 2 + +def main(): + c = Counter() + print(str(c)) + print(repr(c)) + print(c) + print(f"{c}") + print([c]) + n = 0 + for v in c: + n += 1 + print(n) + + b = Bag() + print(b) + print(repr(b)) + print([b]) + total = 0 + for x in b: + total += x + print(total) diff --git a/src/Sharpy.Core.Tests/SharpyInstanceFormattingTests.cs b/src/Sharpy.Core.Tests/SharpyInstanceFormattingTests.cs new file mode 100644 index 000000000..101e6df62 --- /dev/null +++ b/src/Sharpy.Core.Tests/SharpyInstanceFormattingTests.cs @@ -0,0 +1,252 @@ +using System; +using System.Collections; +using System.Collections.Generic; +using System.IO; +using FluentAssertions; +using Xunit; + +namespace Sharpy.Core.Tests; + +/// +/// #2057, R-HO: an instance of a Sharpy class is never a plain CLR sequence. A class without +/// __str__/__repr__ prints python's <mod.C object> (R-CF, the address +/// omitted) at every formatting site, and formatting it never iterates it. Before the cure +/// and ran the #1453 +/// plain-sequence arm first, so a class with __iter__/__next__ printed +/// ['item', 'item'] from str and [] everywhere after, and a loop afterwards saw +/// 0 items. The nested classes mirror what the compiler emits (read from emit csharp): +/// __iter__ returning Self plus __next__ makes the class its own +/// IEnumerator<string>; a generator __iter__ is a re-enumerable +/// IEnumerable<int>; __str__ and __repr__ are each a ToString +/// override until R-CP gives __repr__ its own IRepr member. +/// python3 3.12, each class defined in __main__: str/repr/print/ +/// f"{c}" give <__main__.Counter object at …>, [c] gives +/// [<__main__.Counter object at …>], and a loop afterwards counts 2. +/// +[Collection("ConsoleCapture")] +public class SharpyInstanceFormattingTests : IDisposable +{ + private readonly TextWriter _originalOut = Console.Out; + + public void Dispose() + { + Console.SetOut(_originalOut); + } + + /// `class Counter` with `__iter__ -> Self` and `__next__` only. + [SharpyModuleType("__main__", "Counter")] + public class IterOnlyCounter : IEnumerator, IEnumerable + { + private int _index; + private string _current = ""; + + public IEnumerator GetEnumerator() => this; + IEnumerator IEnumerable.GetEnumerator() => GetEnumerator(); + + public bool MoveNext() + { + if (_index >= 2) + { + return false; + } + + _index++; + _current = "item"; + return true; + } + + public string Current => _current; + object IEnumerator.Current => _current; + public void Reset() => throw new NotSupportedException(); + + public void Dispose() + { + } + } + + /// The same class plus `__str__` returning "Counter!". + [SharpyModuleType("__main__", "Counter")] + public class StrCounter : IterOnlyCounter + { + public override string ToString() => "Counter!"; + } + + /// The same class plus `__repr__` returning "Counter()". + [SharpyModuleType("__main__", "Counter")] + public class ReprCounter : IterOnlyCounter + { + public override string ToString() => "Counter()"; + } + + /// `class Bag` whose `__iter__` is a generator (re-enumerable). + [SharpyModuleType("__main__", "Bag")] + public class GeneratorBag : IEnumerable + { + public IEnumerator GetEnumerator() + { + yield return 1; + yield return 2; + } + + IEnumerator IEnumerable.GetEnumerator() => GetEnumerator(); + } + + /// `class Seq` with `__len__` and `__getitem__` (positive control: not enumerable). + [SharpyModuleType("__main__", "Seq")] + public class LenGetitemSeq : ISized + { + public virtual int Count => 2; + public string this[int i] => "item"; + } + + /// `class Plain` with no dunders (positive control). + [SharpyModuleType("__main__", "Plain")] + public class Plain + { + public int X = 1; + } + + private static object Make(string dunders) => dunders switch + { + "iter-next" => new IterOnlyCounter(), + "iter-next+str" => new StrCounter(), + "iter-next+repr" => new ReprCounter(), + "iter-generator" => new GeneratorBag(), + "len+getitem" => new LenGetitemSeq(), + "none" => new Plain(), + _ => throw new ArgumentOutOfRangeException(nameof(dunders), dunders, null), + }; + + private string Format(object value, string site) + { + switch (site) + { + case "str": + return Builtins.Str(value); + case "repr": + return Builtins.Repr(value); + case "print": + var writer = new StringWriter(); + Console.SetOut(writer); + Builtins.Print(value); + Console.SetOut(_originalOut); + return writer.ToString().TrimEnd('\r', '\n'); + case "fstring-hole": + return PyFormat.Apply(value, ""); + case "list-element": + return Builtins.Str(new List { value }); + default: + throw new ArgumentOutOfRangeException(nameof(site), site, null); + } + } + + /// + /// dunder set × site. expected is the R-CF/python text; null marks the R-CP cell + /// (a __str__-only class's repr is python's <mod.C object>, still + /// Counter! here until R-CP lands on #2057), where only the no-iteration half of the + /// contract is asserted. itemsAfter is what a loop sees after the site formatted the + /// value (python: 2), or -1 for a class that is not iterable. + /// + [Theory] + [InlineData("iter-next", "str", "<__main__.Counter object>", 2)] + [InlineData("iter-next", "repr", "<__main__.Counter object>", 2)] + [InlineData("iter-next", "print", "<__main__.Counter object>", 2)] + [InlineData("iter-next", "fstring-hole", "<__main__.Counter object>", 2)] + [InlineData("iter-next", "list-element", "[<__main__.Counter object>]", 2)] + [InlineData("iter-next+str", "str", "Counter!", 2)] + [InlineData("iter-next+str", "repr", null, 2)] + [InlineData("iter-next+str", "print", "Counter!", 2)] + [InlineData("iter-next+str", "fstring-hole", "Counter!", 2)] + [InlineData("iter-next+str", "list-element", null, 2)] + [InlineData("iter-next+repr", "str", "Counter()", 2)] + [InlineData("iter-next+repr", "repr", "Counter()", 2)] + [InlineData("iter-next+repr", "print", "Counter()", 2)] + [InlineData("iter-next+repr", "fstring-hole", "Counter()", 2)] + [InlineData("iter-next+repr", "list-element", "[Counter()]", 2)] + [InlineData("iter-generator", "str", "<__main__.Bag object>", 2)] + [InlineData("iter-generator", "repr", "<__main__.Bag object>", 2)] + [InlineData("iter-generator", "print", "<__main__.Bag object>", 2)] + [InlineData("iter-generator", "fstring-hole", "<__main__.Bag object>", 2)] + [InlineData("iter-generator", "list-element", "[<__main__.Bag object>]", 2)] + [InlineData("len+getitem", "str", "<__main__.Seq object>", -1)] + [InlineData("len+getitem", "repr", "<__main__.Seq object>", -1)] + [InlineData("len+getitem", "print", "<__main__.Seq object>", -1)] + [InlineData("len+getitem", "fstring-hole", "<__main__.Seq object>", -1)] + [InlineData("len+getitem", "list-element", "[<__main__.Seq object>]", -1)] + [InlineData("none", "str", "<__main__.Plain object>", -1)] + [InlineData("none", "repr", "<__main__.Plain object>", -1)] + [InlineData("none", "print", "<__main__.Plain object>", -1)] + [InlineData("none", "fstring-hole", "<__main__.Plain object>", -1)] + [InlineData("none", "list-element", "[<__main__.Plain object>]", -1)] + public void SharpyInstance_FormatsAsItself_AndIsNotIterated(string dunders, string site, string? expected, int itemsAfter) + { + var value = Make(dunders); + + var text = Format(value, site); + + if (expected != null) + { + text.Should().Be(expected); + } + + text.Should().NotContain("item", "formatting must not enumerate the instance"); + if (itemsAfter >= 0) + { + var seen = 0; + foreach (var unused in (IEnumerable)value) + { + seen++; + } + + seen.Should().Be(itemsAfter, "formatting the instance must not consume it"); + } + else + { + value.Should().NotBeAssignableTo(); + } + } + + /// + /// Positive control for the compiler-generated exclusion: a generator function's state machine + /// is nested in a Sharpy module class, yet it is the lazy sequence #1453's arm renders as a list + /// — not a Sharpy class instance. python3 prints <generator object gen at …>; this + /// pins only that the #2057 cure leaves it where it was. + /// + [Fact] + public void GeneratorFunctionResult_KeepsThePlainSequenceRendering() + { + IEnumerable generated = GeneratorModule.Gen(); + + Builtins.Str(generated).Should().Be("[1, 2]"); + Builtins.Repr(generated).Should().Be("[1, 2]"); + } + + /// + /// Negative control for the stamp test: "abc".reverse() is LINQ's select iterator, + /// a CLR type nested in the static System.Linq.Enumerable — which the instance + /// fallback's "nested in a static class" rule would call Sharpy-declared. It is the plain + /// sequence #1453 renders as the list its semantic type says it is. + /// + [Fact] + public void LinqIteratorNestedInStaticClass_KeepsThePlainSequenceRendering() + { + IEnumerable reversed = System.Linq.Enumerable.Select(new[] { 'c', 'b', 'a' }, c => c.ToString()); + var declaring = reversed.GetType().DeclaringType; + declaring.Should().NotBeNull(); + declaring!.IsAbstract.Should().BeTrue("the precondition is a type nested in a static class"); + declaring.IsSealed.Should().BeTrue(); + + Builtins.Str(reversed).Should().Be("['c', 'b', 'a']"); + Builtins.Repr(reversed).Should().Be("['c', 'b', 'a']"); + } + + [SharpyModule("__main__")] + public static class GeneratorModule + { + public static IEnumerable Gen() + { + yield return 1; + yield return 2; + } + } +} diff --git a/src/Sharpy.Core/PyFormat.cs b/src/Sharpy.Core/PyFormat.cs index d3306ed0b..de8992816 100644 --- a/src/Sharpy.Core/PyFormat.cs +++ b/src/Sharpy.Core/PyFormat.cs @@ -534,6 +534,29 @@ internal static bool TryFormatInstanceFallback(object value, out string formatte /// its declaring chain, or nested in a static (module) class. /// private static bool IsSharpyDeclared(Type type) + { + if (IsSharpyStamped(type)) + { + return true; + } + + Type top = type; + while (top.DeclaringType != null) + { + top = top.DeclaringType; + } + + return top != type && top.IsAbstract && top.IsSealed; + } + + /// + /// Whether the type or a type it is nested in carries a Sharpy name/module attribute — the + /// stamp the compiler puts on what it emits from Sharpy source. Narrower than + /// : a CLR type merely nested in a static class (LINQ's + /// Enumerable+IteratorSelectIterator) is not stamped, which is what keeps the + /// #2057 Sharpy-instance arm of off a plain sequence. + /// + internal static bool IsSharpyStamped(Type type) { for (Type? current = type; current != null; current = current.DeclaringType) { @@ -543,11 +566,6 @@ private static bool IsSharpyDeclared(Type type) { return true; } - - if (current.DeclaringType == null) - { - return current != type && current.IsAbstract && current.IsSealed; - } } return false; diff --git a/src/Sharpy.Core/Repr.cs b/src/Sharpy.Core/Repr.cs index fa61c0cdc..29e46cfc2 100644 --- a/src/Sharpy.Core/Repr.cs +++ b/src/Sharpy.Core/Repr.cs @@ -81,6 +81,14 @@ public static string Repr(object? obj) return FormatValueTuple(obj, type); } + // Same two arms as Str's, in the same order: a Sharpy class instance before the plain + // sequence (#2057, R-HO), so a class with __iter__ reprs as `` and is not + // consumed by being formatted. + if (TryFormatSharpyInstance(obj, out var sharpyInstanceRepr)) + { + return sharpyInstanceRepr; + } + // Same arm as Str's, for the same reason the exception arm above is in both places: a // container holding a plain CLR sequence must not render one way through print() and // another through repr() (#1453). @@ -90,6 +98,7 @@ public static string Repr(object? obj) } // Python's default object.__repr__ for a Sharpy type that does not render itself (#2006). + // Reached by what TryFormatSharpyInstance leaves alone: compiler-generated or unstamped types. if (PyFormat.TryFormatInstanceFallback(obj, out var instanceRepr)) { return instanceRepr; diff --git a/src/Sharpy.Core/Str.cs b/src/Sharpy.Core/Str.cs index 4c82e37b2..aa12be6ed 100644 --- a/src/Sharpy.Core/Str.cs +++ b/src/Sharpy.Core/Str.cs @@ -92,15 +92,23 @@ public static string Str(object x) return FormatValueTuple(x, type); } + // An instance of a Sharpy class is never a plain CLR sequence: it prints python's + // ``, and formatting it never iterates it (#2057, R-HO). Before the + // sequence arm, or a class with __iter__ prints its items and consumes itself. + if (TryFormatSharpyInstance(x, out var sharpyInstanceStr)) + { + return sharpyInstanceStr; + } + // A plain CLR sequence renders as the list its semantic type says it is, rather than - // leaking the name of whatever LINQ node produced it (#1453). Last, so every arm above - // keeps precedence. + // leaking the name of whatever LINQ node produced it (#1453). if (TryFormatPlainClrSequence(x, out var sequenceStr)) { return sequenceStr; } // Python's default object.__str__ for a Sharpy type that does not render itself (#2006). + // Reached by what TryFormatSharpyInstance leaves alone: compiler-generated or unstamped types. if (PyFormat.TryFormatInstanceFallback(x, out var instanceStr)) { return instanceStr; @@ -109,6 +117,36 @@ public static string Str(object x) return x.ToString() ?? ""; } + /// + /// The Sharpy-instance arm and both run + /// BEFORE (#2057, R-HO): python's default + /// <mod.C object> for an instance of a Sharpy class that does not render + /// itself. A class with __iter__ is emitted as IEnumerable<T> (and with + /// __next__ as its own enumerator), so the sequence arm would print its items and + /// consume it — the loop after a print(c) saw nothing. + /// + /// + /// The test is the compiler's stamp (), not the + /// fallback's wider "nested in a static class" rule: LINQ's + /// Enumerable+IteratorSelectIterator is nested in the static Enumerable and is + /// exactly the plain sequence #1453 renders as a list. A compiler-generated type is not a + /// Sharpy class either: a generator's state machine is nested in the (stamped) type that + /// declares it, and is likewise the lazy sequence #1453's arm renders. Both are left to that + /// arm, unchanged. + /// + private static bool TryFormatSharpyInstance(object x, out string formatted) + { + var type = x.GetType(); + if (!PyFormat.IsSharpyStamped(type) + || Attribute.IsDefined(type, typeof(System.Runtime.CompilerServices.CompilerGeneratedAttribute), inherit: false)) + { + formatted = string.Empty; + return false; + } + + return PyFormat.TryFormatInstanceFallback(x, out formatted); + } + /// /// Return the string unchanged. /// From bdd442f44a8623032a3450c38e8fb663fbbfb492 Mon Sep 17 00:00:00 2001 From: antonsynd <2648730+antonsynd@users.noreply.github.com> Date: Thu, 22 Oct 2026 01:39:39 -0400 Subject: [PATCH 02/13] fix(threading): a Thread is named Thread-N or MainThread, and main_thread(), enumerate() and active_count() report CPython's process-wide registry (refs #2261) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit #2261 part 2 (R-FD a, R-FF b): the process-wide state part 1 left out. Change. New Threading/ThreadRegistry.cs models CPython 3.12's threading.py: `_counter` (itertools.count(1), shared by Thread-N and Dummy-N; Interlocked), `_active` ∪ `_limbo` (a list under one lock, as CPython's _active_limbo_lock), and `_main_thread` (captured on the thread that first reaches the class, as CPython's `_MainThread()` runs on the importing thread; an explicit static constructor makes the first touch exact). `Thread.name` is a non-null `str`: the constructor's name, else `Thread-N` (an empty name also takes a number, as python's `if name:`), `MainThread` for the main thread. start() registers the thread before its .NET thread starts; the body's finally removes it, so it is listed while it runs and gone when join() returns. current_thread() returns the started Thread object itself (or the main thread, or a daemon `Dummy-N` that stays listed, as python's _DummyThread); main_thread() is the one main object; enumerate()/active_count() read the registry instead of the caller alone / the OS thread count. The ` (target)` suffix (`Thread-1 (f)`) is NOT here: it is a cell on #2105 (R-FD). Matrix (ThreadStateMatrixTests, extended; one program, 126 lines vs python3 3.12.13): operation {start, join, join(0.01), daemon get/set, is_alive, name get (new), name set} × state {initial, alive, finished} × host {Thread, Timer, Sub}; the current_thread() wrapper on main and worker and a worker on its own Thread; NEW: module function {current_thread().name, main_thread().name, main_thread() is current_thread(), ident equality, main_thread().is_alive(), main_thread().daemon, active_count(), len(enumerate()), enumerate() names} × caller {main, worker} + after the join; enumerate() membership and active_count() per host × state; current_thread() IS the host on each host's own thread. Default names are checked as numbers (Thread-16 … Thread-93). The assertion strips exactly " ()" from the python lines (R-FD, #2105). Concurrency (P54 harness): two new roster rows ThreadRegistry._counter and ThreadRegistry._active (ExpectedWrittenGlobals 8 → 10), each with an exerciser at {write, read, cross-thread} × {1, 2, 8} threads, python-verified facts in the doc comments; new ThreadingRegistry collection shared with the threading spy tests (the cells assert exact numbers and counts). Mutations (build + ThreadStateMatrix|ThreadNameCells|ThreadActiveCells, 19 tests): - M1 current_thread() a fresh wrapper: 1 red (matrix) / 18 green - M2 main_thread() returns the caller: 1 red (matrix) - M3 the body never unregisters: 7 red (matrix + 6 active cells) - M4 start() never registers: 7 red (matrix + 6 active cells) - M5 `++_counter` instead of Interlocked: 2 red (names write/2 8/4001, write/8 724/16001) - M6 Register/Unregister without the lock: 2 red (active write/8, read/8) - M7 Thread-N counted from 0: 1 red (matrix) Restored (cp + touch, rebuild): 19/19 green. Direction record (`sharpyc run` of one probe; python3 3.12.13 in brackets): current_thread().name, main_thread().name: None, None -> MainThread, MainThread [MainThread, MainThread]; Thread(f).name / Sub().name / Timer().name: None x3 -> Thread-1, Thread-2, Thread-3 [Thread-1 (f), Thread-2, Thread-3]; active_count(), len(enumerate()) on main: 9, 1 -> 1, 1 [1, 1]; from a worker, main ident == current ident, len(enumerate()), active_count(), current is its Thread, names: "True 1 10 True None None" -> "False 2 2 True Thread-4 MainThread" [False 2 2 True Thread-4 () MainThread]; after the join active_count(): 9 -> 1 [1]. Expected values changed: none. Totality literals extended for the new rows: ExpectedWrittenGlobals 8 -> 10 and the every-operation module list (+2). Out of this part's scope (issue comments): an exception on a thread ending the process and Timer(1e10) (its member) are the target-raises cell, an operation outside this matrix. Filtered @ this commit's tree (parent e7e0a0ef7) (measured): Sharpy.Stdlib.Tests 4291/0/0; Compiler.Tests Stdlib|Roster|Census|Concurrency|ProcessGlobalState 373/0/0; format whitespace --verify-no-changes clean; allowlist issue-state clean. Co-Authored-By: Claude Opus 5.5 (1M context) Claude-Session: https://claude.ai/code/session_0126pNZqmMsSbn4WFFPhpnxW --- docs/stdlib/threading.md | 12 ++ .../StdlibModuleGlobalConcurrencyTests.cs | 186 +++++++++++++++++- .../StdlibModuleGlobalRosterTests.cs | 21 +- .../Integration/ThreadStateMatrixTests.cs | 182 ++++++++++++++--- .../Spy/SpyTestCollections.cs | 18 ++ src/Sharpy.Stdlib/Threading/Thread.cs | 55 ++++-- src/Sharpy.Stdlib/Threading/ThreadRegistry.cs | 131 ++++++++++++ .../Threading/ThreadingModule.cs | 29 ++- 8 files changed, 584 insertions(+), 50 deletions(-) create mode 100644 src/Sharpy.Stdlib/Threading/ThreadRegistry.cs diff --git a/docs/stdlib/threading.md b/docs/stdlib/threading.md index a1b856ca0..7763a5dac 100644 --- a/docs/stdlib/threading.md +++ b/docs/stdlib/threading.md @@ -16,12 +16,24 @@ import threading ### `threading.current_thread() -> Thread` +Return the Thread object of the calling thread: the object `start()` was called +on, or the main thread. A thread the threading module did not start gets a daemon +`Dummy-N` thread, which stays in `enumerate()`. + ### `threading.active_count() -> int` +Return the number of Thread objects currently alive: the length of `enumerate()`. + ### `threading.main_thread() -> Thread` +Return the main Thread object, named `MainThread`: the thread that first used the +threading module, which in a Sharpy program is the thread `main()` runs on. + ### `threading.enumerate() -> list[Thread]` +Return a list of every Thread object currently alive: the main thread, every started +thread that has not finished, and the dummy threads `current_thread()` created. + ### `threading.lock() -> Lock` ### `threading.r_lock() -> RLock` diff --git a/src/Sharpy.Stdlib.Tests/Conformance/StdlibModuleGlobalConcurrencyTests.cs b/src/Sharpy.Stdlib.Tests/Conformance/StdlibModuleGlobalConcurrencyTests.cs index 071fc6e92..befa35283 100644 --- a/src/Sharpy.Stdlib.Tests/Conformance/StdlibModuleGlobalConcurrencyTests.cs +++ b/src/Sharpy.Stdlib.Tests/Conformance/StdlibModuleGlobalConcurrencyTests.cs @@ -78,6 +78,7 @@ internal sealed record KnownRedRow(bool Deterministic, string Evidence); { new RandomExerciser(), new SocketExerciser(), new CalendarExerciser(), new XmlExerciser(), new LoggingExerciser(), new NumpyRandomExerciser(), new HttpStatusExerciser(), new TypedLoadCacheExerciser(), + new ThreadNameExerciser(), new ThreadActiveExerciser(), }; public static IEnumerable Matrix(string module) @@ -102,7 +103,7 @@ public void EveryRosterRow_HasAnExerciser() Assert.Equal(Exercisers.Length, Exercisers.Select(e => e.Module).Distinct().Count()); } - /// Only the plan's six globals run every operation; a skipped operation names why it cannot race. + /// Only the plan's six globals and the two threading globals (#2261) run every operation; a skipped operation names why it cannot race. [Fact] public void EverySkippedOperation_NamesItsReason() { @@ -116,7 +117,7 @@ public void EverySkippedOperation_NamesItsReason() } Assert.Equal( - new[] { "calendar", "logging", "numpy.random", "random", "socket", "xml" }, + new[] { "calendar", "logging", "numpy.random", "random", "socket", "threading (active)", "threading (names)", "xml" }, Exercisers.Where(e => e.NotApplicable.Count == 0).Select(e => e.Module).OrderBy(m => m, StringComparer.Ordinal)); } @@ -205,6 +206,26 @@ public HttpStatusCells(ITestOutputHelper output) : base(output, "http.HTTPStatus public void Cell(string operation, int threads) => Run(operation, threads); } + [Collection("ThreadingRegistry")] + public sealed class ThreadNameCells : CellRunner + { + public ThreadNameCells(ITestOutputHelper output) : base(output, "threading (names)") { } + + [Theory] + [MemberData(nameof(Matrix), "threading (names)", MemberType = typeof(StdlibModuleGlobalConcurrencyTests))] + public void Cell(string operation, int threads) => Run(operation, threads); + } + + [Collection("ThreadingRegistry")] + public sealed class ThreadActiveCells : CellRunner + { + public ThreadActiveCells(ITestOutputHelper output) : base(output, "threading (active)") { } + + [Theory] + [MemberData(nameof(Matrix), "threading (active)", MemberType = typeof(StdlibModuleGlobalConcurrencyTests))] + public void Cell(string operation, int threads) => Run(operation, threads); + } + public sealed class TypedLoadCacheCells : CellRunner { public TypedLoadCacheCells(ITestOutputHelper output) : base(output, "json typed load") { } @@ -938,6 +959,167 @@ public override void Read(int threads, Tally tally) } } + /// + /// threading's name counter (#2261) — CPython 3.12 (verified 2026-10-22): threading._counter is + /// itertools.count(1), one __next__ per unnamed Thread; 8 threads x 2000 Thread() constructions + /// get pairwise distinct numbers (True), each thread's own numbers increase (True), and the next + /// Thread() afterwards is max + 1 (True) — no number is lost or repeated; a Thread() on one thread + /// then a Thread() on another is the next number. Every read of the counter is the increment that + /// writes it, so "read" checks each thread's own sequence while the others write. + /// + private sealed class ThreadNameExerciser : GlobalExerciser + { + public override string Field => "Sharpy.ThreadRegistry._counter"; + public override string Module => "threading (names)"; + + // An unnamed, never-started Thread: the name is drawn from the counter at construction. + private static int NextNumber() + { + var name = new Sharpy.Thread().Name; + return name.StartsWith("Thread-", StringComparison.Ordinal) + ? int.Parse(name.Substring("Thread-".Length), System.Globalization.CultureInfo.InvariantCulture) + : -1; + } + + public override void Write(int threads, Tally tally) + { + var before = NextNumber(); + var numbers = new int[threads][]; + OnThreads(threads, tally, id => + { + numbers[id] = new int[Iterations]; + for (var i = 0; i < Iterations; i++) + numbers[id][i] = NextNumber(); + }); + var after = OnFreshThread(NextNumber); + var all = numbers.Where(n => n != null).SelectMany(n => n).ToArray(); + var expected = Enumerable.Range(before + 1, threads * Iterations); + var misses = MultisetMisses(expected.Select(n => (double)n), all.Select(n => (double)n)); + tally.Count(all.Length, misses, + $"{misses} of {all.Length} names are not Thread-{before + 1}..Thread-{before + threads * Iterations} exactly once (a lost or repeated number)"); + tally.Check(() => after == before + threads * Iterations + 1, + () => $"after {threads * Iterations} constructions from Thread-{before} the next name is Thread-{after}, expected Thread-{before + threads * Iterations + 1}"); + } + + public override void Read(int threads, Tally tally) + { + OnThreads(threads, tally, _ => + { + var previous = NextNumber(); + for (var i = 0; i < Iterations; i++) + { + var current = NextNumber(); + var seen = previous; + tally.Check(() => current > seen, () => $"Thread-{current} after Thread-{seen} on the same thread"); + previous = current; + } + }); + } + + public override void CrossThread(int threads, Tally tally) + { + var written = new int[Iterations]; + Handoff(threads, tally, + i => written[i] = NextNumber(), + i => + { + var read = NextNumber(); + tally.Check(() => read == written[i] + 1, + () => $"iteration {i}: Thread-{read} after another thread's Thread-{written[i]}"); + }); + } + } + + /// + /// threading's active registry (#2261) — CPython 3.12 (verified 2026-10-22): threading._active and + /// _limbo are dicts read and written under _active_limbo_lock. 8 threads x 250 start()/join() of a + /// Thread: none is in enumerate() after its join (0 of 2000), and active_count() is back to its + /// value before (1 → 1); 8 threads each holding one started Thread see it in enumerate() on every + /// read (True) while the others start theirs; a Thread started on one thread is in enumerate() on + /// another (200 of 200), and is gone from it after the join (200 of 200). In Sharpy only the held + /// Threads are added — the harness's own threads are not Sharpy Threads and never call + /// current_thread() — so the count is the baseline plus the held threads. + /// + private sealed class ThreadActiveExerciser : GlobalExerciser + { + public override string Field => "Sharpy.ThreadRegistry._active"; + public override string Module => "threading (active)"; + + // A start()/join() costs a .NET thread; a quarter of the iterations keeps the 8-thread cell short. + private const int Starts = Iterations / 4; + + private static bool Listed(Sharpy.Thread thread) => ThreadingModule.Enumerate().Contains(thread); + + public override void Write(int threads, Tally tally) + { + var baseline = ThreadingModule.ActiveCount(); + OnThreads(threads, tally, _ => + { + for (var i = 0; i < Starts; i++) + { + var thread = new Sharpy.Thread(() => { }); + tally.Do(thread.Start); + tally.Do(() => thread.Join()); + tally.Check(() => !Listed(thread), () => $"{thread.Name} is in enumerate() after its join"); + } + }); + var after = ThreadingModule.ActiveCount(); + tally.Check(() => after == baseline, + () => $"active_count() is {after} after every start()/join(), {baseline} before"); + } + + public override void Read(int threads, Tally tally) + { + var baseline = ThreadingModule.ActiveCount(); + OnThreads(threads, tally, _ => + { + var hold = new Sharpy.Event(); + var held = new Sharpy.Thread(() => hold.Wait()); + held.Start(); + try + { + for (var i = 0; i < Starts; i++) + { + var count = -1; + tally.Check(() => Listed(held), () => $"{held.Name} (alive, started by this thread) is not in enumerate()"); + tally.Check(() => (count = ThreadingModule.ActiveCount()) >= baseline + 1 && count <= baseline + threads, + () => $"active_count() read {count}; {baseline + 1}..{baseline + threads} expected"); + } + } + finally + { + hold.Set(); + held.Join(); + } + }); + } + + public override void CrossThread(int threads, Tally tally) + { + var baseline = ThreadingModule.ActiveCount(); + var holds = new Sharpy.Event[Iterations]; + var started = new Sharpy.Thread[Iterations]; + Handoff(threads, tally, + i => + { + holds[i] = new Sharpy.Event(); + var hold = holds[i]; + started[i] = new Sharpy.Thread(() => hold.Wait()); + started[i].Start(); + }, + i => + { + var count = -1; + tally.Check(() => Listed(started[i]), () => $"iteration {i}: a Thread another thread started is not in enumerate()"); + tally.Check(() => (count = ThreadingModule.ActiveCount()) == baseline + 1, + () => $"iteration {i}: active_count() is {count} with one held thread, {baseline + 1} expected"); + holds[i].Set(); + tally.Do(() => started[i].Join()); + tally.Check(() => !Listed(started[i]), () => $"iteration {i}: the Thread is in enumerate() after its join"); + }); + } + } + /// A typed-load target: one all-fields constructor, as a dataclass emits. public sealed class P54Point { diff --git a/src/Sharpy.Stdlib.Tests/Conformance/StdlibModuleGlobalRosterTests.cs b/src/Sharpy.Stdlib.Tests/Conformance/StdlibModuleGlobalRosterTests.cs index b15734d99..0ab003dd3 100644 --- a/src/Sharpy.Stdlib.Tests/Conformance/StdlibModuleGlobalRosterTests.cs +++ b/src/Sharpy.Stdlib.Tests/Conformance/StdlibModuleGlobalRosterTests.cs @@ -36,7 +36,8 @@ namespace Sharpy.Stdlib.Tests.Conformance; /// A written static without a row is red (a new process-wide global needs its CPython model /// and synchronization named); a row the census no longer finds is red (stale — drain it). The /// totality is anchored to the literal , counted at landing -/// @ 486c9a88a (the plan's six plus two the census found), not derived from the scan. Each row is exercised at 1/2/8 threads by +/// @ 486c9a88a (the plan's six plus two the census found) plus the two threading globals #2261 part 2 +/// added, not derived from the scan. Each row is exercised at 1/2/8 threads by /// . /// public class StdlibModuleGlobalRosterTests @@ -89,13 +90,27 @@ internal sealed record GlobalRow(string Field, string File, string CPythonModel, new("Sharpy.TypedLoadContract.FieldCache", "src/Sharpy.Stdlib/TypedLoadContract.cs", "no CPython counterpart — an implementation cache whose value is a function of its key (the Type)", "ConcurrentDictionary", null), + + // ---- Added by #2261 part 2 (threading's process-wide state) ---- + + // Threading/ThreadRegistry.cs; every unnamed Thread() and every dummy current_thread() draws a number. + // CPython 3.12 (verified): threading._counter = itertools.count(1), shared by Thread-N and Dummy-N. + new("Sharpy.ThreadRegistry._counter", "src/Sharpy.Stdlib/Threading/ThreadRegistry.cs", + ProcessWideUnderTheGil + " (threading._counter, an itertools.count)", + "Interlocked.Increment — the only write", null), + // Threading/ThreadRegistry.cs; start() adds, the thread's own body removes, current_thread() adds a dummy. + // CPython 3.12 (verified): threading._active / _limbo, dicts read and written under _active_limbo_lock. + new("Sharpy.ThreadRegistry._active", "src/Sharpy.Stdlib/Threading/ThreadRegistry.cs", + "process-wide module global (threading._active and _limbo, under _active_limbo_lock)", + "lock (ThreadRegistry.ActiveLock) around every read and write", null), }; /// /// Written globals @ 486c9a88a, counted at landing — the totality anchor: the plan's six - /// (P54 cells 6–11) plus the two this census found (HTTPStatus._valueMap, TypedLoadContract.FieldCache). + /// (P54 cells 6–11) plus the two this census found (HTTPStatus._valueMap, TypedLoadContract.FieldCache) + /// — plus the two #2261 part 2 added (ThreadRegistry._counter, ThreadRegistry._active). /// - private const int ExpectedWrittenGlobals = 8; + private const int ExpectedWrittenGlobals = 10; /// /// Invoked member names that mutate their receiver — literal: the .NET collection mutators and diff --git a/src/Sharpy.Stdlib.Tests/Integration/ThreadStateMatrixTests.cs b/src/Sharpy.Stdlib.Tests/Integration/ThreadStateMatrixTests.cs index 9bd868051..dc8d7de75 100644 --- a/src/Sharpy.Stdlib.Tests/Integration/ThreadStateMatrixTests.cs +++ b/src/Sharpy.Stdlib.Tests/Integration/ThreadStateMatrixTests.cs @@ -7,19 +7,31 @@ namespace Sharpy.Stdlib.Tests.Integration; /// #2261: every threading.Thread operation gives python's value, or raises python's exception /// type, which a Sharpy except RuntimeError catches. Matrix: operation {start(), /// join(), join(0.01), daemon get, daemon = True, is_alive(), -/// name = "x" read back} × state {initial, alive, finished} × host {Thread, -/// Timer, a subclass}, plus the current_thread() wrapper on the main thread and on a -/// worker, and a worker that operates on its own Thread object. Every cell constructs a fresh -/// thread; the alive state is held by an Event (a Timer by its 30 s interval), so no -/// cell depends on timing. join() on an alive thread is not a cell: it waits for the thread -/// to finish, which join(0.01) covers without blocking. +/// name get, name = "x" read back} × state {initial, alive, finished} × host +/// {Thread, Timer, a subclass}, plus the current_thread() wrapper on the main +/// thread and on a worker, and a worker that operates on its own Thread object. Every cell +/// constructs a fresh thread; the alive state is held by an Event (a Timer by its +/// 30 s interval), so no cell depends on timing. join() on an alive thread is not a cell: it +/// waits for the thread to finish, which join(0.01) covers without blocking. +/// Part 2 (process-wide state) adds: module function {current_thread(), +/// main_thread(), enumerate(), active_count()} × caller {main thread, worker} +/// and after the worker's join; membership in enumerate() and active_count() for each +/// host in each state; and, on each host's own thread, that current_thread() IS the host +/// object. Every default name is checked as a number: Thread-N counts the unnamed threads +/// constructed, so a construction that consumed no number, or two, shifts every later row. /// Expected lines are the output of the same program written in python, run by python3 3.12.13 -/// on 2026-10-06 (python is the generator; the twin is this program line for line with python -/// spellings). Before #2261 the first join() before start() escaped as an unhandled -/// System.Threading.ThreadStateException, as did daemon = True and the daemon -/// read on a finished thread; join() on the current thread hung; and is_alive was a -/// property, so t.is_alive() was refused (SPY0230). The default name, -/// main_thread(), enumerate() and active_count() are #2261 part 2. +/// (part 1 on 2026-10-06, part 2 on 2026-10-22; python is the generator; the twin is this program +/// line for line with python spellings: target= for the positional target, no +/// @override). Python names a target thread Thread-N (target); Sharpy names it +/// Thread-N until the (target) suffix lands, which is a cell on #2105 (ruling R-FD), so +/// the assertion removes exactly (<lambda>) — every target here is a lambda — from the +/// python lines and nothing else. Before #2261 the first join() before start() +/// escaped as an unhandled System.Threading.ThreadStateException, as did +/// daemon = True and the daemon read on a finished thread; join() on the +/// current thread hung; and is_alive was a property, so t.is_alive() was refused +/// (SPY0230). Before part 2, name was None on every host and on both wrappers, +/// main_thread() on a worker was the worker, enumerate() held only the caller, and +/// active_count() counted the process's OS threads (9 after the join, python 1). /// public class ThreadStateMatrixTests : StdlibIntegrationTestBase { @@ -44,6 +56,18 @@ public ThreadStateMatrixTests(ITestOutputHelper output) : base(output) " self.hold.wait()", "", "", + "class Rec(threading.Thread):", + " seen: list[str]", + "", + " def __init__(self, seen: list[str]):", + " super().__init__()", + " self.seen = seen", + "", + " @override", + " def run(self) -> None:", + " self.seen.append(inside_row(self))", + "", + "", "def mk(host: str, hold: threading.Event) -> threading.Thread:", " if host == \"Thread\":", " return threading.Thread(lambda: hold.wait())", @@ -76,14 +100,74 @@ public ThreadStateMatrixTests(ITestOutputHelper output) : base(output) " return \"None\"", " if op == \"is_alive()\":", " return str(t.is_alive())", + " if op == \"name\":", + " return t.name", " t.name = \"x\"", " return str(t.name)", " except RuntimeError:", " return \"RuntimeError\"", "", "", + "def module_rows(caller: str) -> list[str]:", + " rows: list[str] = []", + " rows.append(f\"{caller} current_thread().name -> {threading.current_thread().name}\")", + " rows.append(f\"{caller} main_thread().name -> {threading.main_thread().name}\")", + " rows.append(f\"{caller} main_thread() is current_thread() -> {threading.main_thread() is threading.current_thread()}\")", + " rows.append(f\"{caller} main_thread().ident == current_thread().ident -> {threading.main_thread().ident == threading.current_thread().ident}\")", + " rows.append(f\"{caller} main_thread().is_alive() -> {threading.main_thread().is_alive()}\")", + " rows.append(f\"{caller} main_thread().daemon -> {threading.main_thread().daemon}\")", + " rows.append(f\"{caller} active_count() -> {threading.active_count()}\")", + " rows.append(f\"{caller} len(enumerate()) -> {len(threading.enumerate())}\")", + " names: list[str] = []", + " for th in threading.enumerate():", + " names.append(th.name)", + " names.sort()", + " rows.append(f\"{caller} enumerate() names -> {', '.join(names)}\")", + " return rows", + "", + "", + "def inside_row(t: threading.Thread) -> str:", + " me = threading.current_thread()", + " return f\"current_thread() is it {me is t}, in enumerate() {t in threading.enumerate()}, name {me.name}, active_count() {threading.active_count()}\"", + "", + "", + "def inside(host: str) -> str:", + " seen: list[str] = []", + " box: list[threading.Thread] = []", + " t: threading.Thread = Rec(seen)", + " if host == \"Thread\":", + " t = threading.Thread(lambda: seen.append(inside_row(box[0])))", + " if host == \"Timer\":", + " t = threading.Timer(0.0, lambda: seen.append(inside_row(box[0])))", + " box.append(t)", + " t.start()", + " t.join()", + " return seen[0]", + "", + "", "def main() -> None:", - " ops = [\"start()\", \"join()\", \"join(0.01)\", \"daemon\", \"daemon = True\", \"is_alive()\", \"name = \\\"x\\\"\"]", + " for row in module_rows(\"main\"):", + " print(row)", + " from_worker: list[list[str]] = []", + " w0 = threading.Thread(lambda: from_worker.append(module_rows(\"worker\")))", + " w0.start()", + " w0.join()", + " for row in from_worker[0]:", + " print(row)", + " print(f\"main after join active_count() -> {threading.active_count()}\")", + " print(f\"main after join len(enumerate()) -> {len(threading.enumerate())}\")", + " for host in [\"Thread\", \"Timer\", \"Sub\"]:", + " hold = threading.Event()", + " t = mk(host, hold)", + " print(f\"{host} initial in enumerate() -> {t in threading.enumerate()}, active_count() {threading.active_count()}\")", + " t.start()", + " print(f\"{host} alive in enumerate() -> {t in threading.enumerate()}, active_count() {threading.active_count()}\")", + " release(t, hold)", + " t.join()", + " print(f\"{host} finished in enumerate() -> {t in threading.enumerate()}, active_count() {threading.active_count()}\")", + " for host in [\"Thread\", \"Timer\", \"Sub\"]:", + " print(f\"{host} current thread -> {inside(host)}\")", + " ops = [\"start()\", \"join()\", \"join(0.01)\", \"daemon\", \"daemon = True\", \"is_alive()\", \"name\", \"name = \\\"x\\\"\"]", " for host in [\"Thread\", \"Timer\", \"Sub\"]:", " for st in [\"initial\", \"alive\", \"finished\"]:", " for op in ops:", @@ -101,8 +185,6 @@ public ThreadStateMatrixTests(ITestOutputHelper output) : base(output) " if st != \"initial\":", " t.join()", " for op in ops:", - " print(f\"main current_thread() {op} -> {apply(threading.current_thread(), op)}\")", - " for op in ops:", " out: list[str] = []", " w = threading.Thread(lambda: out.append(apply(threading.current_thread(), op)))", " w.start()", @@ -115,23 +197,60 @@ public ThreadStateMatrixTests(ITestOutputHelper output) : base(output) " box.append(w2)", " w2.start()", " w2.join()", - " print(f\"worker self {op} -> {out2[0]}\")" + " print(f\"worker self {op} -> {out2[0]}\")", + " for op in ops:", + " print(f\"main current_thread() {op} -> {apply(threading.current_thread(), op)}\")", + " print(f\"main main_thread().name after the rename -> {threading.main_thread().name}\")" }; private static readonly string[] PythonOutput = { + "main current_thread().name -> MainThread", + "main main_thread().name -> MainThread", + "main main_thread() is current_thread() -> True", + "main main_thread().ident == current_thread().ident -> True", + "main main_thread().is_alive() -> True", + "main main_thread().daemon -> False", + "main active_count() -> 1", + "main len(enumerate()) -> 1", + "main enumerate() names -> MainThread", + "worker current_thread().name -> Thread-1 ()", + "worker main_thread().name -> MainThread", + "worker main_thread() is current_thread() -> False", + "worker main_thread().ident == current_thread().ident -> False", + "worker main_thread().is_alive() -> True", + "worker main_thread().daemon -> False", + "worker active_count() -> 2", + "worker len(enumerate()) -> 2", + "worker enumerate() names -> MainThread, Thread-1 ()", + "main after join active_count() -> 1", + "main after join len(enumerate()) -> 1", + "Thread initial in enumerate() -> False, active_count() 1", + "Thread alive in enumerate() -> True, active_count() 2", + "Thread finished in enumerate() -> False, active_count() 1", + "Timer initial in enumerate() -> False, active_count() 1", + "Timer alive in enumerate() -> True, active_count() 2", + "Timer finished in enumerate() -> False, active_count() 1", + "Sub initial in enumerate() -> False, active_count() 1", + "Sub alive in enumerate() -> True, active_count() 2", + "Sub finished in enumerate() -> False, active_count() 1", + "Thread current thread -> current_thread() is it True, in enumerate() True, name Thread-6 (), active_count() 2", + "Timer current thread -> current_thread() is it True, in enumerate() True, name Thread-8, active_count() 2", + "Sub current thread -> current_thread() is it True, in enumerate() True, name Thread-9, active_count() 2", "Thread initial start() -> None", "Thread initial join() -> RuntimeError", "Thread initial join(0.01) -> RuntimeError", "Thread initial daemon -> False", "Thread initial daemon = True -> None", "Thread initial is_alive() -> False", + "Thread initial name -> Thread-16 ()", "Thread initial name = \"x\" -> x", "Thread alive start() -> RuntimeError", "Thread alive join(0.01) -> None", "Thread alive daemon -> False", "Thread alive daemon = True -> RuntimeError", "Thread alive is_alive() -> True", + "Thread alive name -> Thread-23 ()", "Thread alive name = \"x\" -> x", "Thread finished start() -> RuntimeError", "Thread finished join() -> None", @@ -139,6 +258,7 @@ public ThreadStateMatrixTests(ITestOutputHelper output) : base(output) "Thread finished daemon -> False", "Thread finished daemon = True -> RuntimeError", "Thread finished is_alive() -> False", + "Thread finished name -> Thread-31 ()", "Thread finished name = \"x\" -> x", "Timer initial start() -> None", "Timer initial join() -> RuntimeError", @@ -146,12 +266,14 @@ public ThreadStateMatrixTests(ITestOutputHelper output) : base(output) "Timer initial daemon -> False", "Timer initial daemon = True -> None", "Timer initial is_alive() -> False", + "Timer initial name -> Thread-39", "Timer initial name = \"x\" -> x", "Timer alive start() -> RuntimeError", "Timer alive join(0.01) -> None", "Timer alive daemon -> False", "Timer alive daemon = True -> RuntimeError", "Timer alive is_alive() -> True", + "Timer alive name -> Thread-46", "Timer alive name = \"x\" -> x", "Timer finished start() -> RuntimeError", "Timer finished join() -> None", @@ -159,6 +281,7 @@ public ThreadStateMatrixTests(ITestOutputHelper output) : base(output) "Timer finished daemon -> False", "Timer finished daemon = True -> RuntimeError", "Timer finished is_alive() -> False", + "Timer finished name -> Thread-54", "Timer finished name = \"x\" -> x", "Sub initial start() -> None", "Sub initial join() -> RuntimeError", @@ -166,12 +289,14 @@ public ThreadStateMatrixTests(ITestOutputHelper output) : base(output) "Sub initial daemon -> False", "Sub initial daemon = True -> None", "Sub initial is_alive() -> False", + "Sub initial name -> Thread-62", "Sub initial name = \"x\" -> x", "Sub alive start() -> RuntimeError", "Sub alive join(0.01) -> None", "Sub alive daemon -> False", "Sub alive daemon = True -> RuntimeError", "Sub alive is_alive() -> True", + "Sub alive name -> Thread-69", "Sub alive name = \"x\" -> x", "Sub finished start() -> RuntimeError", "Sub finished join() -> None", @@ -179,20 +304,15 @@ public ThreadStateMatrixTests(ITestOutputHelper output) : base(output) "Sub finished daemon -> False", "Sub finished daemon = True -> RuntimeError", "Sub finished is_alive() -> False", + "Sub finished name -> Thread-77", "Sub finished name = \"x\" -> x", - "main current_thread() start() -> RuntimeError", - "main current_thread() join() -> RuntimeError", - "main current_thread() join(0.01) -> RuntimeError", - "main current_thread() daemon -> False", - "main current_thread() daemon = True -> RuntimeError", - "main current_thread() is_alive() -> True", - "main current_thread() name = \"x\" -> x", "worker current_thread() start() -> RuntimeError", "worker current_thread() join() -> RuntimeError", "worker current_thread() join(0.01) -> RuntimeError", "worker current_thread() daemon -> False", "worker current_thread() daemon = True -> RuntimeError", "worker current_thread() is_alive() -> True", + "worker current_thread() name -> Thread-85 ()", "worker current_thread() name = \"x\" -> x", "worker self start() -> RuntimeError", "worker self join() -> RuntimeError", @@ -200,7 +320,17 @@ public ThreadStateMatrixTests(ITestOutputHelper output) : base(output) "worker self daemon -> False", "worker self daemon = True -> RuntimeError", "worker self is_alive() -> True", - "worker self name = \"x\" -> x" + "worker self name -> Thread-93 ()", + "worker self name = \"x\" -> x", + "main current_thread() start() -> RuntimeError", + "main current_thread() join() -> RuntimeError", + "main current_thread() join(0.01) -> RuntimeError", + "main current_thread() daemon -> False", + "main current_thread() daemon = True -> RuntimeError", + "main current_thread() is_alive() -> True", + "main current_thread() name -> MainThread", + "main current_thread() name = \"x\" -> x", + "main main_thread().name after the rename -> x" }; [Fact] @@ -212,6 +342,8 @@ public void EveryThreadOperation_InEveryState_GivesPythonsVerdict() Assert.True(result.Success, $"did not compile/run: {string.Join("; ", result.CompilationErrors)}\n{result.StandardError}"); - Assert.Equal(PythonOutput, result.StandardOutput.Replace("\r\n", "\n").TrimEnd('\n').Split('\n')); + // R-FD: the ` (target)` suffix of a target thread's default name is a cell on #2105. + var expected = PythonOutput.Select(line => line.Replace(" ()", "")).ToArray(); + Assert.Equal(expected, result.StandardOutput.Replace("\r\n", "\n").TrimEnd('\n').Split('\n')); } } diff --git a/src/Sharpy.Stdlib.Tests/Spy/SpyTestCollections.cs b/src/Sharpy.Stdlib.Tests/Spy/SpyTestCollections.cs index 2fe45879f..3506a5aab 100644 --- a/src/Sharpy.Stdlib.Tests/Spy/SpyTestCollections.cs +++ b/src/Sharpy.Stdlib.Tests/Spy/SpyTestCollections.cs @@ -53,6 +53,16 @@ public sealed class XmlRegisteredNamespacesCollection { } + /// + /// Tests that construct, start or count threading.Thread objects: the Thread-N counter and the + /// enumerate() / active_count() registry are process-wide (#2261), and the concurrency cells + /// assert exact numbers and counts. + /// + [CollectionDefinition("ThreadingRegistry")] + public sealed class ThreadingRegistryCollection + { + } + /// Tests that seed numpy.random and compare draws. [CollectionDefinition("NumpyRandomState")] public sealed class NumpyRandomStateCollection @@ -92,6 +102,14 @@ public partial class NumpyRandomTestsModuleTests } } +namespace Sharpy.Stdlib.Tests.Spy.Threading.ThreadingModuleTests +{ + [Collection("ThreadingRegistry")] + public partial class ThreadingModuleTestsModuleTests + { + } +} + namespace Sharpy.Stdlib.Tests.Spy.Os.OsModuleTests { // os.chdir (cwd) and os.putenv (environment). diff --git a/src/Sharpy.Stdlib/Threading/Thread.cs b/src/Sharpy.Stdlib/Threading/Thread.cs index 93352cdb2..b7f7b437a 100644 --- a/src/Sharpy.Stdlib/Threading/Thread.cs +++ b/src/Sharpy.Stdlib/Threading/Thread.cs @@ -25,40 +25,57 @@ public class Thread // join after that returns without reading its timeout (#2263). private volatile bool _stopped; - internal Thread(SysThread thread) + // python's Thread._name: never None (#2261, R-FD). Kept here, not in SysThread.Name, which + // the wrapper of a thread this module did not create must not rename. + private string _name; + + // The current_thread() / main_thread() wrapper of a running .NET thread (ThreadRegistry). + internal Thread(SysThread thread, string name, bool daemon) { _thread = thread; _started = 1; - _daemon = thread.IsBackground; + _daemon = daemon; + _name = name; } public Thread(Action? target = null, bool daemon = false, string? name = null) { + _name = ThreadRegistry.NameOrDefault(name); _thread = new SysThread(() => { - if (target != null) + try { - target(); + if (target != null) + { + target(); + } + else + { + Run(); + } } - else + finally { - Run(); + ThreadRegistry.Unregister(this); } }); _thread.IsBackground = daemon; _daemon = daemon; - if (name != null) - { - _thread.Name = name; - } + _thread.Name = _name; } - public string? Name + /// + /// The thread's name: the one passed to the constructor, else Thread-N (N counts the + /// unnamed threads created in the process, from 1); MainThread for the main thread. + /// + public string Name { - get => _thread.Name; - set => _thread.Name = value; + get => _name; + set => _name = value; } + internal SysThread NativeThread => _thread; + public bool Daemon { get => _daemon; @@ -91,7 +108,17 @@ public void Start() { throw new RuntimeError("threads can only be started once"); } - _thread.Start(); + // In enumerate() from start() on (python's _limbo), until the body's finally removes it. + ThreadRegistry.Register(this); + try + { + _thread.Start(); + } + catch + { + ThreadRegistry.Unregister(this); + throw; + } } public void Join(double? timeout = null) diff --git a/src/Sharpy.Stdlib/Threading/ThreadRegistry.cs b/src/Sharpy.Stdlib/Threading/ThreadRegistry.cs new file mode 100644 index 000000000..31e316bae --- /dev/null +++ b/src/Sharpy.Stdlib/Threading/ThreadRegistry.cs @@ -0,0 +1,131 @@ +using System.Globalization; +using SysInterlocked = System.Threading.Interlocked; +using SysThread = System.Threading.Thread; + +namespace Sharpy +{ + /// + /// The threading module's process-wide state, modelled on CPython 3.12's threading.py + /// (#2261 part 2): _counter numbers the default names Thread-N and + /// Dummy-N; _active (with _limbo) holds every thread that + /// enumerate() and active_count() report; _main_thread is the thread on + /// which the module was first used. + /// Like CPython's _main_thread = _MainThread(), which runs on the thread that imports + /// threading, is the thread that first reaches this class: every + /// constructor and every module function touches it, so in a Sharpy program + /// it is the thread main() runs on. The explicit static constructor makes that first + /// touch exact (no beforefieldinit). + /// Thread safety: _active is read and written only under , + /// as CPython reads and writes it only under _active_limbo_lock; _counter is + /// incremented atomically, as itertools.count.__next__ is one call under the GIL. Both + /// are rows of StdlibModuleGlobalRosterTests (Stdlib.Tests). + /// + internal static class ThreadRegistry + { + private static readonly object ActiveLock = new object(); + + // CPython's _active ∪ _limbo: the main thread, every started Thread that has not finished, + // and every dummy, in the order they were added (enumerate()'s order). A Thread is added by + // start() before its .NET thread starts and removed on its own thread as its body ends, so + // it is present while it runs and gone once join() returns. Guarded by ActiveLock. + private static readonly System.Collections.Generic.List _active = new System.Collections.Generic.List(); + + // CPython's _counter = itertools.count(1): Thread-N and Dummy-N share it; a named thread + // and the main thread do not consume a number. Incremented with Interlocked only. + private static int _counter; + + /// CPython's _main_thread: named MainThread, not a daemon, created started. + internal static readonly Thread Main = RegisterMain(); + + static ThreadRegistry() + { + } + + private static Thread RegisterMain() + { + var main = new Thread(SysThread.CurrentThread, "MainThread", daemon: false); + lock (ActiveLock) + { + _active.Add(main); + } + return main; + } + + /// + /// python's name or _newname("Thread-%d"): an empty or absent name takes the next + /// Thread-N. Always touches this class, so is captured first. + /// + internal static string NameOrDefault(string? name) + { + return string.IsNullOrEmpty(name) ? NewName("Thread-") : name!; + } + + private static string NewName(string prefix) + { + return prefix + SysInterlocked.Increment(ref _counter).ToString(CultureInfo.InvariantCulture); + } + + /// Adds a thread that is about to start (python's _limbo[self] = self). + internal static void Register(Thread thread) + { + lock (ActiveLock) + { + _active.Add(thread); + } + } + + /// Removes a thread whose body has ended (python's _delete()). + internal static void Unregister(Thread thread) + { + lock (ActiveLock) + { + _active.Remove(thread); + } + } + + /// + /// python's current_thread(): the object whose .NET thread is the + /// caller's — the same object start() was called on, or . A thread + /// the threading module did not start gets a Dummy-N daemon thread that stays in + /// enumerate(), as python's _DummyThread does; one whose ident a dead dummy + /// still holds replaces that dummy. + /// + internal static Thread Current() + { + var current = SysThread.CurrentThread; + lock (ActiveLock) + { + for (var i = 0; i < _active.Count; i++) + { + if (_active[i].NativeThread == current) + { + return _active[i]; + } + } + + var dummy = new Thread(current, NewName("Dummy-"), daemon: true); + _active.RemoveAll(t => t.Ident == dummy.Ident); + _active.Add(dummy); + return dummy; + } + } + + /// python's active_count(): len(_active) + len(_limbo). + internal static int Count() + { + lock (ActiveLock) + { + return _active.Count; + } + } + + /// python's enumerate(): a snapshot of every thread counts. + internal static Thread[] Snapshot() + { + lock (ActiveLock) + { + return _active.ToArray(); + } + } + } +} diff --git a/src/Sharpy.Stdlib/Threading/ThreadingModule.cs b/src/Sharpy.Stdlib/Threading/ThreadingModule.cs index 043628ae1..82407a80e 100644 --- a/src/Sharpy.Stdlib/Threading/ThreadingModule.cs +++ b/src/Sharpy.Stdlib/Threading/ThreadingModule.cs @@ -1,7 +1,5 @@ using System; using System.Collections.Generic; -using System.Linq; -using SysThread = System.Threading.Thread; namespace Sharpy { @@ -19,25 +17,44 @@ public static partial class ThreadingModule // stdlib-timeout-wait-saturates (R-FH, #2263). public const double TIMEOUT_MAX = 9223372036.0; + /// + /// Return the Thread object of the calling thread: the object start() was called + /// on, or the main thread. A thread the threading module did not start gets a daemon + /// Dummy-N thread, which stays in enumerate(). + /// public static Thread CurrentThread() { - return new Thread(SysThread.CurrentThread); + return ThreadRegistry.Current(); } + /// + /// Return the number of Thread objects currently alive: the length of enumerate(). + /// public static int ActiveCount() { - return System.Diagnostics.Process.GetCurrentProcess().Threads.Count; + return ThreadRegistry.Count(); } + /// + /// Return the main Thread object, named MainThread: the thread that first used the + /// threading module, which in a Sharpy program is the thread main() runs on. + /// public static Thread MainThread() { - return new Thread(SysThread.CurrentThread); + return ThreadRegistry.Main; } + /// + /// Return a list of every Thread object currently alive: the main thread, every started + /// thread that has not finished, and the dummy threads current_thread() created. + /// public static List Enumerate() { var result = new List(); - result.Add(CurrentThread()); + foreach (var thread in ThreadRegistry.Snapshot()) + { + result.Add(thread); + } return result; } From 43f85a943fb62264c635b25a0668c8aa79a80f4a Mon Sep 17 00:00:00 2001 From: antonsynd <2648730+antonsynd@users.noreply.github.com> Date: Thu, 22 Oct 2026 02:29:00 -0400 Subject: [PATCH 03/13] fix(random): the module function is random.random(), as CPython spells it, and the .NET name next_double is gone (refs #2321) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit #2321 cell 1 (R-HL a). Contract: a stdlib module exposes the Python API, never the .NET name. Change (rung 1, Sharpy source): `def next_double()` in spy/random_module.spy is renamed `def random()` and next_double deleted (one way to do it); regenerated Random/Random.cs (RandomModuleModule.NextDouble -> Random), docs/stdlib/random.md, and the spy tests random_tests.spy / random_additional2_tests.spy (calls and the test names test_next_double_* -> test_random_*) with their generated C#; StdlibModuleGlobalConcurrencyTests' random exerciser calls Random(). No new global; StdlibModuleGlobalRosterTests unchanged. Hypothesis measured: the `Random as SysRandom` alias does not block the Python name — the emitter spells global::System.Random, so `def random()` compiles and the regen script, the spy-test regen and both staleness checks pass. Matrix: spelling {module attribute `random.random()`, from-import `from random import random as draw`} × {in [0, 1), seed-determinism} plus the deleted name refused — two new Stdlib.Tests fixtures (python3 3.12.13: True, True, True; AttributeError: module 'random' has no attribute 'next_double'): random_random_python_name_2321 (.expected) and random_next_double_refused_2321 (.error "Module 'random' has no member 'next_double'"). Mutation (rename back: every changed file reverted to the parent's bytes, the new fixtures kept; build Stdlib.Tests; FileBasedIntegrationTests 2321 filter): 2 red (random_next_double_refused_2321, random_random_python_name_2321) / 0 green; restored (cp + touch, rebuild): 2 green / 0 red. Direction record (`sharpyc run`; base = the e7e0a0ef7 binary; random is unchanged at the parent 7c8dd7f4c): `import random; print(0 <= random.random() < 1)`: SPY0203 "Module 'random' has no member 'random'. Did you mean '_random'?" -> True [python True]; `print(0 <= random.next_double() < 1)`: True -> SPY0203 "Module 'random' has no member 'next_double'" [python AttributeError]. Expected values changed: none (the renamed spy tests keep their assertions). Not done (R-HL): cells 2-5 of #2321 and the other random.__all__ names. Filtered @ this commit's tree (measured): Sharpy.Stdlib.Tests 4293/0/0; Compiler.Tests Stdlib|Roster|Census|Concurrency|ProcessGlobalState 373/0/0; check_spy_staleness, check_spy_tests_staleness, stdlib docs regen, format whitespace --verify-no-changes and allowlist issue-state clean. Co-Authored-By: Claude Opus 5.5 (1M context) Claude-Session: https://claude.ai/code/session_0126pNZqmMsSbn4WFFPhpnxW --- docs/stdlib/random.md | 2 +- .../StdlibModuleGlobalConcurrencyTests.cs | 10 +++++----- .../random_next_double_refused_2321.error | 1 + .../random_next_double_refused_2321.spy | 8 ++++++++ .../random_random_python_name_2321.expected | 3 +++ .../random_random_python_name_2321.spy | 14 +++++++++++++ .../Spy/generated/random_additional2_tests.cs | 20 +++++++++---------- .../Spy/generated/random_tests.cs | 14 ++++++------- .../Spy/random/random_additional2_tests.spy | 14 ++++++------- .../Spy/random/random_tests.spy | 10 +++++----- src/Sharpy.Stdlib/Random/Random.cs | 2 +- src/Sharpy.Stdlib/spy/random_module.spy | 2 +- 12 files changed, 63 insertions(+), 37 deletions(-) create mode 100644 src/Sharpy.Stdlib.Tests/Integration/TestFixtures/random_next_double_refused_2321.error create mode 100644 src/Sharpy.Stdlib.Tests/Integration/TestFixtures/random_next_double_refused_2321.spy create mode 100644 src/Sharpy.Stdlib.Tests/Integration/TestFixtures/random_random_python_name_2321.expected create mode 100644 src/Sharpy.Stdlib.Tests/Integration/TestFixtures/random_random_python_name_2321.spy diff --git a/docs/stdlib/random.md b/docs/stdlib/random.md index 71c03ee65..dad736bce 100644 --- a/docs/stdlib/random.md +++ b/docs/stdlib/random.md @@ -12,7 +12,7 @@ import random Initialize the random number generator with the given seed. -### `random.next_double() -> float` +### `random.random() -> float` Return the next random floating point number in the range [0.0, 1.0). diff --git a/src/Sharpy.Stdlib.Tests/Conformance/StdlibModuleGlobalConcurrencyTests.cs b/src/Sharpy.Stdlib.Tests/Conformance/StdlibModuleGlobalConcurrencyTests.cs index befa35283..d716abe1d 100644 --- a/src/Sharpy.Stdlib.Tests/Conformance/StdlibModuleGlobalConcurrencyTests.cs +++ b/src/Sharpy.Stdlib.Tests/Conformance/StdlibModuleGlobalConcurrencyTests.cs @@ -502,7 +502,7 @@ public virtual void Restore() /// then random() on another == the re-seeded draw (True, 0.32383276483316237); 8 threads x 2000 /// draws after seed(7) are, as a multiset, exactly the first 16000 draws of the seed-7 stream /// (True), with 0 zeros; seed(i) on one thread then random() on another == the seed-i stream's - /// first value for every sampled i in [0, 2000) (True). Sharpy spells random() next_double(). + /// first value for every sampled i in [0, 2000) (True). /// private sealed class RandomExerciser : GlobalExerciser { @@ -512,7 +512,7 @@ private sealed class RandomExerciser : GlobalExerciser private static double[] Stream(int n, int seed = 7) { Sharpy.RandomModule.RandomModuleModule.Seed(seed); - return Enumerable.Range(0, n).Select(_ => Sharpy.RandomModule.RandomModuleModule.NextDouble()).ToArray(); + return Enumerable.Range(0, n).Select(_ => Sharpy.RandomModule.RandomModuleModule.Random()).ToArray(); } public override void Write(int threads, Tally tally) @@ -523,7 +523,7 @@ public override void Write(int threads, Tally tally) for (var i = 0; i < Iterations; i++) tally.Do(() => Sharpy.RandomModule.RandomModuleModule.Seed(7)); }); - var draw = OnFreshThread(Sharpy.RandomModule.RandomModuleModule.NextDouble); + var draw = OnFreshThread(Sharpy.RandomModule.RandomModuleModule.Random); tally.Check(() => draw == first, () => $"after every thread seeded 7 the next draw was {draw}, expected {first}"); } @@ -536,7 +536,7 @@ public override void Read(int threads, Tally tally) { draws[id] = new double[Iterations]; for (var i = 0; i < Iterations; i++) - draws[id][i] = Sharpy.RandomModule.RandomModuleModule.NextDouble(); + draws[id][i] = Sharpy.RandomModule.RandomModuleModule.Random(); }); var all = draws.Where(d => d != null).SelectMany(d => d).ToArray(); var outOfDomain = all.Count(d => !(d >= 0.0 && d < 1.0)); @@ -552,7 +552,7 @@ public override void CrossThread(int threads, Tally tally) var firsts = Enumerable.Range(0, Iterations).Select(seed => Stream(1, seed)[0]).ToArray(); Handoff(threads, tally, i => Sharpy.RandomModule.RandomModuleModule.Seed(i), - i => tally.Check(() => Sharpy.RandomModule.RandomModuleModule.NextDouble() == firsts[i], + i => tally.Check(() => Sharpy.RandomModule.RandomModuleModule.Random() == firsts[i], () => $"iteration {i}: the draw after another thread's seed({i}) is not the seed-{i} stream's first value")); } diff --git a/src/Sharpy.Stdlib.Tests/Integration/TestFixtures/random_next_double_refused_2321.error b/src/Sharpy.Stdlib.Tests/Integration/TestFixtures/random_next_double_refused_2321.error new file mode 100644 index 000000000..27197e5a2 --- /dev/null +++ b/src/Sharpy.Stdlib.Tests/Integration/TestFixtures/random_next_double_refused_2321.error @@ -0,0 +1 @@ +Module 'random' has no member 'next_double' diff --git a/src/Sharpy.Stdlib.Tests/Integration/TestFixtures/random_next_double_refused_2321.spy b/src/Sharpy.Stdlib.Tests/Integration/TestFixtures/random_next_double_refused_2321.spy new file mode 100644 index 000000000..3343cefec --- /dev/null +++ b/src/Sharpy.Stdlib.Tests/Integration/TestFixtures/random_next_double_refused_2321.spy @@ -0,0 +1,8 @@ +# #2321 cell 1 (R-HL): next_double was the .NET name of random.random(); it is deleted (one way to +# do it), so it is refused like any other missing module member. python3 3.12.13: +# AttributeError: module 'random' has no attribute 'next_double'. +import random + + +def main() -> None: + print(random.next_double()) diff --git a/src/Sharpy.Stdlib.Tests/Integration/TestFixtures/random_random_python_name_2321.expected b/src/Sharpy.Stdlib.Tests/Integration/TestFixtures/random_random_python_name_2321.expected new file mode 100644 index 000000000..b8ca7e7ef --- /dev/null +++ b/src/Sharpy.Stdlib.Tests/Integration/TestFixtures/random_random_python_name_2321.expected @@ -0,0 +1,3 @@ +True +True +True diff --git a/src/Sharpy.Stdlib.Tests/Integration/TestFixtures/random_random_python_name_2321.spy b/src/Sharpy.Stdlib.Tests/Integration/TestFixtures/random_random_python_name_2321.spy new file mode 100644 index 000000000..a26af3832 --- /dev/null +++ b/src/Sharpy.Stdlib.Tests/Integration/TestFixtures/random_random_python_name_2321.spy @@ -0,0 +1,14 @@ +# #2321 cell 1 (R-HL): the module function is random.random(), as CPython spells it, by module +# attribute and by from-import. python3 3.12.13 prints True three times; before the rename both +# spellings were SPY0203 (the function was next_double). +import random +from random import random as draw + + +def main() -> None: + print(0 <= random.random() < 1) + print(0 <= draw() < 1) + random.seed(3) + a = random.random() + random.seed(3) + print(a == random.random()) diff --git a/src/Sharpy.Stdlib.Tests/Spy/generated/random_additional2_tests.cs b/src/Sharpy.Stdlib.Tests/Spy/generated/random_additional2_tests.cs index 7f6c3453a..5aa7886a2 100644 --- a/src/Sharpy.Stdlib.Tests/Spy/generated/random_additional2_tests.cs +++ b/src/Sharpy.Stdlib.Tests/Spy/generated/random_additional2_tests.cs @@ -75,12 +75,12 @@ public void TestSeedDifferentSeedsProduceDifferentSequences() { #line (36, 5) - (36, 19) 12 "src/Sharpy.Stdlib.Tests/Spy/random/random_additional2_tests.spy" random.Seed(1); -#line (37, 5) - (37, 37) 12 "src/Sharpy.Stdlib.Tests/Spy/random/random_additional2_tests.spy" - double a = random.NextDouble(); +#line (37, 5) - (37, 32) 12 "src/Sharpy.Stdlib.Tests/Spy/random/random_additional2_tests.spy" + double a = random.Random(); #line (38, 5) - (38, 19) 12 "src/Sharpy.Stdlib.Tests/Spy/random/random_additional2_tests.spy" random.Seed(2); -#line (39, 5) - (39, 37) 12 "src/Sharpy.Stdlib.Tests/Spy/random/random_additional2_tests.spy" - double b = random.NextDouble(); +#line (39, 5) - (39, 32) 12 "src/Sharpy.Stdlib.Tests/Spy/random/random_additional2_tests.spy" + double b = random.Random(); #line (41, 5) - (41, 19) 12 "src/Sharpy.Stdlib.Tests/Spy/random/random_additional2_tests.spy" Xunit.Assert.NotEqual(b, a); #line hidden @@ -293,7 +293,7 @@ public void TestUniformBLessThanAReturnsInReversedRange() } [Xunit.FactAttribute] - public void TestNextDoubleNeverReturns1() + public void TestRandomNeverReturns1() { #line (138, 5) - (138, 20) 12 "src/Sharpy.Stdlib.Tests/Spy/random/random_additional2_tests.spy" random.Seed(42); @@ -303,8 +303,8 @@ public void TestNextDoubleNeverReturns1() while (i < 1000) #line hidden { -#line (141, 9) - (141, 43) 16 "src/Sharpy.Stdlib.Tests/Spy/random/random_additional2_tests.spy" - Xunit.Assert.True(random.NextDouble() < 1.0d); +#line (141, 9) - (141, 38) 16 "src/Sharpy.Stdlib.Tests/Spy/random/random_additional2_tests.spy" + Xunit.Assert.True(random.Random() < 1.0d); #line (142, 9) - (142, 18) 16 "src/Sharpy.Stdlib.Tests/Spy/random/random_additional2_tests.spy" i = i + 1; #line hidden @@ -312,7 +312,7 @@ public void TestNextDoubleNeverReturns1() } [Xunit.FactAttribute] - public void TestNextDoubleStatisticalMeanApproximatesHalf() + public void TestRandomStatisticalMeanApproximatesHalf() { #line (146, 5) - (146, 20) 12 "src/Sharpy.Stdlib.Tests/Spy/random/random_additional2_tests.spy" random.Seed(42); @@ -326,8 +326,8 @@ public void TestNextDoubleStatisticalMeanApproximatesHalf() while (i < n) #line hidden { -#line (151, 9) - (151, 45) 16 "src/Sharpy.Stdlib.Tests/Spy/random/random_additional2_tests.spy" - total = total + random.NextDouble(); +#line (151, 9) - (151, 40) 16 "src/Sharpy.Stdlib.Tests/Spy/random/random_additional2_tests.spy" + total = total + random.Random(); #line (152, 9) - (152, 18) 16 "src/Sharpy.Stdlib.Tests/Spy/random/random_additional2_tests.spy" i = i + 1; #line hidden diff --git a/src/Sharpy.Stdlib.Tests/Spy/generated/random_tests.cs b/src/Sharpy.Stdlib.Tests/Spy/generated/random_tests.cs index ca7ef2dd5..8580b6a7e 100644 --- a/src/Sharpy.Stdlib.Tests/Spy/generated/random_tests.cs +++ b/src/Sharpy.Stdlib.Tests/Spy/generated/random_tests.cs @@ -27,19 +27,19 @@ public void TestSeedProducesDeterministicResults() { #line (30, 5) - (30, 20) 12 "src/Sharpy.Stdlib.Tests/Spy/random/random_tests.spy" random.Seed(42); -#line (31, 5) - (31, 41) 12 "src/Sharpy.Stdlib.Tests/Spy/random/random_tests.spy" - double first = random.NextDouble(); +#line (31, 5) - (31, 36) 12 "src/Sharpy.Stdlib.Tests/Spy/random/random_tests.spy" + double first = random.Random(); #line (32, 5) - (32, 20) 12 "src/Sharpy.Stdlib.Tests/Spy/random/random_tests.spy" random.Seed(42); -#line (33, 5) - (33, 42) 12 "src/Sharpy.Stdlib.Tests/Spy/random/random_tests.spy" - double second = random.NextDouble(); +#line (33, 5) - (33, 37) 12 "src/Sharpy.Stdlib.Tests/Spy/random/random_tests.spy" + double second = random.Random(); #line (34, 5) - (34, 28) 12 "src/Sharpy.Stdlib.Tests/Spy/random/random_tests.spy" Xunit.Assert.Equal(second, first); #line hidden } [Xunit.FactAttribute] - public void TestNextDoubleReturnsValueInRange() + public void TestRandomReturnsValueInRange() { #line (40, 5) - (40, 21) 12 "src/Sharpy.Stdlib.Tests/Spy/random/random_tests.spy" random.Seed(123); @@ -49,8 +49,8 @@ public void TestNextDoubleReturnsValueInRange() while (i < 50) #line hidden { -#line (43, 9) - (43, 45) 16 "src/Sharpy.Stdlib.Tests/Spy/random/random_tests.spy" - double value = random.NextDouble(); +#line (43, 9) - (43, 40) 16 "src/Sharpy.Stdlib.Tests/Spy/random/random_tests.spy" + double value = random.Random(); #line (44, 9) - (44, 29) 16 "src/Sharpy.Stdlib.Tests/Spy/random/random_tests.spy" Xunit.Assert.True(value >= 0.0d); #line (45, 9) - (45, 28) 16 "src/Sharpy.Stdlib.Tests/Spy/random/random_tests.spy" diff --git a/src/Sharpy.Stdlib.Tests/Spy/random/random_additional2_tests.spy b/src/Sharpy.Stdlib.Tests/Spy/random/random_additional2_tests.spy index a3563d1ec..74bc5f586 100644 --- a/src/Sharpy.Stdlib.Tests/Spy/random/random_additional2_tests.spy +++ b/src/Sharpy.Stdlib.Tests/Spy/random/random_additional2_tests.spy @@ -34,9 +34,9 @@ def test_seed_produces_identical_sequence_multiple_values(): @test def test_seed_different_seeds_produce_different_sequences(): random.seed(1) - a: float = random.next_double() + a: float = random.random() random.seed(2) - b: float = random.next_double() + b: float = random.random() # Extremely unlikely to be equal with different seeds assert a != b @@ -131,24 +131,24 @@ def test_uniform_b_less_than_a_returns_in_reversed_range(): assert val <= 10.0 i = i + 1 -# --- next_double additional --- +# --- random() additional --- @test -def test_next_double_never_returns_1(): +def test_random_never_returns_1(): random.seed(42) i: int = 0 while i < 1000: - assert random.next_double() < 1.0 + assert random.random() < 1.0 i = i + 1 @test -def test_next_double_statistical_mean_approximates_half(): +def test_random_statistical_mean_approximates_half(): random.seed(42) total: float = 0.0 n: int = 1000 i: int = 0 while i < n: - total = total + random.next_double() + total = total + random.random() i = i + 1 assert abs(total / n - 0.5) < 0.05 diff --git a/src/Sharpy.Stdlib.Tests/Spy/random/random_tests.spy b/src/Sharpy.Stdlib.Tests/Spy/random/random_tests.spy index 971d6fa8d..baedd162b 100644 --- a/src/Sharpy.Stdlib.Tests/Spy/random/random_tests.spy +++ b/src/Sharpy.Stdlib.Tests/Spy/random/random_tests.spy @@ -28,19 +28,19 @@ import random @test def test_seed_produces_deterministic_results(): random.seed(42) - first: float = random.next_double() + first: float = random.random() random.seed(42) - second: float = random.next_double() + second: float = random.random() assert first == second -# --- next_double range --- +# --- random() range --- @test -def test_next_double_returns_value_in_range(): +def test_random_returns_value_in_range(): random.seed(123) i: int = 0 while i < 50: - value: float = random.next_double() + value: float = random.random() assert value >= 0.0 assert value < 1.0 i = i + 1 diff --git a/src/Sharpy.Stdlib/Random/Random.cs b/src/Sharpy.Stdlib/Random/Random.cs index 8dc336cf7..527acbad6 100644 --- a/src/Sharpy.Stdlib/Random/Random.cs +++ b/src/Sharpy.Stdlib/Random/Random.cs @@ -39,7 +39,7 @@ public static void Seed(int s) /// /// Return the next random floating point number in the range [0.0, 1.0). /// - public static double NextDouble() + public static double Random() { { var __ctx_1 = global::Sharpy.RandomModule.RandomModuleModule._Lock; diff --git a/src/Sharpy.Stdlib/spy/random_module.spy b/src/Sharpy.Stdlib/spy/random_module.spy index 363c9a385..14cc69ff0 100644 --- a/src/Sharpy.Stdlib/spy/random_module.spy +++ b/src/Sharpy.Stdlib/spy/random_module.spy @@ -16,7 +16,7 @@ def seed(s: int) -> None: with _lock: _random = SysRandom(s) -def next_double() -> float: +def random() -> float: """Return the next random floating point number in the range [0.0, 1.0).""" with _lock: return _random.next_double() From 817c60e240fddf3a846d18e3518db1acd8dd1aed Mon Sep 17 00:00:00 2001 From: antonsynd <2648730+antonsynd@users.noreply.github.com> Date: Thu, 22 Oct 2026 02:06:36 -0400 Subject: [PATCH 04/13] =?UTF-8?q?fix(parser):=20--max-errors=20N=20reports?= =?UTF-8?q?=20at=20most=20N=20errors=20=E2=80=94=20the=20cap=20is=20enforc?= =?UTF-8?q?ed=20at=20the=20one=20error-counting=20site,=20so=20no=20recove?= =?UTF-8?q?ry=20follows=20the=20Nth=20error=20(refs=20#2311)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Change (R-HD a). The parser read its cap at five parallel panic-mode catch sites (ParseModule, ParseStatements, ParseBlock, the enum body, the union body), each AFTER the statement parse that had already reported a cascade: a suite left at the cap reported its definition's `Expected Dedent` (SPY0104) as error N+1. Every parser error now goes through Parser.RecordError (ReportError and the two direct AddError sites, `raise ... from` and ParseSingleExpression's trailing token). When the error just recorded is the Nth it records the SPY0905 note and throws ParserErrorLimitException, which is not a ParserAbortException, so no recovery site catches it; the four public entry points (ParseModule, ParseStatements, ParseSingleStatement, ParseSingleExpression) stop there. The five catch-site copies are deleted. ParseStatements (the REPL) now records the SPY0905 note at the cap too; it used to stop silently. Matrix (MaxErrorsCapMatrixTests, Sharpy.Cli.Tests, 108 rows + count): phase {parser, semantic (control)} x N {1, 2, default: 25 parser / 100 semantic} x shape {statement, def body, class body, nested suite; parser also enum body, union body} x route {emit parse, emit diagnostics, run, build}; semantic x emit parse does not exist (no semantic phase). Each program has N+2 errors; assert exactly N errors and, on emit diagnostics, exactly one SPY0905. LSP route (MaxErrorsCapLspRouteTests, Sharpy.Lsp.Tests, 16 rows + count): SharpyWorkspace.GetAnalysisAsync and GetParseResultAsync, N = default only — the LSP has no cap option (ForLsp is reached with maxErrors 0), recorded rather than invented. Mutation (Rule 12), measured: - M1 drop the seam (RecordError never stops): Cli 72/109 red (every parser row), LSP 12/17 red (every parser row) - M2 the seam throws a recoverable ParserAbortException, so the catch sites recover past the cap: Cli 48/85 red, LSP 8/13 red (measured on the 4-shape matrix, before the enum/union shapes joined) - M3 the base parser (the five catch-site copies restored, no seam): Cli 52/109 red = def/class/nested x {1, 2, default} x 4 routes (36) + enum/union x {1, default} x 4 routes (16); LSP 10/17 red = the five non-statement parser shapes x 2 routes; statement and semantic rows green (controls). Restoring one catch-site copy WITH the seam in place is dead code (a ParserAbortException is only ever caught below the cap), so the base parser is the catch-site-copy mutation. - restored: Cli 109/0/0, LSP 17/0/0 Direction record (built sharpyc, base e7e0a0ef7 vs this tree; errors counted, every route alike): - def/class/nested x N=1: base 2 (SPY0100 + SPY0104) -> 1 - def/class/nested x N=2: base 3 -> 2 - def/class/nested x default, 27 items: base 26 -> 25 - enum/union x N=1: base 2 -> 1; x default: base 26 -> 25; x N=2: 2 -> 2 (base already at the cap) - statement rows and every semantic row: unchanged (controls) Filtered runs @ e7e0a0ef7 + this change (measured): - Compiler.Tests Parser|Conformance|Census|Roster|FileBasedIntegrationTests|Repl: 27231/0/2 - Lsp.Tests FrontEndParity|MaxErrorsCapLspRouteTests (4-shape): 14/0/0 - Cli.Tests MaxErrors|GlobalOption|CliOptionEffect (4-shape): 378/0/0 - Cli.Tests MaxErrorsCapMatrixTests (6-shape): 109/0/0; Lsp.Tests MaxErrorsCapLspRouteTests (6-shape): 17/0/0 No expected value changed. Out of scope, reported to the lead: an undefined type (SPY0202, type resolution) does not honour the semantic cap — three unions/defs with undefined types under --max-errors 1 report 3 errors on emit diagnostics and run, base and here alike; the semantic matrix therefore has no enum/union rows. Co-Authored-By: Claude Opus 5.5 (1M context) Claude-Session: https://claude.ai/code/session_0126pNZqmMsSbn4WFFPhpnxW --- .../Commands/MaxErrorsCapMatrixTests.cs | 131 ++++++++++++++++++ .../Parser/Parser.Definitions.cs | 10 +- .../Parser/Parser.Expressions.cs | 7 +- .../Parser/Parser.Statements.cs | 13 +- src/Sharpy.Compiler/Parser/Parser.cs | 127 ++++++++++------- .../MaxErrorsCapLspRouteTests.cs | 110 +++++++++++++++ 6 files changed, 331 insertions(+), 67 deletions(-) create mode 100644 src/Sharpy.Cli.Tests/Commands/MaxErrorsCapMatrixTests.cs create mode 100644 src/Sharpy.Lsp.Tests/MaxErrorsCapLspRouteTests.cs diff --git a/src/Sharpy.Cli.Tests/Commands/MaxErrorsCapMatrixTests.cs b/src/Sharpy.Cli.Tests/Commands/MaxErrorsCapMatrixTests.cs new file mode 100644 index 000000000..ca2545ee5 --- /dev/null +++ b/src/Sharpy.Cli.Tests/Commands/MaxErrorsCapMatrixTests.cs @@ -0,0 +1,131 @@ +using System.Text; +using System.Text.RegularExpressions; +using FluentAssertions; +using Xunit; + +namespace Sharpy.Cli.Tests.Commands; + +/// +/// --max-errors N reports at most N errors, plus the SPY0905 "further errors suppressed" +/// note where that phase emits one (#2311, ruling R-HD a). The parser used to read its cap at five +/// parallel panic-mode catch sites, each AFTER the recovery that had already reported a cascade: +/// a suite left at the cap reported its definition's Expected Dedent (SPY0104) as error +/// N+1. The cap is now enforced at the parser's one error-counting site, so no recovery follows +/// the Nth error. +/// +/// +/// +/// Closed-axis matrix: phase {parser, semantic (control)} × N {1, 2, default} × shape +/// {statement, def body, class body, nested suite} × route {emit parse, +/// emit diagnostics, run, build}. The parser phase has two more shapes, +/// enum body and union body, so every one of the five former catch sites (module, suite, enum, +/// union) is a row. Every program has N + 2 errors of its phase, so each row is one the cap must +/// cut. "default" is each phase's own default — 25 for the parser, 100 for semantic analysis. +/// emit diagnostics is the one route that prints warnings next to errors, so it alone also +/// pins the SPY0905 note. The semantic × emit parse cells do not exist (that route runs no +/// semantic phase) and are not generated. The LSP route has no cap option +/// (CompilerOptionsFactory.ForLsp is reached with maxErrors 0 only), so its +/// default-N rows live in Sharpy.Lsp.Tests/MaxErrorsCapLspRouteTests. +/// +/// +/// The parser rows of every shape but the statement are the ones the defect reached (N + 1 at +/// e7e0a0ef7); the statement rows, and every semantic row, are controls that already honoured the +/// cap. The semantic phase has no enum/union rows: an undefined type in a union case is SPY0202, +/// which does not honour the semantic cap at e7e0a0ef7 — a separate cell, outside this parser +/// chore (reported with #2311's round). +/// +/// +public class MaxErrorsCapMatrixTests +{ + private const int ParserDefault = 25; + private const int SemanticDefault = 100; + + private static readonly string[] Phases = { "parser", "semantic" }; + private static readonly string[] SharedShapes = { "statement", "def", "class", "nested" }; + private static readonly string[] ParserOnlyShapes = { "enum", "union" }; + private static readonly int?[] Caps = { 1, 2, null }; + private static readonly string[] Routes = { "emit parse", "emit diagnostics", "run", "build" }; + + public static IEnumerable Cells() + { + foreach (var phase in Phases) + foreach (var shape in phase == "parser" ? SharedShapes.Concat(ParserOnlyShapes) : SharedShapes) + foreach (var cap in Caps) + foreach (var route in Routes) + { + if (phase == "semantic" && route == "emit parse") + continue; // `emit parse` runs no semantic phase. + yield return new object[] { phase, shape, cap?.ToString() ?? "default", route }; + } + } + + [Fact] + public void Matrix_HasEveryCell() + { + // Anchored to literals, not to the axis arrays: parser 6 shapes × 3 N × 4 routes = 72, plus + // semantic 4 shapes × 3 N × 3 routes (no `emit parse`) = 36. + Cells().Should().HaveCount(108); + Cells().Select(c => $"{c[0]}|{c[1]}|{c[2]}|{c[3]}").Should().OnlyHaveUniqueItems(); + } + + [Theory] + [MemberData(nameof(Cells))] + public void MaxErrors_ReportsAtMostNErrors(string phase, string shape, string cap, string route) + { + int? n = cap == "default" ? null : int.Parse(cap); + var effective = n ?? (phase == "parser" ? ParserDefault : SemanticDefault); + + using var ws = new TempWorkspace(); + var spy = ws.WriteSpy(Program(phase, shape, effective + 2)); + var commandLine = new StringBuilder($"{route} \"{spy}\""); + if (n is { } explicitCap) + commandLine.Append($" --max-errors {explicitCap}"); + if (route == "build") + commandLine.Append($" --output \"{ws.PathFor("out.dll")}\""); + + var invocation = CliTestHarness.Invoke(commandLine.ToString()); + var output = invocation.StdOut + invocation.StdErr; + + invocation.ExitCode.Should().NotBe(0, "the program has errors"); + var errorLine = route == "emit diagnostics" + ? new Regex(@"^error SPY\d{4}\b", RegexOptions.Multiline) + : new Regex(@"^error\[SPY\d{4}\]", RegexOptions.Multiline); + var errors = errorLine.Matches(output).Count; + errors.Should().Be(effective, + $"`{commandLine}` must report exactly the cap's {effective} errors of the {effective + 2}+ " + + $"the program has — never a cascade past the cap:\n{output}"); + + if (route == "emit diagnostics") + { + Regex.Matches(output, @"^warning SPY0905\b", RegexOptions.Multiline).Count.Should().Be(1, + $"the phase that stopped at the cap notes it exactly once:\n{output}"); + } + } + + /// + /// A program with at least errors of , one in + /// each of items of , plus a main so no + /// entry-point error joins them. (Uncapped, a union whose only case fails to parse also reports + /// that it has no case.) + /// + private static string Program(string phase, string shape, int count) + { + var sb = new StringBuilder(); + for (var i = 1; i <= count; i++) + { + var bad = phase == "parser" ? $"x{i} = = {i}" : $"x{i}: int = undefined_{i}"; + sb.Append(shape switch + { + "statement" => $"{bad}\n", + "def" => $"def f{i}() -> None:\n {bad}\n\n", + "class" => $"class C{i}:\n {bad}\n\n", + "nested" => $"def f{i}() -> None:\n if True:\n {bad}\n\n", + "enum" => $"enum E{i}:\n A = = {i}\n\n", + "union" => $"union U{i}:\n case A(x: = int)\n\n", + _ => throw new ArgumentOutOfRangeException(nameof(shape), shape, null), + }); + } + sb.Append("\ndef main() -> None:\n pass\n"); + return sb.ToString(); + } +} diff --git a/src/Sharpy.Compiler/Parser/Parser.Definitions.cs b/src/Sharpy.Compiler/Parser/Parser.Definitions.cs index 95f8b2208..4bcc61bce 100644 --- a/src/Sharpy.Compiler/Parser/Parser.Definitions.cs +++ b/src/Sharpy.Compiler/Parser/Parser.Definitions.cs @@ -1149,9 +1149,8 @@ private EnumDef ParseEnumDef() } catch (ParserAbortException) { - // Error already recorded. Skip to the next line within the enum body. - if (_diagnostics.ErrorCount >= _maxErrors) - break; + // Error already recorded (the cap is enforced at RecordError, #2311). + // Skip to the next line within the enum body. Synchronize(itemStart); } SkipNewlines(); @@ -1326,9 +1325,8 @@ private UnionDef ParseUnionDef() } catch (ParserAbortException) { - // Error already recorded. Skip to the next line within the union body. - if (_diagnostics.ErrorCount >= _maxErrors) - break; + // Error already recorded (the cap is enforced at RecordError, #2311). + // Skip to the next line within the union body. Synchronize(itemStart); } SkipNewlines(); diff --git a/src/Sharpy.Compiler/Parser/Parser.Expressions.cs b/src/Sharpy.Compiler/Parser/Parser.Expressions.cs index 3d3ade944..84534ed6a 100644 --- a/src/Sharpy.Compiler/Parser/Parser.Expressions.cs +++ b/src/Sharpy.Compiler/Parser/Parser.Expressions.cs @@ -55,15 +55,14 @@ public Expression ParseSingleExpression() SkipNewlines(); if (!IsAtEnd) { - _diagnostics.AddError( + RecordError( $"Unexpected token '{Current.Value}' after expression", CurrentSpan, Current.Line, Current.Column, - code: DiagnosticCodes.Parser.UnexpectedToken, - phase: CompilerPhase.Parser); + DiagnosticCodes.Parser.UnexpectedToken); } return expr; } - catch (ParserAbortException) + catch (Exception e) when (e is ParserAbortException or ParserErrorLimitException) { return new Identifier { diff --git a/src/Sharpy.Compiler/Parser/Parser.Statements.cs b/src/Sharpy.Compiler/Parser/Parser.Statements.cs index 8708065cf..158877e92 100644 --- a/src/Sharpy.Compiler/Parser/Parser.Statements.cs +++ b/src/Sharpy.Compiler/Parser/Parser.Statements.cs @@ -998,13 +998,12 @@ private RaiseStatement ParseRaiseStatement() var fromToken = Current; Advance(); cause = ParseExpression(); // Parse for error recovery - _diagnostics.AddError( + RecordError( "'raise ... from ...' is not supported in Sharpy. Use 'raise' without a cause.", GetSpanFromToken(fromToken), fromToken.Line, fromToken.Column, - code: DiagnosticCodes.Parser.RaiseFromNotSupported, - phase: CompilerPhase.Parser); + DiagnosticCodes.Parser.RaiseFromNotSupported); } } @@ -1371,12 +1370,8 @@ private List ParseBlock() } catch (ParserAbortException) { - // Error already recorded in _diagnostics by ReportError() - - // Stop after _maxErrors to avoid cascading false errors - if (_diagnostics.ErrorCount >= _maxErrors) - break; - + // Error already recorded in _diagnostics by ReportError(); the error cap is + // enforced there (RecordError), never at a recovery site (#2311). // Panic-mode recovery: synchronize to next statement boundary Synchronize(itemStart); } diff --git a/src/Sharpy.Compiler/Parser/Parser.cs b/src/Sharpy.Compiler/Parser/Parser.cs index 6e67990d0..f1e6a785e 100644 --- a/src/Sharpy.Compiler/Parser/Parser.cs +++ b/src/Sharpy.Compiler/Parser/Parser.cs @@ -21,6 +21,15 @@ public partial class Parser /// private class ParserAbortException : Exception { } + /// + /// Thrown by the moment the error cap is reached. It is deliberately + /// NOT a : no panic-mode recovery site catches it, so it + /// unwinds the whole parse to the public entry point, and nothing a recovery would report — + /// the enclosing suite's Expected Dedent, the next statement's error — is reported after + /// the Nth error (#2311). + /// + private sealed class ParserErrorLimitException : Exception { } + /// /// Record a parser error into diagnostics and return an exception for stack unwinding. /// Usage: throw ReportError("msg", line, col, code); @@ -29,10 +38,32 @@ private class ParserAbortException : Exception { } private ParserAbortException ReportError(string message, int line, int column, string code, Text.TextSpan? span = null) { - _diagnostics.AddError(message, span, line, column, code: code, phase: CompilerPhase.Parser); + RecordError(message, span, line, column, code); return new ParserAbortException(); } + /// + /// The parser's one error-counting site: every parser error is recorded here, and the + /// --max-errors cap is enforced here and nowhere else (#2311). When this error is the + /// Nth, the SPY0905 note is recorded and a stops the + /// parse at once, so no recovery follows the Nth error. The cap used to be read at the + /// panic-mode catch sites, after the recovery that had already reported a cascade error, + /// so --max-errors N could report N+1. + /// + private void RecordError(string message, Text.TextSpan? span, int line, int column, string code) + { + _diagnostics.AddError(message, span, line, column, code: code, phase: CompilerPhase.Parser); + if (_diagnostics.ErrorCount < _maxErrors) + return; + + _diagnostics.AddWarning( + $"Too many errors ({_maxErrors}); further errors suppressed. Use '--max-errors' to increase the limit.", + Current.Line, Current.Column, + code: DiagnosticCodes.Infrastructure.TooManyErrors, + phase: CompilerPhase.Parser); + throw new ParserErrorLimitException(); + } + private readonly List _tokens; private int _position; private readonly ICompilerLogger _logger; @@ -518,40 +549,35 @@ public Module ParseModule() SkipNewlines(); _lastLoopPosition = -1; - while (!IsAtEnd) + try { - if (!CheckLoopProgress()) - break; - - SkipNewlines(); - if (IsAtEnd) - break; - - var itemStart = _position; - try + while (!IsAtEnd) { - var stmt = ParseStatement(); - statements.Add(stmt); - } - catch (ParserAbortException) - { - // Error already recorded in _diagnostics by ReportError() + if (!CheckLoopProgress()) + break; - // Stop after _maxErrors to avoid cascading false errors - if (_diagnostics.ErrorCount >= _maxErrors) - { - _diagnostics.AddWarning( - $"Too many errors ({_maxErrors}); further errors suppressed. Use '--max-errors' to increase the limit.", - Current.Line, Current.Column, - code: DiagnosticCodes.Infrastructure.TooManyErrors, - phase: CompilerPhase.Parser); + SkipNewlines(); + if (IsAtEnd) break; - } - // Panic-mode recovery: synchronize to next statement boundary - Synchronize(itemStart); + var itemStart = _position; + try + { + var stmt = ParseStatement(); + statements.Add(stmt); + } + catch (ParserAbortException) + { + // Error already recorded in _diagnostics by ReportError(). + // Panic-mode recovery: synchronize to next statement boundary + Synchronize(itemStart); + } + SkipNewlines(); } - SkipNewlines(); + } + catch (ParserErrorLimitException) + { + // The error cap was reached at RecordError, which recorded the SPY0905 note. } var docString = TakeDocString(statements); @@ -582,7 +608,7 @@ public Statement ParseSingleStatement() { return ParseStatement(); } - catch (ParserAbortException) + catch (Exception e) when (e is ParserAbortException or ParserErrorLimitException) { return new ExpressionStatement { @@ -614,28 +640,33 @@ public IReadOnlyList ParseStatements() SkipNewlines(); _lastLoopPosition = -1; - while (!IsAtEnd) + try { - if (!CheckLoopProgress()) - break; - - SkipNewlines(); - if (IsAtEnd) - break; - - var itemStart = _position; - try + while (!IsAtEnd) { - var stmt = ParseStatement(); - statements.Add(stmt); - } - catch (ParserAbortException) - { - if (_diagnostics.ErrorCount >= _maxErrors) + if (!CheckLoopProgress()) break; - Synchronize(itemStart); + + SkipNewlines(); + if (IsAtEnd) + break; + + var itemStart = _position; + try + { + var stmt = ParseStatement(); + statements.Add(stmt); + } + catch (ParserAbortException) + { + Synchronize(itemStart); + } + SkipNewlines(); } - SkipNewlines(); + } + catch (ParserErrorLimitException) + { + // The error cap was reached at RecordError, which recorded the SPY0905 note. } return statements; diff --git a/src/Sharpy.Lsp.Tests/MaxErrorsCapLspRouteTests.cs b/src/Sharpy.Lsp.Tests/MaxErrorsCapLspRouteTests.cs new file mode 100644 index 000000000..1e56075bb --- /dev/null +++ b/src/Sharpy.Lsp.Tests/MaxErrorsCapLspRouteTests.cs @@ -0,0 +1,110 @@ +using System.Text; +using FluentAssertions; +using Microsoft.Extensions.Logging.Abstractions; +using Sharpy.Compiler; +using Sharpy.Compiler.Diagnostics; +using Xunit; + +namespace Sharpy.Lsp.Tests; + +/// +/// The LSP route of the --max-errors matrix (#2311, ruling R-HD a): the editor reports at +/// most the cap's errors, plus at most one SPY0905 note, on the analysis it publishes +/// () and on the parse the syntax handlers read +/// (). +/// +/// +/// The LSP has no cap option: its options come from CompilerOptionsFactory.ForLsp, which no +/// LSP configuration reaches with a maxErrors, so only each phase's default (parser 25, +/// semantic 100) is reachable and the N axis is that one value here. The CLI routes × +/// N {1, 2, default} are Sharpy.Cli.Tests/Commands/MaxErrorsCapMatrixTests, whose shape axis +/// this mirrors (enum and union bodies for the parser phase only, as there). Each program has the +/// default + 2 errors; every parser row but the statement one reported 26 at e7e0a0ef7 — the suite +/// left at the cap reported its definition's Expected Dedent (SPY0104). +/// +public sealed class MaxErrorsCapLspRouteTests : IDisposable +{ + private const int ParserDefault = 25; + private const int SemanticDefault = 100; + + private static readonly string[] SharedShapes = { "statement", "def", "class", "nested" }; + private static readonly string[] ParserOnlyShapes = { "enum", "union" }; + + private readonly string _dir = Path.Combine( + Path.GetTempPath(), "sharpy_lsp_maxerrors_" + Guid.NewGuid().ToString("N")); + + public MaxErrorsCapLspRouteTests() => Directory.CreateDirectory(_dir); + + public void Dispose() + { + try + { + Directory.Delete(_dir, recursive: true); + } + catch (IOException) + { + // Best-effort cleanup. + } + } + + public static IEnumerable Cells() + { + foreach (var phase in new[] { "parser", "semantic" }) + foreach (var shape in phase == "parser" ? SharedShapes.Concat(ParserOnlyShapes) : SharedShapes) + foreach (var route in new[] { "analysis", "parse" }) + { + if (phase == "semantic" && route == "parse") + continue; // The parse route runs no semantic phase. + yield return new object[] { phase, shape, route }; + } + } + + // Anchored to a literal: parser 6 shapes × 2 routes + semantic 4 shapes × the analysis route. + [Fact] + public void Matrix_HasEveryCell() => Cells().Should().HaveCount(16); + + [Theory] + [MemberData(nameof(Cells))] + public async Task Lsp_ReportsAtMostTheDefaultCapErrors(string phase, string shape, string route) + { + var cap = phase == "parser" ? ParserDefault : SemanticDefault; + var path = Path.Combine(_dir, $"{phase}_{shape}.spy"); + var source = Program(phase, shape, cap + 2); + File.WriteAllText(path, source); + var uri = new Uri(path).ToString(); + + using var workspace = new SharpyWorkspace(new CompilerApi(), NullLogger.Instance); + workspace.OpenDocument(uri, source, 1); + var diagnostics = route == "analysis" + ? (await workspace.GetAnalysisAsync(uri))!.Diagnostics + : (await workspace.GetParseResultAsync(uri))!.Diagnostics; + + var rendered = string.Join("\n", diagnostics.Select(d => $"{d.Severity} {d.Code} {d.Line}:{d.Column} {d.Message}")); + diagnostics.Count(d => d.IsError).Should().Be(cap, + $"the {route} route must report exactly the default cap's {cap} errors of the {cap + 2}+:\n{rendered}"); + diagnostics.Count(d => d.Code == DiagnosticCodes.Infrastructure.TooManyErrors).Should().BeLessThanOrEqualTo(1, + $"the cap is noted at most once:\n{rendered}"); + } + + /// The CLI matrix's program generator, verbatim: one error per item of the shape, plus a main. + private static string Program(string phase, string shape, int count) + { + var sb = new StringBuilder(); + for (var i = 1; i <= count; i++) + { + var bad = phase == "parser" ? $"x{i} = = {i}" : $"x{i}: int = undefined_{i}"; + sb.Append(shape switch + { + "statement" => $"{bad}\n", + "def" => $"def f{i}() -> None:\n {bad}\n\n", + "class" => $"class C{i}:\n {bad}\n\n", + "nested" => $"def f{i}() -> None:\n if True:\n {bad}\n\n", + "enum" => $"enum E{i}:\n A = = {i}\n\n", + "union" => $"union U{i}:\n case A(x: = int)\n\n", + _ => throw new ArgumentOutOfRangeException(nameof(shape), shape, null), + }); + } + sb.Append("\ndef main() -> None:\n pass\n"); + return sb.ToString(); + } +} From 28a84a9405163e7e963abba59cbd3acdde4a2589 Mon Sep 17 00:00:00 2001 From: antonsynd <2648730+antonsynd@users.noreply.github.com> Date: Thu, 22 Oct 2026 02:30:13 -0400 Subject: [PATCH 05/13] =?UTF-8?q?fix(cli):=20every=20producing=20command's?= =?UTF-8?q?=20output=20runs=20from=20its=20output=20directory=20unless=20-?= =?UTF-8?q?-no-deps=20=E2=80=94=20build=20and=20project=20deploy=20like=20?= =?UTF-8?q?compile=20and=20run=20(refs=20#2313)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Change (#2313 cell 0, ruling R-HF c). `build` and `project` wrote the assembly only, so `dotnet bin/Debug/net10.0/App.exe` died with FileNotFoundException; P46 documented that as by design (10d0b3d9a). They are now two more callers of the one deployment seam, RuntimeDependencyHelper.CopyRuntimeDependencies: - build, in-process: seeded with CompileResult.UsedAssemblyPaths plus the `-r` list resolved as CompileToBinary hands it to the compiler (compile's seed); `build --server`: the server's reply carries the used assemblies (CompileServerClient.TryRun now returns them), and the client deploys, as `run --server` does. One OutputDirectory rule for both routes and CompileToBinary. - project: seeded with every default reference plus the .spyproj's items and the merged `-r`, captured before the defaults join the list — `compile app.spyproj`'s seed. Both gain `--no-deps`, spelled and described exactly as `compile`'s. Registry: `--no-deps` rows (Honoured) in the build and project tables, evidence ReferenceDeploymentMatrixTests.NoDeps_WritesTheAssemblyOnly. `--help`: build/project descriptions say they deploy (`--no-deps` writes the assembly only); the `-r` "not deployed" reason is gone. README (Build/Project options, Options-by-Command deploy column, Output Structure), module_system.md (Deployment paragraph) and gap-discovery-contracts.md reworded; the Output Structure listing is the measured `project` output below. GlobalOptionProbes invoke build and project with `--no-deps`, as they already did compile, so the globals probes stay deployment-free. Matrix (ReferenceDeploymentMatrixTests): reference source {-r absolute, -r bare name + -m, , absolute, relative} x route {compile .spy, compile .spyproj, build, build --server, project, run, run --server} where the source applies ( sources: .spyproj routes only) = 20 honoured cells, each RUN with `dotnet ` from its output directory and asserting the library and Sharpy.Core.dll beside it; x --no-deps on every route that takes it = 16 cells asserting the assembly (positive control) and neither file. The four former assembly-only controls are subsumed (build/project became honoured cells). ReadmeOptionTableTests: DeployingCommands = {build, compile, project, run}; the deploy cell of each --no-deps command must name the opt-out; build/project help carry the deploy sentence; no deploying command's help carries the old assembly-only sentence or "not deployed beside the output". Mutation (Rule 12), measured on ReferenceDeploymentMatrixTests + ReadmeOptionTableTests: - M-a drop build's in-process caller + build help back to the assembly-only sentence: 4/50 red = [r-absolute|r-bare-name+m x build] + ABuildCommand_SaysItDeploysInItsHelp(build) + ADeployingCommand_HasNoAssemblyOnlySentence(build) - M-b drop project's caller + README project row back to "no — writes the assembly only": 6/50 red = the 5 [* x project] cells + TheDeploysColumn_MatchesTheDeployingCommands - M-c drop build's server-route caller + project ignores --no-deps: 7/50 red = [r-* x build --server] (2) + NoDeps [* x project] (5) - restored: 50/0/0 Direction record (built sharpyc, base e7e0a0ef7 vs this tree; a .spyproj with a reference, and build -r): - base `project app/App.spyproj`: bin/Debug/net10.0 = App.deps.json App.exe App.pdb App.runtimeconfig.json; `dotnet App.exe` -> FileNotFoundException 'Tablegreeter, Version=0.0.0.0', exit 134. Now: + Sharpy.Core.dll, Sharpy.Stdlib.dll and its dependencies, Tablegreeter.dll; `dotnet App.exe` prints "hello from lib", exit 0. - base `build main.spy -r Tablegreeter.dll -o out/main.dll`: out = main .deps.json/.dll/.pdb/.runtimeconfig.json; `dotnet main.dll` -> FileNotFoundException, exit 134. Now: + Sharpy.Core.dll, Tablegreeter.dll, libe_sqlite3.dylib; runs, exit 0. - `--no-deps` on build/project: base "Unrecognized command or argument '--no-deps'", exit 1. Now: exit 0, the base layout (the four files). Filtered counts @ 344402c44 + this change (measured): - Sharpy.Cli.Tests (whole project): 1033/0/1 - build_tools/check_spec_blocks.sh (built sharpyc): clean, 0 failing - dotnet format whitespace sharpy.sln --verify-no-changes: clean Expected values changed (they pinned the behaviour this cures): ReadmeOptionTableTests.DeployingCommands/AssemblyOnlyCommands, the deploy-cell rule, AWrongDeploysCell_IsReported's spelling, and the help tests (AnAssemblyOnlyCommand_SaysSoInItsHelp -> ABuildCommand_SaysItDeploysInItsHelp); ReferenceDeploymentMatrixTests' Control_TheOptOuts_WriteTheAssemblyOnly build/project rows (now honoured cells; the compile --no-deps rows live on in NoDeps_WritesTheAssemblyOnly). Not done: the compile server's `project` request (ProjectCommand.CompileProjectForServer) has no CLI client and does not deploy; no producing command reaches it. Co-Authored-By: Claude Opus 5.5 (1M context) Claude-Session: https://claude.ai/code/session_0126pNZqmMsSbn4WFFPhpnxW --- docs/design/gap-discovery-contracts.md | 2 +- docs/language_specification/module_system.md | 11 +- .../CliOptionEffectSweepTests.cs | 4 +- .../E2E/ReferenceDeploymentMatrixTests.cs | 218 +++++++++++------- src/Sharpy.Cli.Tests/GlobalOptionProbes.cs | 4 +- .../ReadmeOptionTableTests.cs | 44 ++-- src/Sharpy.Cli/CommandOptionEffects.cs | 2 + src/Sharpy.Cli/Commands/BuildCommand.cs | 54 ++++- src/Sharpy.Cli/Commands/ProjectCommand.cs | 38 ++- src/Sharpy.Cli/README.md | 38 +-- src/Sharpy.Cli/RuntimeDependencyHelper.cs | 5 +- .../Services/CompileServerClient.cs | 7 +- 12 files changed, 277 insertions(+), 150 deletions(-) diff --git a/docs/design/gap-discovery-contracts.md b/docs/design/gap-discovery-contracts.md index bf4cdf745..12e7435b8 100644 --- a/docs/design/gap-discovery-contracts.md +++ b/docs/design/gap-discovery-contracts.md @@ -39,7 +39,7 @@ source-scan guards run in the regular suite. Well-guarded classes needing no new mechanism: lambda-boundary parsing (differential parse oracle, #1037), unparser fidelity (round-trip property tests), CLR name round-trip (interop sweep + #1040), -deployment closure (`StandaloneDeploymentTests` for the used stdlib closure; `ReferenceDeploymentMatrixTests` for the user-reference axis it lacked, reference source × route with `--no-deps`, `build` and `project` as the assembly-only controls, #2223). +deployment closure (`StandaloneDeploymentTests` for the used stdlib closure; `ReferenceDeploymentMatrixTests` for the user-reference axis it lacked, reference source × route × {default, `--no-deps`} over every producing command, #2223; `build` and `project` deploy since #2313 cell 0). | Emittable-constant admission (#1762, #1769, #1460) | Every parameter default, decorator argument and module `const` initializer the validator admits is one the emitter prints as a C# constant (`const` / `default`): ONE classifier (`Semantic/Validation/ConstantDefaultClassifier`) with per-site admission tables decides, and `CodeGenInfo.IsCompileTimeConstant` is computed from the same `ModuleConst` table — the emitter re-derives nothing | `ParameterDefaultConstantMatrixTests` (kind × host, every admitted cell an executing program, every refused cell SPY0401, SPY0908 in no cell, totality-anchored) + the `DispatchSiteInventoryTests` row for the classifier switch + `EmitterParameterDefaultSeamConformanceTests` (Roslyn syntax scan of `CodeGen/`: every `WithDefault(` takes `GenerateParameterDefault(...)`; the late-bound `x_lb = null` sentinel is the one non-user default and is admitted by exact shape, not by site; seam and sentinel site counts anchored to literals so an empty scan fails) | none | | LiteralString use surface (#1766, #1741) | A `LiteralString` value is a `str` at every value-use route — operators, comparison chains, protocols, truthiness, indexing and slicing, iteration, membership, method dispatch including the static-extension lowering — through ONE `OperandView`; the store seam is untouched (`LiteralString <: str` already) | `LiteralStringSurfaceMatrixTests` (receiver {`str`, `LiteralString`} × operation; the two columns print identical stdout; refusal twin refuses identically; the R-P `x += s` control refuses on the `LiteralString` column only) | none | | SemanticType codec totality (#1751) | The warm (`--incremental`) build round-trips every `SemanticType` kind the cold build produced: the serializer registry is total over the concrete subclasses, or names its exclusion with a reason that is itself asserted | `TypeCodecRegistryTotalityTests` (reflection over every concrete `SemanticType`; the exclusion set is asserted equal to `{ConstructorReferenceType}` and its reason — SPY0342 refuses the export at the cold build — is the positive control) + `ColdWarmBuildDifferentialTests` (`LiteralString` signature and `-> Self` programs, cold == warm) | none | diff --git a/docs/language_specification/module_system.md b/docs/language_specification/module_system.md index bfa1b9b85..10eb52c8c 100644 --- a/docs/language_specification/module_system.md +++ b/docs/language_specification/module_system.md @@ -135,12 +135,13 @@ $ sharpyc project app/app.spyproj error[SPY0305]: Assembly not found: /…/libs/Greeter.dll ``` -Deployment: `compile`, for a `.spy` or a `.spyproj`, and `run` copy every referenced assembly -beside the output, whether the program uses it or not, together with the dependencies that sit -beside it. `compile --no-deps` copies none of them (the assembly and its `.deps.json` / +Deployment: every command that produces an assembly — `build`, `compile` (for a `.spy` or a +`.spyproj`), `project` and `run` — copies every referenced assembly beside the output, whether the +program uses it or not, together with `Sharpy.Core.dll`, the standard-library assemblies and the +dependencies that sit beside each, so the output directory runs with `dotnet`. `--no-deps` on +`build`, `compile` or `project` copies none of them (the assembly and its `.deps.json` / `.runtimeconfig.json` host files are still written), and a `--self-contained` publish carries every -reference. `build` and `project` write the assembly only, without its references or -`Sharpy.Core.dll`, so a directory meant to run with `dotnet` comes from `compile`. +reference. For example, a library built from `greeter.spy` with `sharpyc compile greeter.spy -t library -o libs/Greeter.dll`: diff --git a/src/Sharpy.Cli.Tests/CliOptionEffectSweepTests.cs b/src/Sharpy.Cli.Tests/CliOptionEffectSweepTests.cs index 4b7e25da4..2b91d66a8 100644 --- a/src/Sharpy.Cli.Tests/CliOptionEffectSweepTests.cs +++ b/src/Sharpy.Cli.Tests/CliOptionEffectSweepTests.cs @@ -303,6 +303,7 @@ internal static System.Collections.Generic.List TotalityFailures(RootCom ["build --reference"] = "Sharpy.Cli.Tests.Commands.OptionEffectEvidenceTests.Build_Reference_BindsTheImport", ["build --module-path"] = "Sharpy.Cli.Tests.Commands.OptionEffectEvidenceTests.Build_ModulePath_ResolvesAnImport", ["build --server"] = "Sharpy.Cli.Tests.E2E.CompileServerE2ETests.Client_FallsBackToInProcess_WhenNoServerIsRunning", + ["build --no-deps"] = "Sharpy.Cli.Tests.E2E.ReferenceDeploymentMatrixTests.NoDeps_WritesTheAssemblyOnly", ["compile@.spy --output"] = "Sharpy.Cli.Tests.Commands.CompileCommandTests.Compile_ValidSource_ProducesDll", ["compile@.spyproj --output"] = "Sharpy.Cli.Tests.Commands.GeneratedCSharpLayoutTests.CompileProject_OutputOption_PlacesTheAssemblyAndItsCSharp", @@ -315,7 +316,7 @@ internal static System.Collections.Generic.List TotalityFailures(RootCom ["compile@.spyproj --module-path"] = "Sharpy.Cli.Tests.Commands.CompileOptionEffectTableTests.ModulePath_ResolvesABareReferenceName", ["compile@.spy --self-contained"] = "Sharpy.Cli.Tests.E2E.ReferenceDeploymentMatrixTests.SelfContained_Compile_PublishesTheReference", ["compile@.spy --no-deps"] = "Sharpy.Cli.Tests.Commands.CompileCommandTests.Compile_NoDeps_SkipsCopyingRuntimeDependencies", - ["compile@.spyproj --no-deps"] = "Sharpy.Cli.Tests.E2E.ReferenceDeploymentMatrixTests.Control_TheOptOuts_WriteTheAssemblyOnly", + ["compile@.spyproj --no-deps"] = "Sharpy.Cli.Tests.E2E.ReferenceDeploymentMatrixTests.NoDeps_WritesTheAssemblyOnly", ["compile@.spyproj --incremental"] = "Sharpy.Cli.Tests.Commands.OptionEffectEvidenceTests.CompileProject_Incremental_WritesTheIncrementalCache", ["compile@.spyproj --clean"] = "Sharpy.Cli.Tests.Commands.OptionEffectEvidenceTests.CompileProject_Clean_DeletesBinAndObjFirst", ["compile@.spy --emit-csharp"] = "Sharpy.Cli.Tests.Commands.CompileCommandTests.Compile_EmitCSharp_WritesCSharpFileAlongsideDll", @@ -335,6 +336,7 @@ internal static System.Collections.Generic.List TotalityFailures(RootCom ["project --reference"] = "Sharpy.Cli.Tests.E2E.ProjectReferenceOptionTests.AReference_IsHonoured_AndAMissingOneIsSpy0305", ["project --module-path"] = "Sharpy.Cli.Tests.E2E.ModuleSearchPathCliTests.ProjectModulePathOption_ReachesImportResolution", ["project --incremental"] = "Sharpy.Cli.Tests.Commands.OptionEffectEvidenceTests.Project_Incremental_WritesTheIncrementalCache", + ["project --no-deps"] = "Sharpy.Cli.Tests.E2E.ReferenceDeploymentMatrixTests.NoDeps_WritesTheAssemblyOnly", ["emit csharp --output"] = "Sharpy.Cli.Tests.Commands.EmitCommandTests.CSharp_WritesGeneratedFile", ["emit csharp --reference"] = "Sharpy.Cli.Tests.Commands.OptionEffectEvidenceTests.EmitCSharp_Reference_BindsTheImport", diff --git a/src/Sharpy.Cli.Tests/E2E/ReferenceDeploymentMatrixTests.cs b/src/Sharpy.Cli.Tests/E2E/ReferenceDeploymentMatrixTests.cs index d6acb1e8a..37a96bbb6 100644 --- a/src/Sharpy.Cli.Tests/E2E/ReferenceDeploymentMatrixTests.cs +++ b/src/Sharpy.Cli.Tests/E2E/ReferenceDeploymentMatrixTests.cs @@ -11,21 +11,24 @@ namespace Sharpy.Cli.Tests.E2E; /// directory. The deployment set is the used closure plus every explicit reference, because a /// referenced Sharpy library's imports resolve through the CLR-namespace arms, which record no /// usage. Before the fix no route copied the library, and dotnet out/main.dll died with -/// FileNotFoundException for it. +/// FileNotFoundException for it. Every producing command deploys unless --no-deps (#2313 +/// cell 0, ruling R-HF c): build and project used to write the assembly only, so +/// dotnet bin/Debug/net10.0/App.exe died with FileNotFoundException for Sharpy.Core. /// /// /// The matrix is reference source × route, wherever the source applies. The sources are /// -r as an absolute path, -r as a bare name found on -m, /// <Reference Include="abs"/>, and <Reference Include="Tablegreeter"> with a /// <HintPath> both absolute and relative to the .spyproj directory. The routes are -/// compile x.spy, compile x.spyproj, run and run --server against a -/// sharpyc server started for the cell. The <Reference> sources apply only to a -/// .spyproj. Each cell asserts the output directory holds Tablegreeter.dll and that -/// dotnet <out> prints the library's line; a run cell also reads the line after its -/// banner. -/// The controls are the documented opt-outs. --no-deps on compile (both kinds), -/// build and project write the assembly only, so the library is absent although the -/// assembly is present. +/// compile x.spy, compile x.spyproj, build, project, run, and +/// build --server / run --server against a sharpyc server started for the +/// cell. The <Reference> sources apply only to a .spyproj (compile x.spyproj, +/// project). Each cell asserts the output directory holds Tablegreeter.dll and +/// Sharpy.Core.dll and that dotnet <out>, run from that directory, prints the +/// library's line; a run cell also reads the line after its banner. +/// The --no-deps cells are the same matrix with the opt-out applied, on every route that +/// takes it (all but run): the assembly is written — the positive control — and neither the +/// library nor Sharpy.Core.dll is beside it. /// The two self-contained cells shell out to dotnet publish, so they are slow. They are /// untraited, like , and run in the regular suite. /// Every process is the real sharpyc.dll, as in , @@ -36,6 +39,7 @@ public sealed class ReferenceDeploymentMatrixTests : IClassFixture _ws.Dispose(); - public static TheoryData Cells() + static readonly string[] CommandLineSources = { "r-absolute", "r-bare-name+m" }; + static readonly string[] ProjectItemSources = { "Reference Include", "Reference HintPath absolute", "Reference HintPath relative" }; + + /// Every (source, route) cell; for --no-deps, only the routes that take it. + static IEnumerable<(string Source, string Route)> AllCells(bool noDeps) { - var data = new TheoryData(); - foreach (var source in new[] { "r-absolute", "r-bare-name+m" }) + foreach (var source in CommandLineSources) { - foreach (var route in new[] { "compile .spy", "compile .spyproj", "run", "run --server" }) - data.Add(source, route); + foreach (var route in new[] { "compile .spy", "compile .spyproj", "build", "build --server", "project", "run", "run --server" }) + { + if (noDeps && route.StartsWith("run", StringComparison.Ordinal)) + continue; // `run` deploys beside the program it runs; it has no --no-deps. + yield return (source, route); + } } - foreach (var source in new[] { "Reference Include", "Reference HintPath absolute", "Reference HintPath relative" }) - data.Add(source, "compile .spyproj"); + foreach (var source in ProjectItemSources) + { + foreach (var route in new[] { "compile .spyproj", "project" }) + yield return (source, route); + } + } + public static TheoryData Cells() + { + var data = new TheoryData(); + foreach (var (source, route) in AllCells(noDeps: false)) + data.Add(source, route); + return data; + } + + public static TheoryData NoDepsCells() + { + var data = new TheoryData(); + foreach (var (source, route) in AllCells(noDeps: true)) + data.Add(source, route); return data; } + [Fact] + public void TheMatrix_HasEveryCell() + { + // Anchored to literals: 2 command-line sources × 7 routes + 3 project-item sources × 2 routes, + // and for --no-deps the same less the 2 × 2 run cells. + Cells().Count().Should().Be(20); + NoDepsCells().Count().Should().Be(16); + } + [Theory] [MemberData(nameof(Cells))] public void UserReference_IsDeployedBesideTheOutput_AndTheProgramRunsFromThere(string source, string route) { var cell = $"[{source} × {route}]"; - var outDir = _ws.PathFor("out"); - var (input, extraArgs) = Input(source, route); - - string assembly; - if (route.StartsWith("compile", StringComparison.Ordinal)) - { - assembly = Path.Combine(outDir, route == "compile .spy" ? "main.dll" : "App.dll"); - var compile = ExecCli(new[] { "compile", input, "-o", assembly }.Concat(extraArgs).ToArray()); - compile.ExitCode.Should().Be(0, $"{cell} compile\n{compile.StdOut}\n{compile.StdErr}"); - } - else - { - assembly = Path.Combine(outDir, "main.exe"); - var args = new[] { "run", input, "-o", assembly }.Concat(extraArgs).ToList(); - ProcessResult run; - if (route == "run --server") - { - var pipe = "spy-rd-" + Guid.NewGuid().ToString("N")[..8]; - using var server = StartServer(pipe); - try - { - WaitUntilListening(pipe, server); - args.Add($"--server={pipe}"); - run = ExecCli(args.ToArray()); - } - finally - { - TryShutdown(pipe); - StopServer(server); - } - - run.StdErr.Should().NotContain("compiling in-process", $"{cell} the server must serve the compile"); - } - else - { - run = ExecCli(args.ToArray()); - } - - run.ExitCode.Should().Be(0, $"{cell} run\n{run.StdOut}\n{run.StdErr}"); - AfterBanner(run.StdOut).Should().Be(LibraryLine, $"{cell} the program's own line follows the banner"); - } + var assembly = Produce(source, route, noDeps: false, cell); + var outDir = Path.GetDirectoryName(assembly)!; File.Exists(assembly).Should().BeTrue($"{cell} the assembly is written"); File.Exists(Path.Combine(outDir, LibraryFile)).Should().BeTrue($"{cell} the reference is deployed beside the output"); + File.Exists(Path.Combine(outDir, CoreFile)).Should().BeTrue($"{cell} the runtime is deployed beside the output"); var standalone = Exec(outDir, "dotnet", assembly); standalone.ExitCode.Should().Be(0, $"{cell} dotnet {Path.GetFileName(assembly)}\n{standalone.StdOut}\n{standalone.StdErr}"); @@ -121,51 +120,94 @@ public void UserReference_IsDeployedBesideTheOutput_AndTheProgramRunsFromThere(s } /// - /// The documented opt-outs write the assembly only. Each is the matching matrix cell with the - /// opt-out applied, so the assembly's presence is the positive control for the library's absence. + /// The documented opt-out writes the assembly only. Each cell is the matching matrix cell with + /// --no-deps applied, so the assembly's presence is the positive control for the absence of + /// the library and the runtime. /// [Theory] - [InlineData("compile .spy --no-deps")] - [InlineData("compile .spyproj --no-deps")] - [InlineData("build")] - [InlineData("project")] - public void Control_TheOptOuts_WriteTheAssemblyOnly(string control) + [MemberData(nameof(NoDepsCells))] + public void NoDeps_WritesTheAssemblyOnly(string source, string route) + { + var cell = $"[{source} × {route} --no-deps]"; + var assembly = Produce(source, route, noDeps: true, cell); + var outDir = Path.GetDirectoryName(assembly)!; + + File.Exists(assembly).Should().BeTrue($"{cell} the assembly is written (positive control)"); + File.Exists(Path.Combine(outDir, LibraryFile)).Should().BeFalse($"{cell} writes the assembly only"); + File.Exists(Path.Combine(outDir, CoreFile)).Should().BeFalse($"{cell} writes the assembly only"); + } + + /// + /// Runs on the cell's input and returns the path of the assembly it + /// wrote; a run cell also reads the program's own line after the banner. + /// + string Produce(string source, string route, bool noDeps, string cell) { var outDir = _ws.PathFor("out"); + var (input, extraArgs) = Input(source, route); + string assembly; - string assemblyDir; - ProcessResult result; - switch (control) + var args = new System.Collections.Generic.List(); + switch (route) { - case "compile .spy --no-deps": - assembly = Path.Combine(outDir, "main.dll"); - result = ExecCli("compile", WriteMain(), "-r", _library.DllPath, "-o", assembly, "--no-deps"); - break; - case "compile .spyproj --no-deps": - assembly = Path.Combine(outDir, "App.dll"); - result = ExecCli("compile", WriteProject($""), "-o", assembly, "--no-deps"); + case "compile .spy": + case "compile .spyproj": + assembly = Path.Combine(outDir, route == "compile .spy" ? "main.dll" : "App.dll"); + args.AddRange(new[] { "compile", input, "-o", assembly }); break; case "build": + case "build --server": assembly = Path.Combine(outDir, "main.dll"); - result = ExecCli("build", WriteMain(), "-r", _library.DllPath, "-o", assembly); + args.AddRange(new[] { "build", input, "-o", assembly }); break; case "project": - var project = WriteProject($""); - result = ExecCli("project", project); - // An exe project writes bin///App.exe (ProjectConfig.OutputAssemblyPath). - var binDir = Path.Combine(Path.GetDirectoryName(project)!, "bin"); - assembly = Directory.Exists(binDir) - ? Directory.GetFiles(binDir, "App.exe", SearchOption.AllDirectories).FirstOrDefault() ?? Path.Combine(binDir, "App.exe") - : Path.Combine(binDir, "App.exe"); + // An exe project writes bin///App.exe (ProjectConfig.OutputAssemblyPath); + // `project` takes no -o. + assembly = Path.Combine(Path.GetDirectoryName(input)!, "bin", "Debug", "net10.0", "App.exe"); + args.AddRange(new[] { "project", input }); + break; + case "run": + case "run --server": + assembly = Path.Combine(outDir, "main.exe"); + args.AddRange(new[] { "run", input, "-o", assembly }); break; default: - throw new ArgumentOutOfRangeException(nameof(control), control, null); + throw new ArgumentOutOfRangeException(nameof(route), route, null); } - assemblyDir = Path.GetDirectoryName(assembly)!; - result.ExitCode.Should().Be(0, $"[{control}]\n{result.StdOut}\n{result.StdErr}"); - File.Exists(assembly).Should().BeTrue($"[{control}] the assembly is written (positive control)"); - File.Exists(Path.Combine(assemblyDir, LibraryFile)).Should().BeFalse($"[{control}] writes the assembly only"); + args.AddRange(extraArgs); + if (noDeps) + args.Add("--no-deps"); + + ProcessResult result; + if (route.EndsWith("--server", StringComparison.Ordinal)) + { + var pipe = "spy-rd-" + Guid.NewGuid().ToString("N")[..8]; + using var server = StartServer(pipe); + try + { + WaitUntilListening(pipe, server); + args.Add($"--server={pipe}"); + result = ExecCli(args.ToArray()); + } + finally + { + TryShutdown(pipe); + StopServer(server); + } + + result.StdErr.Should().NotContain("compiling in-process", $"{cell} the server must serve the compile"); + } + else + { + result = ExecCli(args.ToArray()); + } + + result.ExitCode.Should().Be(0, $"{cell}\n{result.StdOut}\n{result.StdErr}"); + if (route.StartsWith("run", StringComparison.Ordinal)) + AfterBanner(result.StdOut).Should().Be(LibraryLine, $"{cell} the program's own line follows the banner"); + + return assembly; } [Fact] @@ -201,7 +243,7 @@ public void SelfContained_Run_PublishesTheReference() _ => Array.Empty(), }; - if (route is "compile .spy" or "run" or "run --server") + if (route is "compile .spy" or "build" or "build --server" or "run" or "run --server") return (WriteMain(), commandLineReference); var projectDir = _ws.PathFor("app"); diff --git a/src/Sharpy.Cli.Tests/GlobalOptionProbes.cs b/src/Sharpy.Cli.Tests/GlobalOptionProbes.cs index 26072846f..2d9013381 100644 --- a/src/Sharpy.Cli.Tests/GlobalOptionProbes.cs +++ b/src/Sharpy.Cli.Tests/GlobalOptionProbes.cs @@ -44,10 +44,10 @@ internal enum Fixture /// internal static readonly IReadOnlyDictionary Invocations = new Dictionary(StringComparer.Ordinal) { - ["build"] = "build \"{input}\" -o \"{out}/a.dll\"", + ["build"] = "build \"{input}\" -o \"{out}/a.dll\" --no-deps", ["compile"] = "compile \"{input}\" -o \"{out}/a.dll\" --no-deps", ["run"] = "run \"{input}\"", - ["project"] = "project \"{input}\"", + ["project"] = "project \"{input}\" --no-deps", ["emit tokens"] = "emit tokens \"{input}\"", ["emit ast"] = "emit ast \"{input}\"", ["emit parse"] = "emit parse \"{input}\"", diff --git a/src/Sharpy.Cli.Tests/ReadmeOptionTableTests.cs b/src/Sharpy.Cli.Tests/ReadmeOptionTableTests.cs index 148402393..d86a309e6 100644 --- a/src/Sharpy.Cli.Tests/ReadmeOptionTableTests.cs +++ b/src/Sharpy.Cli.Tests/ReadmeOptionTableTests.cs @@ -12,7 +12,7 @@ namespace Sharpy.Cli.Tests; /// "global options without effect" column say what the command's table says. The deployment column /// is prose the registry does not hold: it is pinned against the literal set of deploying commands /// (, the callers of RuntimeDependencyHelper.CopyRuntimeDependencies), -/// as are the `build`/`project` help sentences that say they write the assembly only +/// as are the `build`/`project` help sentences that say they deploy unless `--no-deps` /// (ReferenceDeploymentMatrixTests observes the behaviour itself). /// public class ReadmeOptionTableTests @@ -71,16 +71,22 @@ internal static Dictionary ParseDeployColumn(string readme) /// /// The commands whose action copies the runtime dependencies and user references beside the - /// output: the callers of RuntimeDependencyHelper.CopyRuntimeDependencies (#2223). `build` and - /// `project` write the assembly only. A literal, not derived from the registry, so the README - /// and help text are pinned against something other than themselves. + /// output: the callers of RuntimeDependencyHelper.CopyRuntimeDependencies (#2223) — every + /// producing command since `build` and `project` joined (#2313 cell 0, R-HF c). A literal, not + /// derived from the registry, so the README and help text are pinned against something other + /// than themselves. /// - internal static readonly IReadOnlySet DeployingCommands = new HashSet(StringComparer.Ordinal) { "compile", "run" }; + internal static readonly IReadOnlySet DeployingCommands = new HashSet(StringComparer.Ordinal) { "build", "compile", "project", "run" }; - internal static readonly IReadOnlySet AssemblyOnlyCommands = new HashSet(StringComparer.Ordinal) { "build", "project" }; + /// The deploying commands that opt out with --no-deps; `run` deploys beside the program it runs. + internal static readonly IReadOnlySet NoDepsCommands = new HashSet(StringComparer.Ordinal) { "build", "compile", "project" }; + /// The help sentence `build` and `project` carried while they wrote the assembly only (P46, 10d0b3d9a). const string AssemblyOnlySentence = "Writes the assembly only; `compile` deploys its runtime dependencies and references beside the output"; + /// The help sentence of `build` and `project` now that they deploy. + const string DeploysSentence = "Copies its runtime dependencies and references beside the output, so it runs from there (--no-deps writes the assembly only)"; + /// Every way the README's "Deploys references" column disagrees with . internal static System.Collections.Generic.List DeployMismatches(string readme) { @@ -88,13 +94,11 @@ internal static System.Collections.Generic.List DeployMismatches(string var problems = new System.Collections.Generic.List(); foreach (var (path, cell) in deploys) { - var expected = DeployingCommands.Contains(path) ? "yes" - : AssemblyOnlyCommands.Contains(path) ? "no" - : "—"; + var expected = DeployingCommands.Contains(path) ? "yes" : "—"; if (!cell.StartsWith(expected, StringComparison.Ordinal)) problems.Add($"`{path}`: the Deploys references cell says '{cell}', expected it to start with '{expected}'"); - if (AssemblyOnlyCommands.Contains(path) && !cell.Contains("writes the assembly only", StringComparison.Ordinal)) - problems.Add($"`{path}`: the Deploys references cell does not say it writes the assembly only: '{cell}'"); + if (NoDepsCommands.Contains(path) && !cell.Contains("unless `--no-deps`", StringComparison.Ordinal)) + problems.Add($"`{path}`: the Deploys references cell does not name the --no-deps opt-out: '{cell}'"); } return problems; @@ -185,31 +189,33 @@ public void AWrongDeploysCell_IsReported() { var readme = ReadmeText(); var line = readme.Split('\n').Single(l => l.StartsWith("| `project` |", StringComparison.Ordinal)); - var wrong = readme.Replace(line, line.Replace("| no — writes the assembly only |", "| yes |", StringComparison.Ordinal), StringComparison.Ordinal); + var wrong = readme.Replace(line, line.Replace("| yes, unless `--no-deps` |", "| no — writes the assembly only |", StringComparison.Ordinal), StringComparison.Ordinal); DeployMismatches(wrong).Should().HaveCount(2) .And.AllSatisfy(p => p.Should().StartWith("`project`:")); } /// - /// `build --help` and `project --help` say they write the assembly only and name `compile` as the - /// deploying command; their `-r` line says the reference is not deployed. The deploying commands' - /// help carries no such sentence (positive control: the sentence is specific, not boilerplate). + /// `build --help` and `project --help` say they deploy and name the `--no-deps` opt-out. No + /// deploying command's help carries the assembly-only sentence or the "not deployed" `-r` reason + /// `build` and `project` carried before #2313 cell 0; each one that takes `--no-deps` lists it + /// (the positive control for those absences). /// [Theory] [InlineData("build")] [InlineData("project")] - public void AnAssemblyOnlyCommand_SaysSoInItsHelp(string command) + public void ABuildCommand_SaysItDeploysInItsHelp(string command) { var help = CliTestHarness.Invoke($"{command} --help"); help.ExitCode.Should().Be(0); - help.StdOut.Should().Contain(AssemblyOnlySentence); - help.StdOut.Should().Contain("not deployed beside the output", $"the -r line of `{command} --help` carries the deploy reason"); + help.StdOut.Should().Contain(DeploysSentence); } [Theory] + [InlineData("build")] [InlineData("compile")] + [InlineData("project")] [InlineData("run")] public void ADeployingCommand_HasNoAssemblyOnlySentence(string command) { @@ -218,5 +224,7 @@ public void ADeployingCommand_HasNoAssemblyOnlySentence(string command) help.ExitCode.Should().Be(0); help.StdOut.Should().NotContain(AssemblyOnlySentence); help.StdOut.Should().NotContain("not deployed beside the output"); + if (NoDepsCommands.Contains(command)) + help.StdOut.Should().Contain("--no-deps", $"`{command}` takes the opt-out"); } } diff --git a/src/Sharpy.Cli/CommandOptionEffects.cs b/src/Sharpy.Cli/CommandOptionEffects.cs index 0077d53a2..357904f31 100644 --- a/src/Sharpy.Cli/CommandOptionEffects.cs +++ b/src/Sharpy.Cli/CommandOptionEffects.cs @@ -244,6 +244,7 @@ static CommandEffectTable CompileTable() ["--reference"] = H("a reference that cannot be found is SPY0305"), ["--project-reference"] = R(CliHelpers.ProjectReferenceUnsupportedReason), ["--module-path"] = H("a directory that does not exist is an error"), + ["--no-deps"] = H("runtime dependencies are not copied beside the output"), ["--server"] = H("compiles through a running 'sharpyc server'"), }), @@ -271,6 +272,7 @@ static CommandEffectTable CompileTable() ["--incremental"] = H("unchanged files are not recompiled"), ["--reference"] = H("added to the .spyproj's items; a reference that cannot be found is SPY0305"), ["--module-path"] = H("added to the .spyproj's items; a directory that does not exist is an error"), + ["--no-deps"] = H("runtime dependencies are not copied beside the output"), }), ["emit tokens"] = Table(CommandPerforms.Lexes, new[] { InputKind.SpyFile }, diff --git a/src/Sharpy.Cli/Commands/BuildCommand.cs b/src/Sharpy.Cli/Commands/BuildCommand.cs index 5c72acc36..470023a0f 100644 --- a/src/Sharpy.Cli/Commands/BuildCommand.cs +++ b/src/Sharpy.Cli/Commands/BuildCommand.cs @@ -12,7 +12,7 @@ internal static class BuildCommand { internal static void Configure(RootCommand root, GlobalOptions globals) { - var command = new Command("build", "Compile a Sharpy source file to a binary or library. Writes the assembly only; `compile` deploys its runtime dependencies and references beside the output"); + var command = new Command("build", "Compile a Sharpy source file to a binary or library. Copies its runtime dependencies and references beside the output, so it runs from there (--no-deps writes the assembly only)"); var inputArg = new Argument("input") { Description = "Sharpy source file to compile" }; var typeOpt = new Option("--type") { Description = "Output type: 'exe' or 'library' (default: exe)" }; @@ -20,12 +20,14 @@ internal static void Configure(RootCommand root, GlobalOptions globals) var outputOpt = new Option("--output") { Description = "Output file path" }; outputOpt.Aliases.Add("-o"); // One value per occurrence — repeat the flag to collect more (#1179, #1215). - var refOpt = new Option("--reference") { Description = "Add a .NET assembly reference (repeatable); not deployed beside the output — `compile` deploys" }; + var refOpt = new Option("--reference") { Description = "Add a .NET assembly reference (repeatable)" }; refOpt.Aliases.Add("-r"); var projRefOpt = new Option("--project-reference") { Description = "Add a .NET project reference (repeatable)" }; projRefOpt.Aliases.Add("-p"); var modPathOpt = new Option("--module-path") { Description = "Additional path to search for modules (repeatable)" }; modPathOpt.Aliases.Add("-m"); + // The spelling `compile` has always used (#2313 cell 0, R-HF c). + var noDepsOpt = new Option("--no-deps") { Description = "Skip copying runtime dependencies alongside the output" }; var serverOpt = new Option("--server") { Description = "Compile via a keep-alive 'sharpyc server' on the given pipe (default pipe if no name); falls back to in-process if none is running", Arity = ArgumentArity.ZeroOrOne }; command.Arguments.Add(inputArg); @@ -34,6 +36,7 @@ internal static void Configure(RootCommand root, GlobalOptions globals) command.Options.Add(refOpt); command.Options.Add(projRefOpt); command.Options.Add(modPathOpt); + command.Options.Add(noDepsOpt); command.Options.Add(serverOpt); command.SetAction((parseResult) => @@ -50,6 +53,7 @@ internal static void Configure(RootCommand root, GlobalOptions globals) var reference = parseResult.GetValue(refOpt) ?? Array.Empty(); var projectReference = parseResult.GetValue(projRefOpt) ?? Array.Empty(); var modulePath = parseResult.GetValue(modPathOpt) ?? Array.Empty(); + var noDeps = parseResult.GetValue(noDepsOpt); var logLevel = globals.ResolveLogLevel(parseResult); CliHelpers.ShowDiagnosticProvenance = parseResult.GetValue(globals.Verbose); var logFile = parseResult.GetValue(globals.LogFile); @@ -89,20 +93,58 @@ internal static void Configure(RootCommand root, GlobalOptions globals) }; var serverExit = CompileServerClient.TryRun( - pipeName, request, Console.Out, Console.Error, verbose: parseResult.GetValue(globals.Verbose)); + pipeName, request, Console.Out, Console.Error, out var serverUsedPaths, verbose: parseResult.GetValue(globals.Verbose)); if (serverExit.HasValue) { + // The server wrote the assembly; the deployment is the client's, as for `run --server`. + if (serverExit.Value == 0 && !noDeps) + { + Deploy(OutputDirectory(output), serverUsedPaths, reference, modulePath); + } return serverExit.Value; } } var compileResult = CompileToBinary(input, type, output, reference, modulePath, logger, metricsFormat, metricsOutput, out var failureExitCode, warnAsError, nowarn, maxErrors, configuration: "Debug", features: features); - return compileResult == null ? failureExitCode : 0; + if (compileResult == null) + { + return failureExitCode; + } + + if (!noDeps) + { + var outputDir = compileResult.OutputAssemblyPath != null + ? Path.GetDirectoryName(compileResult.OutputAssemblyPath) ?? OutputDirectory(output) + : OutputDirectory(output); + Deploy(outputDir, compileResult.UsedAssemblyPaths, reference, modulePath); + } + return 0; }); root.Subcommands.Add(command); } + /// + /// `build` deploys like `compile` and `run` (#2313 cell 0, R-HF c): Sharpy.Core, the used + /// standard-library assemblies and every `-r` reference, with what sits beside each, are copied + /// beside the output, so it runs in place as `dotnet build`'s output does. The deployment seed's + /// user half is the `-r` list resolved exactly as hands it to the + /// compiler — the same seed `compile` forms (#2223). + /// + static void Deploy(string outputDir, IReadOnlySet usedAssemblyPaths, string[] references, string[] modulePaths) + => RuntimeDependencyHelper.CopyRuntimeDependencies( + outputDir, usedAssemblyPaths, CliHelpers.ResolveReferences(references, modulePaths)); + + /// + /// The directory `build` writes the assembly to: -o's directory, else the working + /// directory — the rule applies, and the one a server compile + /// applies with the client's working directory. + /// + static string OutputDirectory(FileInfo? output) + => output != null + ? Path.GetDirectoryName(output.FullName) ?? Directory.GetCurrentDirectory() + : Directory.GetCurrentDirectory(); + /// /// Compiles a Sharpy source file to a binary. Returns the /// on success, or null if compilation or assembly generation failed; then @@ -145,9 +187,7 @@ internal static void Configure(RootCommand root, GlobalOptions globals) var source = File.ReadAllText(inputFile.FullName); var inputFileName = Path.GetFileNameWithoutExtension(inputFile.Name); - var outputDir = output != null - ? Path.GetDirectoryName(output.FullName) ?? Directory.GetCurrentDirectory() - : Directory.GetCurrentDirectory(); + var outputDir = OutputDirectory(output); var assemblyName = output != null ? Path.GetFileNameWithoutExtension(output.Name) diff --git a/src/Sharpy.Cli/Commands/ProjectCommand.cs b/src/Sharpy.Cli/Commands/ProjectCommand.cs index 47b1b517f..50a5b6492 100644 --- a/src/Sharpy.Cli/Commands/ProjectCommand.cs +++ b/src/Sharpy.Cli/Commands/ProjectCommand.cs @@ -11,7 +11,7 @@ internal static class ProjectCommand { internal static void Configure(RootCommand root, GlobalOptions globals) { - var command = new Command("project", "Build a Sharpy project from a .spyproj file. Writes the assembly only; `compile` deploys its runtime dependencies and references beside the output"); + var command = new Command("project", "Build a Sharpy project from a .spyproj file. Copies its runtime dependencies and references beside the output, so it runs from there (--no-deps writes the assembly only)"); var projFileArg = new Argument("project") { Description = "Path to .spyproj file (auto-discovers if not specified)", Arity = ArgumentArity.ZeroOrOne }; var configOpt = new Option("--configuration") { Description = "Build configuration (Debug or Release)" }; @@ -21,10 +21,12 @@ internal static void Configure(RootCommand root, GlobalOptions globals) var incrementalOpt = new Option("--incremental") { Description = "Enable incremental compilation (only recompile changed files)" }; // `compile app.spyproj` takes these and `project` did not (a parse error), so the two project // commands diverged (#2223); both now merge them through CliHelpers.MergeCommandLineIntoProject. - var refOpt = new Option("--reference") { Description = "Add a .NET assembly reference (repeatable); not deployed beside the output — `compile` deploys" }; + var refOpt = new Option("--reference") { Description = "Add a .NET assembly reference (repeatable)" }; refOpt.Aliases.Add("-r"); var modPathOpt = new Option("--module-path") { Description = "Additional path to search for modules (repeatable)" }; modPathOpt.Aliases.Add("-m"); + // The spelling `compile` has always used (#2313 cell 0, R-HF c). + var noDepsOpt = new Option("--no-deps") { Description = "Skip copying runtime dependencies alongside the output" }; command.Arguments.Add(projFileArg); command.Options.Add(configOpt); @@ -33,6 +35,7 @@ internal static void Configure(RootCommand root, GlobalOptions globals) command.Options.Add(incrementalOpt); command.Options.Add(refOpt); command.Options.Add(modPathOpt); + command.Options.Add(noDepsOpt); command.SetAction((parseResult) => { @@ -52,6 +55,7 @@ internal static void Configure(RootCommand root, GlobalOptions globals) var features = parseResult.GetValue(globals.EnableFeature); var reference = parseResult.GetValue(refOpt) ?? Array.Empty(); var modulePath = parseResult.GetValue(modPathOpt) ?? Array.Empty(); + var noDeps = parseResult.GetValue(noDepsOpt); // The input-kind refusal comes before anything is written or validated: a .spy (Sharpy // source, not XML) or a directory used to reach the XML loader as "Unexpected error: @@ -68,7 +72,7 @@ internal static void Configure(RootCommand root, GlobalOptions globals) } var logger = CliHelpers.CreateLogger(logLevel, logFile); - return HandleProjectCommand(project, configuration, clean, incremental, emitCsTo, logger, logLevel, metricsFormat, metricsOutput, warnAsError, nowarn, maxErrors, features, reference, modulePath); + return HandleProjectCommand(project, configuration, clean, incremental, emitCsTo, logger, logLevel, metricsFormat, metricsOutput, warnAsError, nowarn, maxErrors, features, reference, modulePath, noDeps); }); root.Subcommands.Add(command); @@ -89,7 +93,8 @@ static int HandleProjectCommand( int? maxErrors = null, string[]? features = null, string[]? references = null, - string[]? modulePaths = null) + string[]? modulePaths = null, + bool noDeps = false) { FileInfo? resolvedProjectFile = projectFile; @@ -109,7 +114,7 @@ static int HandleProjectCommand( Console.WriteLine($"Building project: {Path.GetFileName(discoveredPath)}"); } - return CompileProject(resolvedProjectFile, configuration, clean, incremental, emitCsTo, logger, logLevel, metricsFormat, metricsOutput, warnAsError, nowarn, maxErrors, features, references, modulePaths); + return CompileProject(resolvedProjectFile, configuration, clean, incremental, emitCsTo, logger, logLevel, metricsFormat, metricsOutput, warnAsError, nowarn, maxErrors, features, references, modulePaths, noDeps); } /// @@ -151,7 +156,7 @@ static bool RestorePackages(ProjectConfig projectConfig, ICompilerLogger logger) return true; } - static int CompileProject(FileInfo projectFile, string configuration, bool clean, bool incremental, DirectoryInfo? emitCsTo, ICompilerLogger logger, CompilerLogLevel logLevel = CompilerLogLevel.None, string? metricsFormat = null, FileInfo? metricsOutput = null, bool warnAsError = false, string? nowarn = null, int? maxErrors = null, string[]? features = null, string[]? references = null, string[]? modulePaths = null) + static int CompileProject(FileInfo projectFile, string configuration, bool clean, bool incremental, DirectoryInfo? emitCsTo, ICompilerLogger logger, CompilerLogLevel logLevel = CompilerLogLevel.None, string? metricsFormat = null, FileInfo? metricsOutput = null, bool warnAsError = false, string? nowarn = null, int? maxErrors = null, string[]? features = null, string[]? references = null, string[]? modulePaths = null, bool noDeps = false) { try { @@ -189,9 +194,15 @@ static int CompileProject(FileInfo projectFile, string configuration, bool clean } Console.WriteLine(); + // The deployment seed's user half, captured before the default references join the list + // below: the .spyproj's own items plus the merged `-r` — the seed + // `compile app.spyproj` forms (#2223). + var userReferences = projectConfig.References.ToList(); + var defaultReferences = CliHelpers.GetDefaultReferences(); + var compilerOptions = CompilerOptionsFactory.ForProject( projectConfig, - defaultReferences: CliHelpers.GetDefaultReferences(), + defaultReferences: defaultReferences, warningsAsErrors: warnAsError, additionalSuppressedWarnings: CliHelpers.ParseNowarnCodes(nowarn), maxErrors: maxErrors ?? 0, @@ -200,7 +211,7 @@ static int CompileProject(FileInfo projectFile, string configuration, bool clean // Inject CLI default references into ProjectConfig so AssemblyCompiler // can resolve types from Sharpy.Core/Stdlib during Roslyn compilation - DefaultReferences.ApplyTo(projectConfig, CliHelpers.GetDefaultReferences()); + DefaultReferences.ApplyTo(projectConfig, defaultReferences); var compiler = new Sharpy.Compiler.Compiler(compilerOptions, logger); @@ -226,6 +237,17 @@ static int CompileProject(FileInfo projectFile, string configuration, bool clean return CliHelpers.MapFailureExitCode(errors); } + // `project` deploys like `compile app.spyproj` (#2313 cell 0, R-HF c): the output runs + // in place, as `dotnet build`'s does. The project result does not expose per-module used + // assemblies, so — as there — every default reference seeds the closure, plus every user + // reference. + if (result.OutputAssemblyPath != null && !noDeps) + { + var outputDir = Path.GetDirectoryName(result.OutputAssemblyPath) ?? Directory.GetCurrentDirectory(); + RuntimeDependencyHelper.CopyRuntimeDependencies( + outputDir, new HashSet(defaultReferences, StringComparer.OrdinalIgnoreCase), userReferences); + } + Console.WriteLine("Build succeeded."); Console.WriteLine($"Output: {result.OutputAssemblyPath}"); diff --git a/src/Sharpy.Cli/README.md b/src/Sharpy.Cli/README.md index b0958045a..008a3da43 100644 --- a/src/Sharpy.Cli/README.md +++ b/src/Sharpy.Cli/README.md @@ -35,6 +35,7 @@ sharpyc emit csharp # Generated C# code - `-o, --output ` - Output file path - `-r, --reference ` - Add .NET assembly references (a path, or a file name found on a module path). A reference that resolves nowhere is an error (SPY0305). With `compile app.spyproj` and `project`, added to the project's own `` items. Also taken by `emit diagnostics` and `emit hover`, so an entry importing a referenced library can be analysed. - `-p, --project-reference ` - Refused: sharpyc does not build .NET projects; build it with `dotnet build` and pass the assembly with `-r` +- `--no-deps` - Write the assembly only: skip copying `Sharpy.Core.dll`, the standard-library assemblies and the references beside the output (also taken by `compile` and `project`) - `-m, --module-path ` - Additional module search paths (a relative path is relative to the working directory). An absolute import is resolved below each one, on every command that takes it (`run`, `compile`, `build`, `project`, `emit csharp`, `emit diagnostics`, `emit hover`; lexing and parsing need no search path, so `emit tokens`/`ast`/`parse` do not): with `-m mods`, `from helper import f` finds `mods/helper.spy` and `import pkg.mod` finds `mods/pkg/mod.spy`. A directory that does not exist is an error. With `compile app.spyproj` and `project`, added to the project's own `` items. `compile` accepts both a `.spy` file and a `.spyproj`. Every option it accepts is either honoured, refused for that input kind, or documented in `--help` as ignored for it (`--type` for a `.spyproj`, `--incremental`/`--clean` for a `.spy` file; `--self-contained` is refused for a `.spyproj`) — its table is the `"compile"` table of `CommandOptionEffects`, the one option-to-effect registry for every command. @@ -46,6 +47,7 @@ sharpyc emit csharp # Generated C# code - `--clean` - Delete bin/ and obj/ directories before building - `--emit-cs-to ` - Save generated C# code to directory, mirroring each source's path relative to the project directory (`pkg/lib.spy` → `/pkg/lib.cs`) - `-r, --reference ` / `-m, --module-path ` - As for `compile app.spyproj`: added to the project's own ``/`` items; a reference that resolves nowhere is SPY0305 and a missing directory an error, before anything is built +- `--no-deps` - Write the assembly only, as for `build` and `compile` ### Options by Command @@ -53,10 +55,10 @@ Every option a command accepts — its own and the global ones — is honoured, | Command | Input | `-r` / `-m` | Deploys references | Global options without effect | |---|---|---|---|---| -| `build` | `.spy` | yes | no — writes the assembly only | none | +| `build` | `.spy` | yes | yes, unless `--no-deps` | none | | `compile` | `.spy`, `.spyproj` | yes | yes, unless `--no-deps` | none | | `run` | `.spy` | yes | yes, beside the program it runs | none | -| `project` | `.spyproj` | yes | no — writes the assembly only | none | +| `project` | `.spyproj` | yes | yes, unless `--no-deps` | none | | `emit tokens` | `.spy` | no — lexing needs no references | — | `--metrics-format`, `--metrics-output`, `--warn-as-error`, `--nowarn`, `--enable-feature` | | `emit ast` | `.spy` | no — parsing needs no references | — | `--metrics-format`, `--metrics-output`, `--warn-as-error`, `--nowarn`, `--enable-feature` | | `emit parse` | `.spy` | no — parsing needs no references | — | `--metrics-format`, `--metrics-output`, `--warn-as-error`, `--nowarn`, `--enable-feature` | @@ -176,8 +178,8 @@ Create a `.spyproj` file to define your multi-file Sharpy project: **Output Structure:** -`project` writes the assembly only, under `bin//net10.0/` in the `.spyproj`'s -directory: +`project` writes the assembly under `bin//net10.0/` in the `.spyproj`'s directory, +and deploys beside it what it needs to run: ``` bin/ @@ -187,25 +189,27 @@ bin/ MyApp.pdb # Debug only MyApp.deps.json MyApp.runtimeconfig.json + Sharpy.Core.dll + Sharpy.Stdlib.dll ... # the standard-library assemblies and their dependencies + Greeter.dll # each or -r assembly ``` -Nothing else is written beside it: not `Sharpy.Core.dll`, not the standard-library assemblies, and -not the project's `` assemblies. `dotnet bin/Debug/net10.0/MyApp.exe` therefore fails to -load them. `build` is the same for a single file: it writes the assembly only. - -`compile` and `run` deploy. Beside the assembly they copy `Sharpy.Core.dll`, the standard-library -assemblies the program uses (every default one for a `.spyproj`), and every `-r` or `` -assembly whether the program uses it or not. Each copied assembly brings the dependencies that sit -beside it, managed and native. The output directory then runs with `dotnet`: +Every command that produces an assembly deploys, as `dotnet build` does: `build`, `compile`, +`project` and `run` copy `Sharpy.Core.dll`, the standard-library assemblies the program uses (every +default one for a `.spyproj`), and every `-r` or `` assembly whether the program uses it +or not. Each copied assembly brings the dependencies that sit beside it, managed and native. The +output directory then runs with `dotnet`: ```bash -sharpyc compile app.spyproj -o out/MyApp.dll # or: sharpyc compile main.spy -r libs/Greeter.dll -o out/main.dll -dotnet out/MyApp.dll +sharpyc project app.spyproj # or: sharpyc build main.spy -r libs/Greeter.dll -o out/main.dll +dotnet bin/Debug/net10.0/MyApp.exe ``` -`compile --no-deps` skips the copy. `compile --self-contained` and `run --self-contained` publish -an executable that carries every reference. `compile` defaults to the `Release` configuration, so -without `-o` it writes `bin/Release/net10.0/` for a `.spyproj` and `bin/Release/` for a `.spy`. +`--no-deps` on `build`, `compile` or `project` skips the copy and writes the assembly only (with +its `.pdb`, `.deps.json` and `.runtimeconfig.json`). `compile --self-contained` and +`run --self-contained` publish an executable that carries every reference. `compile` defaults to +the `Release` configuration, so without `-o` it writes `bin/Release/net10.0/` for a `.spyproj` and +`bin/Release/` for a `.spy`; `build` without `-o` writes to the working directory. ## Implementation Status diff --git a/src/Sharpy.Cli/RuntimeDependencyHelper.cs b/src/Sharpy.Cli/RuntimeDependencyHelper.cs index 45da28214..bf0017800 100644 --- a/src/Sharpy.Cli/RuntimeDependencyHelper.cs +++ b/src/Sharpy.Cli/RuntimeDependencyHelper.cs @@ -7,8 +7,9 @@ namespace Sharpy.Cli; /// /// Copies the runtime dependencies (Sharpy.Core.dll, used stdlib assemblies, every explicit user /// reference, and their transitive dependencies — managed assemblies and native assets alike) needed to -/// execute a compiled Sharpy assembly via dotnet output.dll. Shared by the run -/// and compile commands. +/// execute a compiled Sharpy assembly via dotnet output.dll. The one deployment seam of every +/// producing command — run, compile, build and project (the last two since +/// #2313 cell 0, R-HF c) — each of which but run skips it under --no-deps. /// /// /// The dependency set is computed mechanically by from the diff --git a/src/Sharpy.Cli/Services/CompileServerClient.cs b/src/Sharpy.Cli/Services/CompileServerClient.cs index e03a9f842..8c3fa5640 100644 --- a/src/Sharpy.Cli/Services/CompileServerClient.cs +++ b/src/Sharpy.Cli/Services/CompileServerClient.cs @@ -20,10 +20,14 @@ internal static class CompileServerClient /// code on success (having written the server's captured stdout/stderr to the console), or /// null if no usable server responded — signalling the caller to compile in-process. The /// fallback reason is written to so the choice is visible. + /// is the compile's runtime-dependency set, which the + /// caller deploys beside the output (empty when no server answered). /// public static int? TryRun( - string pipeName, CompileServerRequest request, TextWriter stdout, TextWriter stderr, bool verbose = false) + string pipeName, CompileServerRequest request, TextWriter stdout, TextWriter stderr, + out IReadOnlySet usedAssemblyPaths, bool verbose = false) { + usedAssemblyPaths = new HashSet(); if (!TrySend(pipeName, request, out var response, out var failureReason) || response == null) { stderr.WriteLine($"sharpyc: no compile server on pipe '{pipeName}' ({failureReason}); compiling in-process."); @@ -42,6 +46,7 @@ internal static class CompileServerClient if (verbose) WriteVerboseSummary(stderr, pipeName, response); + usedAssemblyPaths = new HashSet(response.UsedAssemblyPaths, StringComparer.OrdinalIgnoreCase); return response.ExitCode; } From 64ff07e975b1a8b6b62e157e28096d96e1659de4 Mon Sep 17 00:00:00 2001 From: antonsynd <2648730+antonsynd@users.noreply.github.com> Date: Thu, 22 Oct 2026 02:05:15 -0400 Subject: [PATCH 06/13] =?UTF-8?q?fix(semantic):=20a=20numeric=20literal=20?= =?UTF-8?q?outside=20its=20own=20type's=20range=20is=20SPY0210=20at=20one?= =?UTF-8?q?=20seam=20=E2=80=94=201e400,=201e39f=20and=201e29m=20never=20re?= =?UTF-8?q?ach=20Roslyn=20or=20the=20emitter=20(refs=20#2340)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit R-HU (a): the float half of the literal-range seam. FloatLiteralClassifier (Shared/, twin of IntegerLiteralClassifier) types a float literal by suffix and asks the ONE range question, Fits(text, double|float|decimal); the literal-typing arm of CheckExpression refuses a miss with SPY0210 ("Float literal '1e400' is too large for 'double'" / 'float' / 'decimal'), and the float32/decimal slot-narrowing predicates (ImplicitConversions) ask the same Fits, so the suffixed and the slot-narrowed spelling of one value cannot disagree. float32 is now parsed as a float directly (Roslyn's own verdict on the emitted `...f` text), not routed through double. SPY0210's constant is renamed NumericLiteralOutOfRange; its DiagnosticExplanations entry names the float types. float_literals.md gains "A Literal Must Fit Its Own Type" beside the float32 limit, and the decimal slot limit (`d: decimal = 1e29` SPY0220). The theory's negative x unsigned cells were not range cells: CheckUnaryOp's negated-literal arm typed `-1U` 'uint32' (C#: long) and handed `-1UL` to C# (CS0023), both SPY0908. Only an unsuffixed/L literal now negates as a literal (IntegerLiteralClassifier.NegatesAsLiteral); `-1U` / `-1UL` take the unary operator path, int64 and SPY0223, exactly as `-v` on a uint32/uint64 variable. Matrix (NumericLiteralRangeMatrixTests, 84 rows): type {int32->int64 promotion, int64, uint32, uint64, float64, float32, decimal} x spelling {unsuffixed, suffixed, slot-narrowed} x magnitude {max, max + one unit/ulp, far, negative}; axis changes POSITION {pattern, parameter default, const, field, lambda body, f-string} and CASE {1E400, F, M}; plus a Roslyn differential (13 texts: the seam admits exactly what CS0594 admits). Mutation (Rule 12): - float64 arm of Fits -> `return true`: 14 red (8 float64 cells, the 2 PastFloat64 slot cells, 4 positions, 2 Roslyn rows; 70 green) -> restored 84/0/0. - NegatesAsLiteral -> `true` + float32 narrowing double-routed again: 3 red (UInt64_Suffixed_Negative, UInt32_Suffixed_Negative_IntoUInt32Slot, Float32_SlotNarrowed_BelowMidpoint; 81 green) -> restored 84/0/0. Direction (run, base e7e0a0ef7 -> this commit): - SPY0599 (CS0594) -> SPY0210: print(1e400), print(-1e400), print(1e400d), print(1.7976931348623159e308), x: float = 1e400, print(1e39f), print(-1e39f), print(3.4028236e38f), case 1e400:, def f(x: float = 1e400), const C: float = 1e400. - SPY0909 ICE -> SPY0210: print(1e29m), print(-1e29m), print(79228162514264337593543950336m), class field `v: decimal = 1e29m`. - SPY0908 -> SPY0223: print(-1UL). SPY0908 -> SPY0220: x: uint32 = -1U. - SPY0220 -> SPY0210 + SPY0220 (still refused, cause named): x: float32 = 1e400, d: decimal = 1e400. - SPY0220 -> runs (a false refusal cured; the suffixed spelling already ran): x: float32 = 340282356779733661637539395458142568447.0 prints 3.4028235e+38 — below the float32 rounding midpoint, double-rounded to inf by the old predicate. - Unchanged: every in-range cell, print(-1U) -> -1, print(1e-400) -> 0.0, print(1e-30m) -> 0.0000000000000000000000000000, all integer refusals. No program that ran changes its output. No fixture or snapshot moves. Filtered @ this tree (measured): Compiler.Tests NumericLiteralRangeMatrix | TypeChecker | Semantic | Conformance | Census | Roster | EmitterCarrier | ProcessWide | FileBasedIntegrationTests 32311/0/1; Lsp.Tests FrontEndParity 1/0/0; Stdlib.Tests 4273/0/0; build 0 warnings; check_spec_blocks.sh clean (1207 blocks, failing 0); the new spec block run: prints the three maxima. Co-Authored-By: Claude Opus 5.5 (1M context) Claude-Session: https://claude.ai/code/session_0126pNZqmMsSbn4WFFPhpnxW --- docs/language_specification/float_literals.md | 25 ++ .../NumericLiteralRangeMatrixTests.cs | 229 ++++++++++++++++++ .../Diagnostics/DiagnosticCodes.cs | 7 +- .../DiagnosticExplanations.Semantic.cs | 8 +- .../Semantic/ImplicitConversions.cs | 37 +-- .../TypeChecker.Expressions.Operators.cs | 11 +- .../Semantic/TypeChecker.Expressions.cs | 28 ++- .../Shared/FloatLiteralClassifier.cs | 80 ++++++ .../Shared/IntegerLiteralClassifier.cs | 10 + 9 files changed, 394 insertions(+), 41 deletions(-) create mode 100644 src/Sharpy.Compiler.Tests/Semantic/NumericLiteralRangeMatrixTests.cs create mode 100644 src/Sharpy.Compiler/Shared/FloatLiteralClassifier.cs diff --git a/docs/language_specification/float_literals.md b/docs/language_specification/float_literals.md index aea027943..facbbd31b 100644 --- a/docs/language_specification/float_literals.md +++ b/docs/language_specification/float_literals.md @@ -80,9 +80,34 @@ Two limits: - Only a **literal** narrows. A `float64`-typed *expression* or variable still requires an explicit cast: `x: float32 = y` is SPY0220 for `y: float`. - The value must be within `float32`'s range. `x: float32 = 1e40` is SPY0220, not a silent infinity. + The same holds for `decimal`, whose range is about ±7.9e28: `d: decimal = 1e29` is SPY0220. This is a deliberate divergence from C#, which rejects `float f = 0.1;` outright. Precision loss within range is accepted — writing the annotation is taken as asking for it. +## A Literal Must Fit Its Own Type + +Whatever its spelling, a float literal must have a finite value in the type it resolves to — +`float64` unsuffixed, `float32` with `f`, `decimal` with `m`. A literal outside that range is +refused at compile time with SPY0210, the code an out-of-range integer literal draws, and never +carried as an infinity. Python reads `1e400` as `inf`; Sharpy refuses it, because a silent infinity +would flow on through arithmetic. + +```python +def main() -> None: + big = 1.7976931348623157e308 # float64's largest value + print(big) # 1.7976931348623157e+308 + f = 3.4028235e38f # float32's largest value + print(f) # 3.4028235e+38 + m = 79228162514264337593543950335m # decimal's largest value + print(m) # 79228162514264337593543950335 + # x = 1e400 # ERROR (SPY0210): Float literal '1e400' is too large for 'double' + # y = 1e39f # ERROR (SPY0210): Float literal '1e39' is too large for 'float' + # z = 1e29m # ERROR (SPY0210): Float literal '1e29' is too large for 'decimal' +``` + +A negated literal is checked by its digits, so `-1e400` is refused the same way. A literal too +small to represent is not out of range: `1e-400` is `0.0`, as in Python. + *Implementation* - *✅ Native - Direct mapping to C# float literals with compiler-inserted suffixes.* diff --git a/src/Sharpy.Compiler.Tests/Semantic/NumericLiteralRangeMatrixTests.cs b/src/Sharpy.Compiler.Tests/Semantic/NumericLiteralRangeMatrixTests.cs new file mode 100644 index 000000000..447ccff8a --- /dev/null +++ b/src/Sharpy.Compiler.Tests/Semantic/NumericLiteralRangeMatrixTests.cs @@ -0,0 +1,229 @@ +using FluentAssertions; +using Microsoft.CodeAnalysis; +using Microsoft.CodeAnalysis.CSharp; +using Sharpy.Compiler.Diagnostics; +using Sharpy.Compiler.Shared; +using Sharpy.TestInfrastructure.Integration; +using Xunit; +using Xunit.Abstractions; + +namespace Sharpy.Compiler.Tests.Semantic; + +/// +/// The numeric-literal range matrix (#2340, rulings R-HU / R-HV). +/// +/// Contract. A numeric literal outside the range of its OWN resolved type is refused +/// at one semantic seam with the SPY0210 family — never by Roslyn (CS0594 → SPY0599) and never by +/// the emitter (SPY0909). The integer half of the seam is ; +/// the float half is , whose Fits is also the range +/// question the float32/decimal slot-narrowing predicates ask, so the suffixed and the +/// slot-narrowed spelling of one value cannot disagree. Before #2340 only the integer arm and +/// the narrowing arm existed: 1e400 and 1e39f were SPY0599, 1e29m SPY0909. +/// +/// Axes (closed). TYPE {int32→int64 promotion, int64, uint32, uint64, float64, +/// float32, decimal} × SPELLING {unsuffixed, suffixed, slot-narrowed} × MAGNITUDE {max, max + one +/// unit/ulp, far, negative}. A negative UNSIGNED literal is not a range cell: -1U is the +/// unary operator over a uint32 operand, typed int64 as C# types it, and +/// -1UL is SPY0223 as -u is for a uint64 variable. Two axis changes ride +/// along: POSITION (pattern, parameter default, const, field, lambda body, f-string) and CASE +/// (1E400, F, M). Spellings avoid the forms #2339 re-reads in the lexer. +/// +/// Every cell executes. An accepted cell prints its value; a refused cell asserts +/// the code, that the message names the type, and that no ICE code rides along. +/// is the differential half: the seam's answer +/// for each float text is Roslyn's own CS0594 verdict on the text the emitter writes. +/// +public class NumericLiteralRangeMatrixTests : IntegrationTestBase +{ + public NumericLiteralRangeMatrixTests(ITestOutputHelper output) : base(output) { } + + private const string Refused0210 = "SPY0210"; + + private static readonly string[] IceCodes = { "SPY0599", "SPY0908", "SPY0909" }; + + // ── Accepted cells: the program runs and prints the literal's value ───────────────────── + + [Theory] + // int32 → int64 promotion (unsuffixed) + [InlineData("Promoted_Unsuffixed_Max", "print(9223372036854775807)", "9223372036854775807")] + [InlineData("Promoted_Unsuffixed_Int32MaxPlusOne", "print(2147483648)", "2147483648")] + [InlineData("Promoted_Unsuffixed_NegativeMin", "print(-9223372036854775808)", "-9223372036854775808")] + [InlineData("Promoted_SlotNarrowed_Int64", "x: int64 = 2147483648\n print(x)", "2147483648")] + [InlineData("Int32_SlotNarrowed_Max", "x: int32 = 2147483647\n print(x)", "2147483647")] + // int64 + [InlineData("Int64_Suffixed_Max", "print(9223372036854775807L)", "9223372036854775807")] + [InlineData("Int64_Suffixed_NegativeMin", "print(-9223372036854775808L)", "-9223372036854775808")] + // uint32 + [InlineData("UInt32_Suffixed_Max", "print(4294967295U)", "4294967295")] + // An unsuffixed literal above int32 is int64, and an int64 constant narrows only into uint64 + // (integer_literals.md), so the widest unsuffixed constant a uint32 slot admits is int32's max. + [InlineData("UInt32_SlotNarrowed_Int32Max", "x: uint32 = 2147483647\n print(x)", "2147483647")] + [InlineData("UInt32_Suffixed_Negative_IsInt64", "y = -1U\n z: int64 = y\n print(z)", "-1")] + [InlineData("UInt32_Suffixed_NegativeMax_IsInt64", "print(-4294967295U)", "-4294967295")] + // uint64 + [InlineData("UInt64_Suffixed_Max", "print(18446744073709551615UL)", "18446744073709551615")] + [InlineData("UInt64_SlotNarrowed_Int64Max", "x: uint64 = 9223372036854775807\n print(x)", "9223372036854775807")] + // float64 + [InlineData("Float64_Unsuffixed_Max", "print(1.7976931348623157e308)", "1.7976931348623157e+308")] + [InlineData("Float64_Unsuffixed_NegativeMax", "print(-1.7976931348623157e308)", "-1.7976931348623157e+308")] + [InlineData("Float64_Suffixed_Max", "print(1.7976931348623157e308d)", "1.7976931348623157e+308")] + [InlineData("Float64_SlotNarrowed_Max", "x: float = 1.7976931348623157e308\n print(x)", "1.7976931348623157e+308")] + [InlineData("Float64_Unsuffixed_Underflow_IsZero", "print(1e-400)", "0.0")] + // float32 + [InlineData("Float32_Suffixed_Max", "print(3.4028235e38f)", "3.4028235e+38")] + [InlineData("Float32_Suffixed_NegativeMax", "print(-3.4028235e38f)", "-3.4028235e+38")] + [InlineData("Float32_SlotNarrowed_Max", "x: float32 = 3.4028235e38\n print(x)", "3.4028235e+38")] + // Below the float32 rounding midpoint 2^128 - 2^103 by one: a correctly-rounded float parse is + // float.MaxValue (Roslyn's verdict on the emitted `...447.0f`), but routed through double it + // first rounds UP to the midpoint and then ties to infinity — the narrowing predicate's old + // double-routed question refused a literal the suffixed spelling of the same digits ran. + [InlineData("Float32_SlotNarrowed_BelowMidpoint", + "x: float32 = 340282356779733661637539395458142568447.0\n print(x)", "3.4028235e+38")] + [InlineData("Float32_Suffixed_BelowMidpoint", + "print(340282356779733661637539395458142568447.0f)", "3.4028235e+38")] + [InlineData("Float32_Suffixed_Small", "print(0.5f)", "0.5")] + // decimal + [InlineData("Decimal_Suffixed_Max", "print(79228162514264337593543950335m)", "79228162514264337593543950335")] + [InlineData("Decimal_Suffixed_NegativeMax", "print(-79228162514264337593543950335m)", "-79228162514264337593543950335")] + [InlineData("Decimal_Suffixed_Small", "print(0.5m)", "0.5")] + [InlineData("Decimal_SlotNarrowed_Exponent", "d: decimal = 1.5e2\n print(d)", "150")] + public void AcceptedCell_RunsAndPrintsTheValue(string cell, string body, string expected) + { + var source = InMain(body); + var result = CompileAndExecute(source); + + result.Success.Should().BeTrue( + $"[{cell}] must compile and run. Diagnostics: {Describe(result)}\n{source}"); + result.StandardOutput.TrimEnd('\n', '\r').Should().Be(expected, $"[{cell}]\n{source}"); + } + + // ── Refused cells: the code, the named type, no ICE ────────────────────────────────────── + + [Theory] + // int32 → int64 promotion (unsuffixed) + [InlineData("Promoted_Unsuffixed_MaxPlusOne", "print(9223372036854775808)", Refused0210, "'long'")] + [InlineData("Promoted_Unsuffixed_Far", "print(99999999999999999999)", Refused0210, "any integer type")] + [InlineData("Promoted_Unsuffixed_NegativePastMin", "print(-9223372036854775809)", Refused0210, "any integer type")] + [InlineData("Int32_SlotNarrowed_MaxPlusOne", "x: int32 = 2147483648", "SPY0220", "'int32'")] + // int64 + [InlineData("Int64_Suffixed_MaxPlusOne", "print(9223372036854775808L)", Refused0210, "'long'")] + [InlineData("Int64_Suffixed_Far", "print(99999999999999999999L)", Refused0210, "'long'")] + [InlineData("Int64_Suffixed_NegativePastMin", "print(-9223372036854775809L)", Refused0210, "'long'")] + [InlineData("Int64_SlotNarrowed_MaxPlusOne", "x: int64 = 9223372036854775808", Refused0210, "'long'")] + // uint32 + [InlineData("UInt32_Suffixed_MaxPlusOne", "print(4294967296U)", Refused0210, "'uint'")] + [InlineData("UInt32_Suffixed_Far", "print(99999999999999999999U)", Refused0210, "'uint'")] + [InlineData("UInt32_Suffixed_NegativeFar", "print(-4294967296U)", Refused0210, "'uint'")] + [InlineData("UInt32_Suffixed_Negative_IntoUInt32Slot", "x: uint32 = -1U", "SPY0220", "'int64'")] + [InlineData("UInt32_SlotNarrowed_MaxPlusOne", "x: uint32 = 4294967296", "SPY0220", "'uint32'")] + // uint64 + [InlineData("UInt64_Suffixed_MaxPlusOne", "print(18446744073709551616UL)", Refused0210, "'ulong'")] + [InlineData("UInt64_Suffixed_Far", "print(99999999999999999999UL)", Refused0210, "'ulong'")] + [InlineData("UInt64_Suffixed_Negative", "print(-1UL)", "SPY0223", "'uint64'")] + [InlineData("UInt64_SlotNarrowed_MaxPlusOne", "x: uint64 = 18446744073709551616", Refused0210, "any integer type")] + // float64 + [InlineData("Float64_Unsuffixed_MaxPlusOneUlp", "print(1.7976931348623159e308)", Refused0210, "'double'")] + [InlineData("Float64_Unsuffixed_Far", "print(1e400)", Refused0210, "'double'")] + [InlineData("Float64_Unsuffixed_Negative", "print(-1e400)", Refused0210, "'double'")] + [InlineData("Float64_Suffixed_Far", "print(1e400d)", Refused0210, "'double'")] + [InlineData("Float64_SlotNarrowed_Far", "x: float = 1e400", Refused0210, "'double'")] + // float32 + [InlineData("Float32_Suffixed_MaxPlusOneUlp", "print(3.4028236e38f)", Refused0210, "'float'")] + [InlineData("Float32_Suffixed_Far", "print(1e39f)", Refused0210, "'float'")] + [InlineData("Float32_Suffixed_Negative", "print(-1e39f)", Refused0210, "'float'")] + [InlineData("Float32_SlotNarrowed_MaxPlusOneUlp", "x: float32 = 3.4028236e38", "SPY0220", "'float32'")] + [InlineData("Float32_SlotNarrowed_Far", "x: float32 = 1e40", "SPY0220", "'float32'")] + [InlineData("Float32_SlotNarrowed_PastFloat64", "x: float32 = 1e400", Refused0210, "'double'")] + // decimal + [InlineData("Decimal_Suffixed_MaxPlusOne", "print(79228162514264337593543950336m)", Refused0210, "'decimal'")] + [InlineData("Decimal_Suffixed_Far", "print(1e29m)", Refused0210, "'decimal'")] + [InlineData("Decimal_Suffixed_Negative", "print(-1e29m)", Refused0210, "'decimal'")] + [InlineData("Decimal_SlotNarrowed_MaxPlusOne", "d: decimal = 79228162514264337593543950336.0", "SPY0220", "'decimal'")] + [InlineData("Decimal_SlotNarrowed_Far", "d: decimal = 1e29", "SPY0220", "'decimal'")] + [InlineData("Decimal_SlotNarrowed_PastFloat64", "d: decimal = 1e400", Refused0210, "'double'")] + // Axis change: case of the exponent and suffix letters. + [InlineData("Case_Float64_UpperExponent", "print(1E400)", Refused0210, "'double'")] + [InlineData("Case_Float32_UpperSuffix", "print(1e39F)", Refused0210, "'float'")] + [InlineData("Case_Decimal_UpperSuffix", "print(1e29M)", Refused0210, "'decimal'")] + public void RefusedCell_CarriesTheCodeAndNoIce(string cell, string body, string code, string typeNamed) + { + AssertRefused(cell, InMain(body), code, typeNamed); + } + + // ── Axis change: the literal's POSITION — one seam, so every position refuses ──────────── + + [Theory] + [InlineData("Position_Pattern", + "def f(x: float) -> None:\n match x:\n case 1e400:\n print(\"a\")\n case _:\n print(\"b\")\n\ndef main() -> None:\n f(1.0)\n", + "'double'")] + [InlineData("Position_ParameterDefault", + "def f(x: float = 1e400) -> None:\n print(x)\n\ndef main() -> None:\n f()\n", + "'double'")] + [InlineData("Position_Const", + "const C: float = 1e400\n\ndef main() -> None:\n print(C)\n", + "'double'")] + [InlineData("Position_Field", + "class K:\n v: decimal = 1e29m\n\ndef main() -> None:\n print(K().v)\n", + "'decimal'")] + [InlineData("Position_LambdaBody", + "def main() -> None:\n g = lambda x: x * 1e39f\n print(g(1.0f))\n", + "'float'")] + [InlineData("Position_FString", + "def main() -> None:\n print(f\"{1e400}\")\n", + "'double'")] + public void RefusedPosition_CarriesSpy0210AndNoIce(string cell, string source, string typeNamed) + { + AssertRefused(cell, source, Refused0210, typeNamed); + } + + // ── Differential: the seam's answer is Roslyn's own verdict on the emitted text ────────── + + [Theory] + [InlineData("1.7976931348623157e308", "d")] + [InlineData("1.7976931348623159e308", "d")] + [InlineData("1e400", "d")] + [InlineData("1e-400", "d")] + [InlineData("3.4028235e38", "f")] + [InlineData("3.4028236e38", "f")] + [InlineData("340282356779733661637539395458142568447.0", "f")] + [InlineData("340282356779733661637539395458142568448.0", "f")] + [InlineData("1e39", "f")] + [InlineData("79228162514264337593543950335", "m")] + [InlineData("79228162514264337593543950336", "m")] + [InlineData("1e29", "m")] + [InlineData("1.5e2", "m")] + public void FloatSeam_AgreesWithRoslynsLexer(string digits, string suffix) + { + var clrType = suffix switch { "f" => typeof(float), "m" => typeof(decimal), _ => typeof(double) }; + var seamAdmits = FloatLiteralClassifier.Fits(digits, clrType); + + var tree = CSharpSyntaxTree.ParseText($"class C {{ object V = {digits}{suffix}; }}"); + var roslynRefuses = tree.GetDiagnostics().Any(d => d.Id == "CS0594"); + + seamAdmits.Should().Be(!roslynRefuses, + $"'{digits}{suffix}': the seam must admit exactly the texts Roslyn's lexer admits " + + $"(Roslyn: {string.Join(", ", tree.GetDiagnostics().Select(d => d.Id))})"); + } + + private void AssertRefused(string cell, string source, string code, string typeNamed) + { + var result = CompileAndExecute(source); + + result.Success.Should().BeFalse($"[{cell}] must be refused; it printed '{result.StandardOutput}'\n{source}"); + + var matching = result.RawDiagnostics.Where(d => d.Code == code).ToList(); + matching.Should().HaveCount(1, $"[{cell}] must report {code} exactly once. Got: {Describe(result)}\n{source}"); + matching[0].Message.Should().Contain(typeNamed, $"[{cell}] the refusal names the type\n{source}"); + + result.RawDiagnostics.Should().NotContain(d => IceCodes.Contains(d.Code), + $"[{cell}] the semantic seam refuses first — no ICE rides along. Got: {Describe(result)}"); + result.CompilationErrors.Should().NotContain(e => IceCodes.Any(e.Contains), + $"[{cell}] no ICE in the error text either\n{source}"); + } + + private static string InMain(string body) => $"def main() -> None:\n {body}\n"; + + private static string Describe(ExecutionResult result) + => string.Join(" | ", result.RawDiagnostics.Select(d => $"{d.Code}@{d.Line}: {d.Message}")) + + (result.CompilationErrors.Count > 0 ? " || " + string.Join(" | ", result.CompilationErrors) : ""); +} diff --git a/src/Sharpy.Compiler/Diagnostics/DiagnosticCodes.cs b/src/Sharpy.Compiler/Diagnostics/DiagnosticCodes.cs index 45d141504..d3464cbe5 100644 --- a/src/Sharpy.Compiler/Diagnostics/DiagnosticCodes.cs +++ b/src/Sharpy.Compiler/Diagnostics/DiagnosticCodes.cs @@ -267,9 +267,10 @@ public static class Semantic public const string InvalidTypeAlias = "SPY0209"; // Active // Question mark operator in finally (SPY0211) public const string QuestionMarkInFinally = "SPY0211"; // Active - // SPY0210: Integer literal out of range for its type (#1314). Never previously issued — - // established via full-history pickaxe (3 comment-only commits: e98b3d7c4, 4738b9265, 5b40df995). - public const string IntegerLiteralOutOfRange = "SPY0210"; // Active + // SPY0210: Numeric literal out of range for its own type. Integer literals since #1314 + // (never previously issued — established via full-history pickaxe, 3 comment-only commits: + // e98b3d7c4, 4738b9265, 5b40df995); float, float32 and decimal literals since #2340 (R-HU). + public const string NumericLiteralOutOfRange = "SPY0210"; // Active // Claims the SPY0212 reservation made for round-8 batch D. The bare spelling of a builtin // name always denotes the builtin — types and functions uniformly — so a user symbol with // that spelling must be backtick-escaped. Extends the rule hard keywords already follow diff --git a/src/Sharpy.Compiler/Diagnostics/DiagnosticExplanations.Semantic.cs b/src/Sharpy.Compiler/Diagnostics/DiagnosticExplanations.Semantic.cs index af456dd6c..0e00ef9ec 100644 --- a/src/Sharpy.Compiler/Diagnostics/DiagnosticExplanations.Semantic.cs +++ b/src/Sharpy.Compiler/Diagnostics/DiagnosticExplanations.Semantic.cs @@ -50,10 +50,10 @@ private static void AddSemanticEntries(Dictionary "def process() -> int !str:\n try:\n pass\n finally:\n val: int = get_value()? # error", "Move the '?' expression outside the finally block:\ndef process() -> int !str:\n val: int = get_value()?\n try:\n pass\n finally:\n cleanup()"); - Add(dict, DiagnosticCodes.Semantic.IntegerLiteralOutOfRange, "Integer literal out of range", "Semantic", - "An integer literal exceeds the range of any Sharpy integer type (int, long, or unsigned variants). Sharpy integer types are fixed-width: int is 32-bit and long is 64-bit. Literals that exceed these ranges are caught at compile time.", - "x = 99999999999999999999999 # too large for any integer type", - "Use a smaller value, break the computation into parts, or use a float/double if approximate values are acceptable."); + Add(dict, DiagnosticCodes.Semantic.NumericLiteralOutOfRange, "Numeric literal out of range", "Semantic", + "A numeric literal exceeds the range of its own type. Sharpy numeric types are fixed-width: int is 32-bit and long is 64-bit, an unsuffixed integer literal is int or, when too large for int, int64, and a suffix (L, U, UL) names the type outright. A float literal is float64, or float32 with 'f', or decimal with 'm'; its value must be finite in that type — float64 tops out near 1.8e308, float32 near 3.4e38 and decimal near 7.9e28. A literal outside its type's range is refused at compile time, never carried as an overflowed or infinite value (Python reads 1e400 as inf; Sharpy refuses it).", + "x = 99999999999999999999999 # too large for any integer type\ny = 1e400 # too large for 'double'\nz = 1e29m # too large for 'decimal'", + "Use a value inside the type's range, or a wider type: the 'UL' suffix for an integer up to 18446744073709551615, an unsuffixed float64 literal instead of 'f' or 'm' for a large magnitude, or float('inf') when infinity is the intent."); Add(dict, DiagnosticCodes.Semantic.BuiltinNameShadowed, "Builtin name shadowed", "Semantic", "A TYPE declaration used the bare spelling of a builtin type name. A type declaration enters the type namespace, so after it two different things answer to that name in annotation position and 'x: double' becomes genuinely ambiguous — Sharpy resolves annotations statically, so it cannot leave that unresolved. This extends the rule hard keywords already follow: bare 'class' is the keyword, '`class`' is your symbol. Only type declarations are affected. Bindings in VALUE position — variables, parameters, for-targets, and function declarations — may shadow a builtin freely, because they never enter the type namespace and so cannot make an annotation ambiguous; those draw a warning instead. Python differs here — CPython allows the rebinding, because annotations are not a static resolution surface there.", diff --git a/src/Sharpy.Compiler/Semantic/ImplicitConversions.cs b/src/Sharpy.Compiler/Semantic/ImplicitConversions.cs index 6527aef51..77b419f49 100644 --- a/src/Sharpy.Compiler/Semantic/ImplicitConversions.cs +++ b/src/Sharpy.Compiler/Semantic/ImplicitConversions.cs @@ -77,15 +77,11 @@ public static bool IsFloat32LiteralNarrowing( || PrimitiveCatalog.GetPrimitiveInfo(initType)?.ClrType != typeof(double)) return false; - // Non-throwing and range-checked in one step: a literal above float32's maximum becomes - // Infinity when narrowed, which is the refusal (`1e40` is SPY0220, never a crash), and an - // exponent form that fits (`1.5e2`) narrows exactly as a plain one does. - return double.TryParse( - literal.Value, - NumberStyles.Float, - CultureInfo.InvariantCulture, - out var value) - && !double.IsInfinity((double)(float)value); + // The range question is the literal-typing arm's own (#2340): one seam answers "does this + // text have a finite float32 value" for `1e39f` and for `x: float32 = 1e39` alike. A miss + // is the refusal (`1e40` is SPY0220, never a crash), and an exponent form that fits + // (`1.5e2`) narrows exactly as a plain one does. + return Shared.FloatLiteralClassifier.Fits(literal.Value, typeof(float)); } /// @@ -118,25 +114,18 @@ public static bool IsDecimalLiteralNarrowing( || PrimitiveCatalog.GetPrimitiveInfo(initType)?.ClrType != typeof(double)) return false; - if (!double.TryParse( - literal.Value, - NumberStyles.Float, - CultureInfo.InvariantCulture, - out var asDouble) - || !double.IsFinite(asDouble)) - { - return false; - } - - if (!decimal.TryParse( - literal.Value, - NumberStyles.Float, - CultureInfo.InvariantCulture, - out var asDecimal)) + // FORM and RANGE are the literal-typing arm's own question (#2340): one seam answers + // "does this text have a decimal value" for `1e29m` and for `d: decimal = 1e29` alike. + if (!Shared.FloatLiteralClassifier.Fits(literal.Value, typeof(double)) + || !Shared.FloatLiteralClassifier.Fits(literal.Value, typeof(decimal))) { return false; } + // UNDERFLOW stays a narrowing-only question: an unsuffixed literal is a float64, and + // re-typing a non-zero one to a decimal zero would change its value. + var asDouble = double.Parse(literal.Value, NumberStyles.Float, CultureInfo.InvariantCulture); + var asDecimal = decimal.Parse(literal.Value, NumberStyles.Float, CultureInfo.InvariantCulture); return asDecimal != 0m || asDouble == 0.0; } } diff --git a/src/Sharpy.Compiler/Semantic/TypeChecker.Expressions.Operators.cs b/src/Sharpy.Compiler/Semantic/TypeChecker.Expressions.Operators.cs index 5c4f976e7..57342dcf1 100644 --- a/src/Sharpy.Compiler/Semantic/TypeChecker.Expressions.Operators.cs +++ b/src/Sharpy.Compiler/Semantic/TypeChecker.Expressions.Operators.cs @@ -1256,15 +1256,20 @@ private SemanticType CheckPipeForwardWithFunctionCall(SemanticType pipedType, Fu private SemanticType CheckUnaryOp(UnaryOp unOp) { // Constant unary minus over an integer literal: classify by the negated magnitude - // so -2147483648 is int and -9223372036854775808 is long (#1304). - if (unOp.Operator == UnaryOperator.Minus && unOp.Operand is IntegerLiteral il) + // so -2147483648 is int and -9223372036854775808 is long (#1304). Only a SIGNED literal + // negates as a literal; `-1U` / `-1UL` are the unary operator over an unsigned operand and + // take the operator path below, exactly as `-v` does for a `uint32`/`uint64` variable — + // int64 and SPY0223, C#'s own answers (#2340: this arm typed `-1U` 'uint32' while C# typed + // it 'long', and passed `-1UL` to C# as CS0023, both SPY0908). + if (unOp.Operator == UnaryOperator.Minus && unOp.Operand is IntegerLiteral il + && Shared.IntegerLiteralClassifier.NegatesAsLiteral(il.Suffix)) { var result = Shared.IntegerLiteralClassifier.ClassifyNegated(il.Value, il.Suffix); if (result.IsError) { AddError(result.ErrorMessage!, unOp.LineStart, unOp.ColumnStart, - code: DiagnosticCodes.Semantic.IntegerLiteralOutOfRange, + code: DiagnosticCodes.Semantic.NumericLiteralOutOfRange, span: unOp.Span); } else if (result.Type == SemanticType.Int) diff --git a/src/Sharpy.Compiler/Semantic/TypeChecker.Expressions.cs b/src/Sharpy.Compiler/Semantic/TypeChecker.Expressions.cs index aee2f91f2..e6f283494 100644 --- a/src/Sharpy.Compiler/Semantic/TypeChecker.Expressions.cs +++ b/src/Sharpy.Compiler/Semantic/TypeChecker.Expressions.cs @@ -30,12 +30,7 @@ public SemanticType CheckExpression(Expression expr) SemanticType type = expr switch { IntegerLiteral il => ClassifyIntegerLiteral(il), - FloatLiteral fl => fl.Suffix?.ToUpperInvariant() switch - { - "F" => SemanticType.Float32, - "M" => SemanticType.Decimal, - _ => SemanticType.Double, - }, + FloatLiteral fl => ClassifyFloatLiteral(fl), StringLiteral sl => CheckStringLiteral(sl), BytesLiteralExpression bytesLit => CheckBytesLiteral(bytesLit), BooleanLiteral => SemanticType.Bool, @@ -233,13 +228,32 @@ private SemanticType ClassifyIntegerLiteral(IntegerLiteral il) { AddError(result.ErrorMessage!, il.LineStart, il.ColumnStart, - code: DiagnosticCodes.Semantic.IntegerLiteralOutOfRange, + code: DiagnosticCodes.Semantic.NumericLiteralOutOfRange, span: il.Span); return result.Type; } return result.Type; } + /// + /// A float literal is refused at its own type's range exactly as an integer literal is + /// (SPY0210, #2340): 1e400 is not a silent infinity, and neither 1e39f nor + /// 1e29m reaches code generation. The resolved type is returned either way, as + /// does. + /// + private SemanticType ClassifyFloatLiteral(FloatLiteral fl) + { + var result = Shared.FloatLiteralClassifier.Classify(fl.Value, fl.Suffix); + if (result.IsError) + { + AddError(result.ErrorMessage!, + fl.LineStart, fl.ColumnStart, + code: DiagnosticCodes.Semantic.NumericLiteralOutOfRange, + span: fl.Span); + } + return result.Type; + } + private SemanticType CheckModifiedArgument(ModifiedArgument modArg) { // Handle inline out declarations: out value: int, out value: auto, out let value[: int] diff --git a/src/Sharpy.Compiler/Shared/FloatLiteralClassifier.cs b/src/Sharpy.Compiler/Shared/FloatLiteralClassifier.cs new file mode 100644 index 000000000..6d6c95463 --- /dev/null +++ b/src/Sharpy.Compiler/Shared/FloatLiteralClassifier.cs @@ -0,0 +1,80 @@ +using System.Globalization; +using Sharpy.Compiler.Semantic; +using Sharpy.Compiler.Semantic.Registry; + +namespace Sharpy.Compiler.Shared; + +/// +/// Classifies float literals by suffix and asks the ONE range question every float-literal +/// position shares (#2340, ruling R-HU): does the literal's text have a finite value in the +/// type it resolves to? Twin of , and the same SPY0210 +/// family refuses a miss. +/// +/// Before this seam only one of the four float cells was checked: the unsuffixed literal +/// narrowed into a float32/decimal slot (x: float32 = 1e40 is SPY0220, +/// float_literals.md). The literal's OWN type was never checked, so 1e400 and +/// 1e39f reached Roslyn (CS0594 → SPY0599) and 1e29m crashed the emitter's decimal +/// parse (SPY0909). Both the literal-typing arm and the slot-narrowing predicates now call +/// , so a literal the store seam admits is one code generation can spell. +/// +/// Each arm parses exactly as Roslyn's lexer judges the emitted text: double and +/// float are both correctly-rounded parses of the digits (a float32 literal is +/// parsed as a float directly, never through double — the emitted text carries an +/// f suffix, so a double-rounded answer could disagree with the C# compiler at the +/// boundary), and decimal is +/// under , which is what admits an exponent form. +/// +internal static class FloatLiteralClassifier +{ + public readonly record struct Result(SemanticType Type, bool IsError, string? ErrorMessage); + + /// The type a float literal's suffix resolves it to (f, m, else float64). + public static SemanticType TypeOf(string? suffix) => suffix?.ToUpperInvariant() switch + { + "F" => SemanticType.Float32, + "M" => SemanticType.Decimal, + _ => SemanticType.Double, + }; + + /// + /// Classifies a float literal's type from its and refuses a value + /// outside that type's range. + /// + public static Result Classify(string value, string? suffix) + { + var type = TypeOf(suffix); + var clrType = PrimitiveCatalog.GetPrimitiveInfo(type)!.ClrType; + return Fits(value, clrType) + ? new Result(type, false, null) + : new Result(type, true, $"Float literal '{value}' is too large for '{CSharpName(clrType)}'"); + } + + /// + /// Whether the float literal text has a finite value in + /// (, or + /// ). The one range question for every float-literal position. + /// + public static bool Fits(string value, Type clrType) + { + if (clrType == typeof(double)) + { + return double.TryParse(value, NumberStyles.Float, CultureInfo.InvariantCulture, out var d) + && double.IsFinite(d); + } + + if (clrType == typeof(float)) + { + return float.TryParse(value, NumberStyles.Float, CultureInfo.InvariantCulture, out var f) + && float.IsFinite(f); + } + + if (clrType == typeof(decimal)) + return decimal.TryParse(value, NumberStyles.Float, CultureInfo.InvariantCulture, out _); + + throw new ArgumentException( + $"A float literal resolves to double, float or decimal, never '{clrType}'.", nameof(clrType)); + } + + private static string CSharpName(Type clrType) + => clrType == typeof(float) ? "float" : clrType == typeof(decimal) ? "decimal" : "double"; +} diff --git a/src/Sharpy.Compiler/Shared/IntegerLiteralClassifier.cs b/src/Sharpy.Compiler/Shared/IntegerLiteralClassifier.cs index 50da11af0..f3e1d63a2 100644 --- a/src/Sharpy.Compiler/Shared/IntegerLiteralClassifier.cs +++ b/src/Sharpy.Compiler/Shared/IntegerLiteralClassifier.cs @@ -61,9 +61,19 @@ public static Result Classify(string value, string? suffix) }; } + /// + /// Whether a unary minus over an integer literal with is a negated + /// LITERAL (): unsuffixed or L. An unsigned literal has no + /// negative value in its own type, so -1U / -1UL are the unary operator over an + /// unsigned operand (int64 and a refusal, as C# types them), never a negated literal (#2340). + /// + public static bool NegatesAsLiteral(string? suffix) + => string.IsNullOrEmpty(suffix) || suffix.Equals("L", StringComparison.OrdinalIgnoreCase); + /// /// Classifies a negated integer literal (unary minus over integer literal). /// The negated magnitude determines the type: -2147483648 fits int, -9223372036854775808 fits long. + /// Callers gate on ; an unsigned suffix is classified unnegated. /// public static Result ClassifyNegated(string value, string? suffix) { From f6cf5c9b23507bc5aeea8fd826bc52c807731187 Mon Sep 17 00:00:00 2001 From: antonsynd <2648730+antonsynd@users.noreply.github.com> Date: Thu, 22 Oct 2026 03:06:09 -0400 Subject: [PATCH 07/13] =?UTF-8?q?refactor(codegen):=20the=20emitter=20perf?= =?UTF-8?q?orms=20no=20reflection=20=E2=80=94=20whether=20an=20imported=20?= =?UTF-8?q?CLR=20name=20is=20an=20open=20generic=20is=20a=20CodeGenInfo=20?= =?UTF-8?q?fact=20decided=20against=20the=20registry's=20visibility=20(ref?= =?UTF-8?q?s=20#2320)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit R-HM (a). CodeGenInfo gains IsOpenGenericDefinition: for a from-imported type symbol, the name denotes a generic type definition — its own type parameters, or a generic member of the CLR arity group discovery built for it (ModuleRegistry.GetNamespaceTypes, against the registry's own visibility: framework + references). CodeGenInfoComputer.ProcessFromImport records it; it freezes with the entry at MaterializeCodeGenInfo. The emitter's alias loop reads it and nothing else; IsClrTypeGenericInNamespace, its AssemblyVisibility.FrameworkOnly read and the TODO(#2320) are deleted. SemanticBinding.MergeFrom carries the fact onto an existing entry, as it carries the layout (#2102): an `as`-aliased import binds a clone whose import-resolution entry is merged first, and without the arm `from system import Action as A` emitted `using A = global::System.Action;` (measured on the first cut of this change, which the old reflection had skipped). SymbolFactMirrorConformanceTests classifies the new member same-file-only. Premise correction: neither EmitterCarrierOnlyConformanceTests nor ProcessWideResolutionScanTests had a census entry for this site — the first's deny universe is the Semantic namespaces (AssemblyVisibility is in Discovery) and the second matches process-wide sites (AppDomain, Type.GetType, Assembly.Load*), not AssemblyVisibility reads. Nothing to drain; the guard is new instead: EmitterCarrierOnlyConformanceTests. CodeGenSources_NameNoClrVisibilitySeam denies CodeGen every Discovery type that IS AssemblyVisibility or carries it on its non-private surface (derived by reflection, so a new resolver taking the seam is denied the day it is added), with SyntheticVisibilitySeamRead_IsFlagged as positive/negative control (the deleted read in three spellings flagged; ClrTypeBridge's catalog lookup not). Matrix (OpenGenericImportAliasTests, both cells run and print): visibility {framework, referenced System.Widgets built in-test} x spelling {bare, as} x name {arity group with a generic member, non-generic (positive control: alias emitted)}. Mutation (Rule 12): - emitter re-reads AssemblyVisibility.FrameworkOnly: 1 red (CodeGenSources_NameNoClrVisibilitySeam; 18 green) -> restored 19/0/0. - MergeFrom arm disabled: 1 red (Framework_ArityGroupIsNotAliased_BareOrAsAlias; 18 green) -> restored. - DenotesGenericDefinition ignores the arity group: 2 red (both OpenGenericImportAliasTests; 17 green) -> restored 19/0/0. Direction (run, base e7e0a0ef7 -> this commit): - framework `from system import Action, Nullable, Math, Guid, Tuple` + `from system.collections.generic import IList, Dictionary, KeyValuePair` + `from system import Action as A`: prints 2/4/9/2 before and after; generated C# byte-identical. - referenced System.Widgets.dll (Box, Box, Pair, Gadget) via -r: prints a/box/7 before and after; the only C# change is `using Box = global::System.Widgets.Box;` no longer emitted (Box is visible through the reference, which the framework-only read missed). No snapshot moved (FileBasedIntegrationTests green). Filtered @ this tree (measured): Compiler.Tests OpenGenericImportAlias | Compiler.Tests.CodeGen | NumericLiteralRangeMatrix | TypeChecker | Semantic | Conformance | Census | Roster | EmitterCarrier | ProcessWide | FileBasedIntegrationTests 33751/0/1; Lsp.Tests FrontEndParity 1/0/0; Stdlib.Tests 4273/0/0; build 0 warnings. Co-Authored-By: Claude Opus 5.5 (1M context) Claude-Session: https://claude.ai/code/session_0126pNZqmMsSbn4WFFPhpnxW --- .../EmitterCarrierOnlyConformanceTests.cs | 121 ++++++++++++++++++ .../CodeGen/OpenGenericImportAliasTests.cs | 88 +++++++++++++ .../SymbolFactMirrorConformanceTests.cs | 1 + .../CodeGen/RoslynEmitter.CompilationUnit.cs | 37 +----- src/Sharpy.Compiler/Semantic/CodeGenInfo.cs | 13 ++ .../Semantic/CodeGenInfoComputer.cs | 14 +- .../Semantic/SemanticBinding.cs | 10 +- 7 files changed, 250 insertions(+), 34 deletions(-) create mode 100644 src/Sharpy.Compiler.Tests/CodeGen/OpenGenericImportAliasTests.cs diff --git a/src/Sharpy.Compiler.Tests/CodeGen/EmitterCarrierOnlyConformanceTests.cs b/src/Sharpy.Compiler.Tests/CodeGen/EmitterCarrierOnlyConformanceTests.cs index 012c3a4c7..65a3ee145 100644 --- a/src/Sharpy.Compiler.Tests/CodeGen/EmitterCarrierOnlyConformanceTests.cs +++ b/src/Sharpy.Compiler.Tests/CodeGen/EmitterCarrierOnlyConformanceTests.cs @@ -256,6 +256,127 @@ public void Ratchet_HasNoStaleEntries() } } + // ---- the CLR-name visibility seam (#2320) ------------------------------------------------ + + /// The namespace root of the CLR-discovery layer, whose visibility seam CodeGen may not name. + private const string DiscoveryNamespaceRoot = "Sharpy.Compiler.Discovery"; + + /// + /// The emitter performs no reflection (Critical Rule 2), so it may not name the CLR-name + /// visibility seam: AssemblyVisibility — the ONE place a CLR name is resolved against + /// {framework} ∪ {the registry's references} (#2250) — or any discovery type whose own + /// surface carries it (a resolver handed the visibility to answer from). The deny set is + /// derived by reflection, so a resolver added tomorrow is denied the day it takes the seam. + /// + /// The deny universe is + /// the Semantic namespaces, so it never saw this: IsClrTypeGenericInNamespace + /// (RoslynEmitter.CompilationUnit.cs) asked AssemblyVisibility.FrameworkOnly + /// whether a using alias named an open generic, under a guard admitting any + /// System.* module — a referenced package in such a namespace was visible to the + /// checker and invisible there. The answer is now the imported symbol's + /// CodeGenInfo.IsOpenGenericDefinition (#2320). + /// + [Fact] + public void CodeGenSources_NameNoClrVisibilitySeam() + { + var seam = VisibilitySeamNames(); + seam.Should().Contain("AssemblyVisibility", + "the visibility seam is the type the deny set is anchored on; if it was renamed or moved " + + "out of the Discovery namespace this guard is derived from a type that no longer exists"); + + var violations = ScanCodeGenSources(seam, DiscoveryNamespaceSegments()); + foreach (var violation in violations) + _output.WriteLine($" {violation}"); + + violations.Should().BeEmpty( + "code generation performs no reflection: which CLR names are visible to this compilation " + + "is decided against the registry's visibility in discovery/semantic analysis and " + + "materialized for the emitter (CodeGenInfo / SemanticInfo). Seam types: " + + string.Join(", ", seam.OrderBy(n => n, StringComparer.Ordinal)) + + "\nViolations:\n " + string.Join("\n ", violations)); + } + + /// + /// Positive and negative controls for : + /// the deleted emitter read, in every spelling it can take, is flagged; the name catalog + /// CodeGen legitimately consults (ClrTypeBridge, which answers from a static table) + /// is not in the seam set and is not flagged. + /// + [Fact] + public void SyntheticVisibilitySeamRead_IsFlagged() + { + var seam = VisibilitySeamNames(); + var segments = DiscoveryNamespaceSegments(); + + var spellings = new (string Label, string Source)[] + { + ("the deleted #2320 read, namespace-qualified", + "class C { bool M(string n) { var framework = Discovery.AssemblyVisibility.FrameworkOnly; return framework.ResolveType(n) != null; } }"), + ("unqualified static read", + "class C { object M() => AssemblyVisibility.FrameworkOnly; }"), + ("fully qualified declaration", + "class C { private Sharpy.Compiler.Discovery.AssemblyVisibility _v; }"), + }; + foreach (var (label, source) in spellings) + { + ScanSource("Mutation.cs", source, seam, segments).Should().NotBeEmpty( + $"a {label} is the emitter asking reflection a semantic question"); + } + + seam.Should().NotContain("ClrTypeBridge", + "ClrTypeBridge's surface takes no visibility — it is the Sharpy-name → C#-name catalog " + + "TypeSyntaxMapper reads, not a resolver"); + ScanSource("Control.cs", + "class C { string? M(string n) => ClrTypeBridge.TryGetCSharpTypeName(n); }", seam, segments) + .Should().BeEmpty("a catalog lookup is not a visibility read"); + } + + /// + /// AssemblyVisibility and every type in the Discovery namespaces whose non-private + /// signature (property, field, method, constructor) carries it. + /// + private static HashSet VisibilitySeamNames() + { + var discovery = DiscoveryUniverse(); + var visibility = discovery.FirstOrDefault(t => t.Name == "AssemblyVisibility"); + visibility.Should().NotBeNull("AssemblyVisibility is expected in the Discovery namespaces"); + + return discovery + .Where(t => t == visibility || SignatureTypes(t).Contains(visibility!)) + .Select(SimpleName) + .ToHashSet(StringComparer.Ordinal); + } + + private static IReadOnlyList DiscoveryUniverse() + { + var assembly = typeof(SemanticInfo).Assembly; + Type?[] types; + try + { + types = assembly.GetTypes(); + } + catch (ReflectionTypeLoadException ex) + { + types = ex.Types; + } + + return types + .Where(t => t is not null) + .Select(t => t!) + .Where(t => t.Namespace is { } ns + && (ns == DiscoveryNamespaceRoot + || ns.StartsWith(DiscoveryNamespaceRoot + ".", StringComparison.Ordinal))) + .Where(t => !t.Name.Contains('<', StringComparison.Ordinal)) + .ToList(); + } + + private static HashSet DiscoveryNamespaceSegments() + => DiscoveryUniverse() + .Select(t => t.Namespace!) + .Distinct(StringComparer.Ordinal) + .SelectMany(ns => ns.Split('.')) + .ToHashSet(StringComparer.Ordinal); + // ---- executable mutation cells (#1475, Task 3) -------------------------------------------- /// diff --git a/src/Sharpy.Compiler.Tests/CodeGen/OpenGenericImportAliasTests.cs b/src/Sharpy.Compiler.Tests/CodeGen/OpenGenericImportAliasTests.cs new file mode 100644 index 000000000..2f0d0e93e --- /dev/null +++ b/src/Sharpy.Compiler.Tests/CodeGen/OpenGenericImportAliasTests.cs @@ -0,0 +1,88 @@ +using FluentAssertions; +using Sharpy.Compiler.Tests.Integration; +using Sharpy.TestInfrastructure.Integration; +using Xunit; +using Xunit.Abstractions; + +namespace Sharpy.Compiler.Tests.CodeGen; + +/// +/// A from-import of a CLR-namespace name emits a using alias only when the name denotes no +/// generic type definition — C# cannot alias an open generic (CS0305) — and that is decided from +/// the imported symbol's materialized CodeGenInfo.IsOpenGenericDefinition, against the +/// compilation registry's OWN visibility (#2320, ruling R-HM (a)). +/// +/// Cells. Visibility {framework, referenced package in a System.* namespace} × +/// spelling {bare, as alias} × name {arity group with a generic member, non-generic only}. +/// The emitter used to ask reflection against the FRAMEWORK set alone, so a referenced +/// System.Widgets.Box beside Box<T> was aliased while the framework's +/// Action beside Action<T> was not; and the aliased spelling's fact rides +/// SemanticBinding.MergeFrom, where import resolution's earlier entry for the clone wins. +/// +/// Every cell runs and prints: the alias set is the observable, but a program that stopped +/// running would be the regression. The non-generic names are the positive controls — an alias IS +/// emitted for them, so "no alias" cannot pass because the alias path never ran. +/// +public class OpenGenericImportAliasTests : IntegrationTestBase +{ + public OpenGenericImportAliasTests(ITestOutputHelper output) : base(output) { } + + private const string WidgetsSource = """ + namespace System.Widgets + { + public class Box { public static string Describe() => "box"; } + public class Box { public T Value; public Box(T value) { Value = value; } } + public class Pair { public A First; public Pair(A first, B second) { First = first; } } + public class Gadget { public static int Size() => 7; } + } + """; + + private static readonly Lazy WidgetsAssembly = + new(() => IsNoneReferenceCheckMatrixTests.BuildSpecimenAssembly("SystemWidgets2320", WidgetsSource)); + + protected override IEnumerable GetAdditionalReferenceAssemblyPaths() + => base.GetAdditionalReferenceAssemblyPaths().Append(WidgetsAssembly.Value); + + [Fact] + public void Framework_ArityGroupIsNotAliased_BareOrAsAlias() + { + var result = CompileAndExecute( + "from system import Action, Math\n" + + "from system import Action as A\n\n" + + "def main() -> None:\n" + + " act: Action[int] = lambda x: print(x)\n" + + " act(9)\n" + + " print(Math.Max(3, 4))\n"); + + result.Success.Should().BeTrue(string.Join(" | ", result.CompilationErrors)); + result.StandardOutput.Should().Be("9\n4\n"); + result.GeneratedCSharp.Should().Contain("using Math = global::System.Math;", + "positive control: a non-generic framework name is aliased, so the alias path ran"); + result.GeneratedCSharp.Should().NotContain("using Action =", + "Action is the Action / Action… group — no alias for an open generic name"); + result.GeneratedCSharp.Should().NotContain("using A =", + "the `as` spelling binds a clone whose import-resolution entry precedes the fact; the " + + "fact must survive the merge"); + } + + [Fact] + public void ReferencedSystemNamespace_ArityGroupIsNotAliased() + { + var result = CompileAndExecute( + "from system.widgets import Box, Pair, Gadget\n\n" + + "def main() -> None:\n" + + " p = Pair[str, int](\"a\", 1)\n" + + " print(p.First)\n" + + " print(Box.Describe())\n" + + " print(Gadget.Size())\n"); + + result.Success.Should().BeTrue(string.Join(" | ", result.CompilationErrors)); + result.StandardOutput.Should().Be("a\nbox\n7\n"); + result.GeneratedCSharp.Should().Contain("using Gadget = global::System.Widgets.Gadget;", + "positive control: a non-generic referenced name is aliased"); + result.GeneratedCSharp.Should().NotContain("using Box =", + "Box is visible to this compilation through its reference — the registry's " + + "visibility, not the framework set alone, decides the alias"); + result.GeneratedCSharp.Should().NotContain("using Pair ="); + } +} diff --git a/src/Sharpy.Compiler.Tests/Semantic/SymbolFactMirrorConformanceTests.cs b/src/Sharpy.Compiler.Tests/Semantic/SymbolFactMirrorConformanceTests.cs index d86fdb3d3..859367283 100644 --- a/src/Sharpy.Compiler.Tests/Semantic/SymbolFactMirrorConformanceTests.cs +++ b/src/Sharpy.Compiler.Tests/Semantic/SymbolFactMirrorConformanceTests.cs @@ -1018,6 +1018,7 @@ public void EverySymbolProperty_IsClassified() ["OperatorParameterShape"] = "same-file-only — read at RoslynEmitter.Operators.cs equality synthesis (#1719)", ["HasRuntimeAssignedFlag"] = "same-file-only — a runtime-checked LOCAL fact recomputed by LocalNameAllocator every build (never module-level, so never served from a warm cache); read at RoslynEmitter.Statements.Assignments.cs (declaration flag + MaybeWrapRuntimeAssignedStore) and RoslynEmitter.Expressions.cs (CheckedLocal read) (#1839)", ["RequiresPerInstanceDefault"] = "same-file-only — a dataclass/struct field's mutable-collection-default fact (#1684, R-A), computed by CodeGenInfoComputer.ProcessField from ConstantDefaultClassifier for the file being emitted; read by the synthesized-constructor emitter (RoslynEmitter.ClassMembers.Dataclass.cs / RoslynEmitter.ClassMembers.Constructors.cs) once P3.5 lands the per-instance lowering — same invariant as IsCompileTimeConstant, since constructor generation is always same-file", + ["IsOpenGenericDefinition"] = "same-file-only — a from-imported type's 'names a generic definition' fact (#2320), computed by CodeGenInfoComputer.ProcessFromImport from the symbol's arity group for the importing file; read only by that file's using-alias emission (RoslynEmitter.CompilationUnit.cs GenerateFromImportUsings), which runs only when no ProjectNamespace is set — a single-file build, never served from a warm cache", ["RequiresInstanceImpl"] = "same-file-only — a method's method-lowered-super-call fact (#1740), marked by TypeChecker.RecordSuperDunderLoweringFact and bridged at MaterializeCodeGenInfo; read by the operator-dunder emitter (RoslynEmitter.Operators.cs :250/:313/:368) once P4.2 lands the instance-_Impl-split decision — same invariant as OverridesClrBaseMember, since a method's own emission shape is always same-file", }; diff --git a/src/Sharpy.Compiler/CodeGen/RoslynEmitter.CompilationUnit.cs b/src/Sharpy.Compiler/CodeGen/RoslynEmitter.CompilationUnit.cs index bd5968901..0cc9e2c71 100644 --- a/src/Sharpy.Compiler/CodeGen/RoslynEmitter.CompilationUnit.cs +++ b/src/Sharpy.Compiler/CodeGen/RoslynEmitter.CompilationUnit.cs @@ -490,8 +490,10 @@ private IEnumerable GenerateFromImportUsings(FromImportSta { // from system import Math -> using Math = global::System.Math; // Emit per-name type aliases with global:: prefix to avoid ambiguity. - // Skip generic types — C# cannot alias open generic types (e.g., IEquatable), - // and the base class list already uses fully-qualified names. + // Skip generic types — C# cannot alias open generic types (CS0305: IEquatable, + // the Action / Action group), and the base class list already uses + // fully-qualified names. Which names those are is decided in semantic analysis and + // read off the imported symbol's CodeGenInfo (#2320). // When ProjectNamespace is set, skip aliases entirely — identifier expressions // use global:: qualified CLR names directly, avoiding shadowing conflicts // (e.g., Sharpy.Math would shadow a using Math = global::System.Math alias). @@ -503,12 +505,10 @@ private IEnumerable GenerateFromImportUsings(FromImportSta // to ToNamespacePart("Guid") = "GUID". Look up AsName first so a CLR-backed // symbol reaches its reflected clrType.Name. var symbol = _context.LookupSymbol(importedName.AsName ?? importedName.Name); - if (symbol is TypeSymbol { IsGeneric: true }) + if (symbol != null && GetCodeGenInfo(symbol) is { IsOpenGenericDefinition: true }) continue; var clrTypeName = ResolveClrTypeName(symbol, importedName.Name); - if (IsClrTypeGenericInNamespace(namespaceName, clrTypeName)) - continue; var csharpName = importedName.AsName ?? clrTypeName; var qualifiedName = $"global::{namespaceName}.{clrTypeName}"; @@ -578,33 +578,6 @@ private static string ResolveClrTypeName(Symbol? symbol, string fallbackName) return NameMangler.ToNamespacePart(fallbackName); } - /// - /// Check whether the CLR type with the given name in the given namespace is a generic - /// type definition. Used when emitting using-aliases for CLR imports: C# cannot alias - /// open generic types (CS0305), so we must skip them. This directly queries the CLR - /// metadata rather than the symbol table, which may return a Sharpy type that shadows - /// the CLR name (e.g., Sharpy.IList vs System.Collections.Generic.IList<T>). - /// - private static bool IsClrTypeGenericInNamespace(string namespaceName, string typeName) - { - // TODO(#2320): Rule-2 cell — this is the emitter asking reflection a semantic question, which - // belongs in semantic analysis as a materialized fact (CodeGenInfo). Until it moves, - // it asks the FRAMEWORK visibility — a process constant — instead of every assembly the process - // has loaded; its only caller passes a .NET framework namespace (IsNetFrameworkNamespace). - var framework = Discovery.AssemblyVisibility.FrameworkOnly; - - // Try with generic arity suffixes `1 through `4 (covers most BCL types) - for (int arity = 1; arity <= 4; arity++) - { - var fullName = $"{namespaceName}.{typeName}`{arity}"; - var type = framework.ResolveType(fullName); - if (type != null && type.IsGenericTypeDefinition) - return true; - } - - return false; - } - /// /// Check if a module name corresponds to a Sharpy stdlib (.NET) module. /// Uses the SemanticBinding registry populated during import resolution. diff --git a/src/Sharpy.Compiler/Semantic/CodeGenInfo.cs b/src/Sharpy.Compiler/Semantic/CodeGenInfo.cs index bb40f3ca2..6e3813c54 100644 --- a/src/Sharpy.Compiler/Semantic/CodeGenInfo.cs +++ b/src/Sharpy.Compiler/Semantic/CodeGenInfo.cs @@ -240,6 +240,19 @@ public sealed record CodeGenInfo /// public bool IsNamespaceSibling { get; init; } + /// + /// For a from-imported type symbol: the imported name denotes a generic type DEFINITION — the + /// symbol's own type parameters, or, where a CLR namespace spells the name as a group of types + /// differing only in arity (Action / Action`1…, ), + /// any generic member of that group. C# cannot alias an open generic type (CS0305), so code + /// generation emits no using alias for such an import (#2320). Decided here from the + /// arity group discovery built against the compilation registry's OWN visibility — the + /// framework set plus the registry's references — so a referenced package declaring a type in + /// a System.* namespace is seen exactly as the type checker sees it; the emitter used to + /// re-ask reflection against the framework set alone. + /// + public bool IsOpenGenericDefinition { get; init; } + /// /// Get the versioned C# name (includes version suffix for redeclared variables). /// diff --git a/src/Sharpy.Compiler/Semantic/CodeGenInfoComputer.cs b/src/Sharpy.Compiler/Semantic/CodeGenInfoComputer.cs index 22d880fba..89739e179 100644 --- a/src/Sharpy.Compiler/Semantic/CodeGenInfoComputer.cs +++ b/src/Sharpy.Compiler/Semantic/CodeGenInfoComputer.cs @@ -440,12 +440,24 @@ private void ProcessFromImport(FromImportStatement fromImport) // namespace of the file that DECLARES them — a package __init__ re-exporting a // submodule's type does not move it (#2039). IsNamespaceSibling = siblingSegments != null, - NamespaceSegments = siblingSegments + NamespaceSegments = siblingSegments, + IsOpenGenericDefinition = symbol is TypeSymbol importedTypeSymbol + && DenotesGenericDefinition(importedTypeSymbol), }); } } } + /// + /// Whether an imported type name denotes a generic type definition (#2320): its own type + /// parameters, or a generic member of the CLR arity group discovery recorded for the name. + /// The group is built by ModuleRegistry.GetNamespaceTypes from the registry's + /// visibility, so this reads a discovered fact and performs no reflection of its own. + /// + private static bool DenotesGenericDefinition(TypeSymbol type) + => type.IsGeneric + || (type.ClrArityGroup is { } group && group.Keys.Any(arity => arity > 0)); + private string DetermineCSharpNameForFromImport(string name, Symbol symbol, bool isNetModule) { // A from-imported Sharpy function is spelled by its DECLARATION's materialized name — one diff --git a/src/Sharpy.Compiler/Semantic/SemanticBinding.cs b/src/Sharpy.Compiler/Semantic/SemanticBinding.cs index c654dbb5e..dd3512f24 100644 --- a/src/Sharpy.Compiler/Semantic/SemanticBinding.cs +++ b/src/Sharpy.Compiler/Semantic/SemanticBinding.cs @@ -606,13 +606,21 @@ public void MergeFrom(SemanticBinding other) // entry written earlier by import resolution — an aliased from-import's clone — carries // no layout; it gains the recorded one here, or the emitter would find none (#2102). var existing = _codeGenInfo[symbol]; + var merged = existing; if (existing.NamespaceSegments == null && info.NamespaceSegments != null) - _codeGenInfo[symbol] = existing with + merged = merged with { IsNamespaceSibling = info.IsNamespaceSibling, NamespaceSegments = info.NamespaceSegments, MembersClassName = existing.MembersClassName ?? info.MembersClassName, }; + // The from-import's open-generic fact is recorded by CodeGenInfoComputer only, as the + // layout is: an aliased clone's earlier import-resolution entry gains it here, or + // `from system import Action as A` would alias an open generic group (#2320). + if (!merged.IsOpenGenericDefinition && info.IsOpenGenericDefinition) + merged = merged with { IsOpenGenericDefinition = true }; + if (!ReferenceEquals(merged, existing)) + _codeGenInfo[symbol] = merged; } // CLR-base-override marks are consumed at MaterializeCodeGenInfo, which runs on the From 0b30fa9087a59ebff100a4922d6acf9b5db951bc Mon Sep 17 00:00:00 2001 From: antonsynd <2648730+antonsynd@users.noreply.github.com> Date: Thu, 22 Oct 2026 01:14:52 -0400 Subject: [PATCH 08/13] fix(lexer): a numeric suffix is keyed by the literal's kind and read on every ReadNumber path (refs #2339) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Contract (integer_literals.md / float_literals.md): every suffix the spec lists is accepted on every spelling of its literal kind and refused (SPY0023) on every other kind. Class: the parallel-path meta-class — the hex/bin/oct path was keyed by kind since #2208 (IntegerSuffixes), the decimal path validated against one kind-blind roster, and the leading-dot path returned before any suffix arm. Cure (R-HX a): ONE table, Lexer.NumericSuffixes (suffix -> NumericSuffixKind {Integer, Float}), and ONE reader, ReadNumericSuffix(sb, NumberSpelling), called by all five paths (decimal, leading-dot, hex, binary, octal) after their digits. SpellingAccepts states the C# rule: decimal-integer digits take both kinds (42L; 42f is a float), a float spelling (fraction or exponent, any spelling) takes only f/d/m, a prefixed literal only u/l/ul. The prefixed-literal refusal text is unchanged (the spec quotes it); a float spelling's refusal says "(a float literal accepts only the float suffixes f, d and m)". The table is an ImmutableDictionary (StaticStateConformanceTests refused a static Dictionary). float_literals.md gains the sentence and an error example for 3.14L / 1e5u. Fixture-hit count FIRST (grep over every .spy/.md under src docs build_tools examples): integer suffix on a float spelling 0, leading-dot literal with a suffix 0 in code position (3 hits, all inside format-spec strings such as ",.1f"). Positive control of the grep: 58 float-suffix hits. Matrix (NumericLiteralTwinTests.SharpyOnlyRows_LexAsTheSpecSays, now generated from SuffixSpellings): kind {decimal int, decimal float, hex, binary/octal} x spelling {42, 1_000, 0.5, 3.14, .5, .1_5, 1e5, 1E+5, .5e3, 1_0.2_5, 1_0e1_0, 0xFF, 0xF_F, 0b101, 0b1_01, 0o17, 0o1_7} x suffix {l L u U ul UL uL Ul, f F d D m M, j J lu x, none} (both cases carried by the rosters) + the trailing-dot 5. rows (a letter after the dot is a member name) = 345 rows (was 83). The old comment admitting the integer-suffix-on-float cell as unpinned is gone. Direction record (sharpyc run, base binary @ e7e0a0ef7 -> this commit): print(3.14L, 1e5u) 3.14 100000.0 -> SPY0023 Invalid numeric suffix: L (a float literal ...) a: int64 = 3.14L SPY0220 float64 -> SPY0023 print(3.14u) runs -> SPY0023 print(1.5ul) SPY0909 ICE (FormatException '1.5u') -> SPY0023 y = .5f; print(y) SPY0103 got Identifier -> 0.5 print(.5f, .5m, .5d, .5e3f, .1_5F) SPY0104 -> 0.5 0.5 0.5 500.0 0.15 a: int = .5f / .5m (SPY0103) -> SPY0220 'float32' / 'decimal' (same as 0.5f) print(.5j) SPY0104 got Identifier -> SPY0023 print(.5x) SPY0104 -> SPY0023 (as 0.5x) print(1e5f, 1E5m, 1_0e1_0d) 100000.0 100000 100000000000.0 -> unchanged print(1j, 1.5j) SPY0023 -> unchanged Mutation (Rule 12; NumericLiteralTwin filter, 348 tests): M1 leading-dot path skips ReadNumericSuffix -> 294/54/0, the 54 red rows exactly the 3 leading-dot spellings x 18 suffixed rows M2 SpellingAccepts(DecimalFloat) => true -> 276/72/0, the 72 red rows exactly the 9 float spellings x 8 integer suffixes restored (cp + touch, rebuilt) -> all 348 green (inside the 23799/0/5 run below) (measured on the Dictionary spelling of the table; the ImmutableDictionary swap changes no lookup.) Filtered runs @ this commit's tree (measured): Compiler.Tests Lexer|IntegerSuffix|Conformance|Census|Roster| FileBasedIntegrationTests|StaticState 23799/0/5 Lsp.Tests FrontEndParity 1/0/0 Lsp.Tests Category!=Benchmark 21944/0/0 dotnet format whitespace --verify-no-changes clean check_spec_blocks.sh clean (failing 0) No expected value changed; no snapshot changed. Co-Authored-By: Claude Opus 5.5 (1M context) Claude-Session: https://claude.ai/code/session_0126pNZqmMsSbn4WFFPhpnxW --- docs/language_specification/float_literals.md | 9 ++ .../Lexer/NumericLiteralTwinTests.cs | 118 ++++++++++----- src/Sharpy.Compiler/Lexer/Lexer.Literals.cs | 134 ++++++++++++------ 3 files changed, 180 insertions(+), 81 deletions(-) diff --git a/docs/language_specification/float_literals.md b/docs/language_specification/float_literals.md index facbbd31b..fc175c602 100644 --- a/docs/language_specification/float_literals.md +++ b/docs/language_specification/float_literals.md @@ -26,6 +26,15 @@ precise = 3.141_592_653 - `d` or `D` for `float64` (System.Double): `3.14d` (redundant but allowed) - `m` or `M` for `decimal` (System.Decimal): `3.14m` +A suffix means the same on every spelling of a float — `.5f` is `0.5f`, and `1e5f` and `1_0.2_5m` +take it too. The integer suffixes `L`, `u` and `ul` (see [Integer Literals](integer_literals.md)) +belong to integers; on a float literal they are refused (SPY0023), as in C#: + +```python +# x = 3.14L # ERROR (SPY0023): Invalid numeric suffix: L (a float literal accepts only the float suffixes f, d and m) +# y = 1e5u # ERROR (SPY0023): Invalid numeric suffix: u (a float literal accepts only the float suffixes f, d and m) +``` + ## Narrowing an Unsuffixed Literal to `float32` or `decimal` An **unsuffixed** float literal may narrow to `float32` or `decimal` at **every** store diff --git a/src/Sharpy.Compiler.Tests/Lexer/NumericLiteralTwinTests.cs b/src/Sharpy.Compiler.Tests/Lexer/NumericLiteralTwinTests.cs index d44cbbbed..fa1c0c47d 100644 --- a/src/Sharpy.Compiler.Tests/Lexer/NumericLiteralTwinTests.cs +++ b/src/Sharpy.Compiler.Tests/Lexer/NumericLiteralTwinTests.cs @@ -566,7 +566,46 @@ public void SharedGrammar_AgreesWithCPython() private static readonly string[] FloatSuffixes = { "f", "F", "d", "D", "m", "M" }; private static readonly string[] IntegerSuffixes = { "l", "L", "u", "U", "ul", "UL", "uL", "Ul" }; - private static readonly string[] InvalidSuffixes = { "lu", "x", "j" }; + + /// Not in either roster: j/J (no complex literal), lu (C# spelling the spec does not list), x. + private static readonly string[] InvalidSuffixes = { "j", "J", "lu", "x" }; + + private enum SpellingKind + { + /// Decimal digits, no ./exponent: takes integer AND float suffixes (42L, 42f). + DecimalInteger, + /// A fraction or exponent, any spelling: takes only the float suffixes. + DecimalFloat, + Hex, + BinaryOrOctal, + } + + /// + /// The suffix matrix's spelling axis (#2339): (source digits, the token value the lexer gives + /// them, kind). Every spelling of a kind must answer a suffix the same way — the leading-dot + /// .5 exactly as 0.5 (float_literals.md: ".5 — Same as 0.5"), the exponent-only + /// 1e5 and the underscored spellings as their plain twins. + /// + private static readonly (string Source, string Value, SpellingKind Kind)[] SuffixSpellings = + { + ("42", "42", SpellingKind.DecimalInteger), + ("1_000", "1000", SpellingKind.DecimalInteger), + ("0.5", "0.5", SpellingKind.DecimalFloat), + ("3.14", "3.14", SpellingKind.DecimalFloat), + (".5", "0.5", SpellingKind.DecimalFloat), + (".1_5", "0.15", SpellingKind.DecimalFloat), + ("1e5", "1e5", SpellingKind.DecimalFloat), + ("1E+5", "1E+5", SpellingKind.DecimalFloat), + (".5e3", "0.5e3", SpellingKind.DecimalFloat), + ("1_0.2_5", "10.25", SpellingKind.DecimalFloat), + ("1_0e1_0", "10e10", SpellingKind.DecimalFloat), + ("0xFF", "0xFF", SpellingKind.Hex), + ("0xF_F", "0xFF", SpellingKind.Hex), + ("0b101", "0b101", SpellingKind.BinaryOrOctal), + ("0b1_01", "0b101", SpellingKind.BinaryOrOctal), + ("0o17", "0o17", SpellingKind.BinaryOrOctal), + ("0o1_7", "0o17", SpellingKind.BinaryOrOctal), + }; /// /// (source, expected): expected is the token list without Eof as Type 'Value' @@ -575,50 +614,53 @@ public void SharedGrammar_AgreesWithCPython() public static TheoryData SharpyOnlyCases() { var data = new TheoryData(); - const string Refused = "error SPY0023"; - - // integer_literals.md: L/l, u/U, ul/UL on decimal integers; float_literals.md: f/F, d/D, m/M. - foreach (var s in IntegerSuffixes) - data.Add("42" + s, $"Integer '42{s}'"); - foreach (var s in FloatSuffixes) - data.Add("42" + s, $"Float '42{s}'"); - // float_literals.md: "3.14f" / "3.14d" / "3.14m". (An INTEGER suffix on a float is spec-silent - // and accepted by the lexer today — a findings-ledger line, deliberately not pinned here.) - foreach (var s in FloatSuffixes) - data.Add("3.14" + s, $"Float '3.14{s}'"); - - // integer_literals.md: "The integer suffixes work on every base"; "A hex digit is never read as a - // suffix … 0xFFf is 0xFFF"; "on a binary or octal literal, and m on a hex literal, they are refused (SPY0023)". - foreach (var s in IntegerSuffixes) - { - data.Add("0xFF" + s, $"Integer '0xFF{s}'"); - data.Add("0b101" + s, $"Integer '0b101{s}'"); - data.Add("0o17" + s, $"Integer '0o17{s}'"); - } - foreach (var s in FloatSuffixes) + var seen = new HashSet(StringComparer.Ordinal); + void Add(string source, string expected) { - data.Add("0b101" + s, Refused); - data.Add("0o17" + s, Refused); - data.Add("0xFF" + s, s is "m" or "M" ? Refused : $"Integer '0xFF{s}'"); + if (seen.Add(source)) + data.Add(source, expected); } + const string Refused = "error SPY0023"; - // Not in the roster: refused on every base. - foreach (var s in InvalidSuffixes) + // kind {decimal int, decimal float, hex, binary/octal} × spelling (SuffixSpellings) × suffix + // {every integer suffix, every float suffix, j/J and the other non-roster suffixes, none} — the + // suffix rosters carry both cases. integer_literals.md: L/l, u/U, ul/UL, "The integer suffixes work + // on every base"; "A hex digit is never read as a suffix … 0xFFf is 0xFFF"; the float suffixes + // "apply to decimal literals only; on a binary or octal literal, and m on a hex literal, they are + // refused (SPY0023)". float_literals.md: f/F, d/D, m/M on floats; an integer suffix on a float is + // refused (SPY0023), as in C# (#2339). + foreach (var (source, value, kind) in SuffixSpellings) { - foreach (var number in new[] { "42", "3.14", "0xFF", "0b101", "0o17" }) - data.Add(number + s, Refused); + string integerToken = $"Integer '{value}'"; + string floatToken = $"Float '{value}'"; + Add(source, kind == SpellingKind.DecimalFloat ? floatToken : integerToken); + + foreach (var s in IntegerSuffixes) + Add(source + s, kind == SpellingKind.DecimalFloat ? Refused : $"Integer '{value}{s}'"); + + foreach (var s in FloatSuffixes) + { + Add(source + s, kind switch + { + SpellingKind.DecimalInteger or SpellingKind.DecimalFloat => $"Float '{value}{s}'", + SpellingKind.Hex when s is not ("m" or "M") => $"Integer '{value}{s}'", + _ => Refused, + }); + } + + foreach (var s in InvalidSuffixes) + Add(source + s, Refused); } // float_literals.md: "Invalid - trailing decimal point without digit … ERROR: use 5.0 instead" — - // the lexer leaves the dot to the parser. - data.Add("1.", "Integer '1', Dot '.'"); - data.Add("12.", "Integer '12', Dot '.'"); - data.Add("1_000.", "Integer '1000', Dot '.'"); - data.Add("1.e5", "Integer '1', Dot '.', Identifier 'e5'"); - - // No complex literal: the imaginary j is an invalid suffix. - data.Add("1j", Refused); - data.Add("1.5j", Refused); + // the lexer leaves the dot to the parser, and a letter after the dot is a member name, never a + // suffix, for every suffix of either kind. + Add("1.", "Integer '1', Dot '.'"); + Add("12.", "Integer '12', Dot '.'"); + Add("1_000.", "Integer '1000', Dot '.'"); + Add("1.e5", "Integer '1', Dot '.', Identifier 'e5'"); + foreach (var s in IntegerSuffixes.Concat(FloatSuffixes).Concat(InvalidSuffixes)) + Add("5." + s, $"Integer '5', Dot '.', Identifier '{s}'"); return data; } diff --git a/src/Sharpy.Compiler/Lexer/Lexer.Literals.cs b/src/Sharpy.Compiler/Lexer/Lexer.Literals.cs index 1bf94bece..6e8d4cf9b 100644 --- a/src/Sharpy.Compiler/Lexer/Lexer.Literals.cs +++ b/src/Sharpy.Compiler/Lexer/Lexer.Literals.cs @@ -1,3 +1,4 @@ +using System.Collections.Immutable; using System.Text; using Sharpy.Compiler.Diagnostics; @@ -784,6 +785,9 @@ private Token ReadNumber() throw ReportError("Invalid number: cannot end with underscore", startLine, startColumn, DiagnosticCodes.Lexer.InvalidNumber); } + // `.5f` is `0.5f` (float_literals.md: ".5 — Same as 0.5"): the same suffix table as every + // other path, never an early return that leaves the suffix to lex as an Identifier (#2339). + ReadNumericSuffix(sb, NumberSpelling.DecimalFloat, startLine, startColumn); return CreateToken(TokenType.Float, sb.ToString(), startLine, startColumn, startPosition); } @@ -904,32 +908,11 @@ private Token ReadNumber() throw ReportError("Invalid number: cannot end with underscore", startLine, startColumn, DiagnosticCodes.Lexer.InvalidNumber); } - // Check for suffix (f, L, ul, etc.) - if (_position < _source.Length && char.IsLetter(_source[_position])) - { - var suffix = _source[_position].ToString(); - _position++; - _column++; - - // Check for two-letter suffix (ul, UL) - if (_position < _source.Length && char.IsLetter(_source[_position])) - { - suffix += _source[_position]; - _position++; - _column++; - } - - // Validate suffix - var validSuffixes = new[] { "f", "F", "d", "D", "m", "M", "l", "L", "u", "U", "ul", "UL", "uL", "Ul" }; - if (!validSuffixes.Contains(suffix)) - throw ReportError($"Invalid numeric suffix: {suffix}", startLine, startColumn, DiagnosticCodes.Lexer.InvalidNumericSuffix); - - sb.Append(suffix); - - // Suffix on integer makes it that type, but we still parse as Float token if it has suffix - if (suffix.ToLower() == "f" || suffix.ToLower() == "d" || suffix.ToLower() == "m") - isFloat = true; - } + // A float suffix on integer digits makes the literal a float (`42f`); an integer suffix on a + // float spelling (`3.14L`, `1e5u`) is refused, never silently dropped (#2339). + var suffixKind = ReadNumericSuffix(sb, isFloat ? NumberSpelling.DecimalFloat : NumberSpelling.DecimalInteger, startLine, startColumn); + if (suffixKind == NumericSuffixKind.Float) + isFloat = true; var tokenType = isFloat ? TokenType.Float : TokenType.Integer; return CreateToken(tokenType, sb.ToString(), startLine, startColumn, startPosition); @@ -981,30 +964,94 @@ private Token ReadHexNumber(int startLine, int startColumn, int startPosition) // A hex digit is never a suffix: `0xFFf` is 0xFFF and `0x1D` is 29. The digit loop above // has already consumed every a-f/A-F, so what follows can only be an integer suffix. - ReadPrefixedIntegerSuffix(sb, "a hexadecimal", startLine, startColumn); + ReadNumericSuffix(sb, NumberSpelling.Hexadecimal, startLine, startColumn); return CreateToken(TokenType.Integer, sb.ToString(), startLine, startColumn, startPosition); } + /// The literal kind a numeric suffix belongs to (integer_literals.md, float_literals.md). + private enum NumericSuffixKind + { + Integer, + Float, + } + + /// + /// How a number's digits were spelled, which decides the suffix kinds it accepts + /// () and names it in the refusal. + /// + private enum NumberSpelling + { + /// Decimal digits with no . and no exponent: 42, 1_000. + DecimalInteger, + /// A decimal fraction or exponent: 3.14, .5, 1e5. + DecimalFloat, + Hexadecimal, + Binary, + Octal, + } + /// - /// The integer suffixes a literal of ANY base accepts (#2208): the decimal integer roster of - /// minus the float suffixes (f d m), which make a decimal - /// literal a float and have no meaning on a hex, binary or octal literal (C# refuses - /// 0x1m too). f/d are hex digits, so on a hex literal they never reach - /// this roster. + /// THE numeric suffix table (#2339): every suffix the spec lists, keyed by the literal kind it + /// belongs to. Integer suffixes (integer_literals.md): L/l, u/U, + /// ul/UL (and the mixed-case uL/Ul). Float suffixes + /// (float_literals.md): f/F, d/D, m/M. Every + /// path consults it after its digits; anything else (j, + /// lu, x) is refused with SPY0023. /// - private static readonly string[] IntegerSuffixes = { "l", "L", "u", "U", "ul", "UL", "uL", "Ul" }; + private static readonly ImmutableDictionary NumericSuffixes = new Dictionary(StringComparer.Ordinal) + { + ["l"] = NumericSuffixKind.Integer, + ["L"] = NumericSuffixKind.Integer, + ["u"] = NumericSuffixKind.Integer, + ["U"] = NumericSuffixKind.Integer, + ["ul"] = NumericSuffixKind.Integer, + ["UL"] = NumericSuffixKind.Integer, + ["uL"] = NumericSuffixKind.Integer, + ["Ul"] = NumericSuffixKind.Integer, + ["f"] = NumericSuffixKind.Float, + ["F"] = NumericSuffixKind.Float, + ["d"] = NumericSuffixKind.Float, + ["D"] = NumericSuffixKind.Float, + ["m"] = NumericSuffixKind.Float, + ["M"] = NumericSuffixKind.Float, + }.ToImmutableDictionary(StringComparer.Ordinal); + + /// + /// Whether a spelling takes a suffix of , as C# rules: decimal integer + /// digits take both kinds (42L; 42f is a float); a float spelling takes only the + /// float suffixes (C# refuses 3.14L and 1e5u); a hex, binary or octal literal + /// takes only the integer suffixes (#2208 — C# refuses 0x1m; f/d are hex + /// digits, so on a hex literal they never reach the table). + /// + private static bool SpellingAccepts(NumberSpelling spelling, NumericSuffixKind kind) => spelling switch + { + NumberSpelling.DecimalInteger => true, + NumberSpelling.DecimalFloat => kind == NumericSuffixKind.Float, + _ => kind == NumericSuffixKind.Integer, + }; + + /// The refusal's parenthetical: what the spelling does accept. + private static string AcceptedSuffixesNote(NumberSpelling spelling) => spelling switch + { + NumberSpelling.DecimalInteger => "", + NumberSpelling.DecimalFloat => " (a float literal accepts only the float suffixes f, d and m)", + NumberSpelling.Hexadecimal => " (a hexadecimal literal accepts only the integer suffixes u, l and ul)", + NumberSpelling.Binary => " (a binary literal accepts only the integer suffixes u, l and ul)", + _ => " (an octal literal accepts only the integer suffixes u, l and ul)", + }; /// - /// Reads the optional integer suffix after the digits of a 0x/0b/0o - /// literal, the same way reads a decimal literal's suffix: up to two - /// letters, validated against . Without this the suffix lexed as - /// a separate identifier and the statement failed with SPY0103 (#2208). + /// Reads the optional suffix after a number's digits — up to two letters — and validates it + /// against for . Returns the suffix's + /// kind, or null when no letter follows. Called by EVERY path + /// (decimal, leading-dot, hex, binary, octal) after its digits, so a suffix means the same on + /// every spelling of its kind (#2208, #2339). /// - private void ReadPrefixedIntegerSuffix(StringBuilder sb, string articleAndBase, int startLine, int startColumn) + private NumericSuffixKind? ReadNumericSuffix(StringBuilder sb, NumberSpelling spelling, int startLine, int startColumn) { if (_position >= _source.Length || !char.IsLetter(_source[_position])) - return; + return null; var suffix = _source[_position].ToString(); _position++; @@ -1017,12 +1064,13 @@ private void ReadPrefixedIntegerSuffix(StringBuilder sb, string articleAndBase, _column++; } - if (!IntegerSuffixes.Contains(suffix)) + if (!NumericSuffixes.TryGetValue(suffix, out var kind) || !SpellingAccepts(spelling, kind)) throw ReportError( - $"Invalid numeric suffix: {suffix} ({articleAndBase} literal accepts only the integer suffixes u, l and ul)", + $"Invalid numeric suffix: {suffix}{AcceptedSuffixesNote(spelling)}", startLine, startColumn, DiagnosticCodes.Lexer.InvalidNumericSuffix); sb.Append(suffix); + return kind; } private Token ReadBinaryNumber(int startLine, int startColumn, int startPosition) @@ -1074,7 +1122,7 @@ private Token ReadBinaryNumber(int startLine, int startColumn, int startPosition if (lastChar == '_') throw ReportError("Invalid number: cannot end with underscore", startLine, startColumn, DiagnosticCodes.Lexer.InvalidBinaryLiteral); - ReadPrefixedIntegerSuffix(sb, "a binary", startLine, startColumn); + ReadNumericSuffix(sb, NumberSpelling.Binary, startLine, startColumn); return CreateToken(TokenType.Integer, sb.ToString(), startLine, startColumn, startPosition); } @@ -1128,7 +1176,7 @@ private Token ReadOctalNumber(int startLine, int startColumn, int startPosition) if (lastChar == '_') throw ReportError("Invalid number: cannot end with underscore", startLine, startColumn, DiagnosticCodes.Lexer.InvalidOctalLiteral); - ReadPrefixedIntegerSuffix(sb, "an octal", startLine, startColumn); + ReadNumericSuffix(sb, NumberSpelling.Octal, startLine, startColumn); return CreateToken(TokenType.Integer, sb.ToString(), startLine, startColumn, startPosition); } From 9e6dcb2f6d52973c0d433aa3df1632fcf4ffc372 Mon Sep 17 00:00:00 2001 From: antonsynd <2648730+antonsynd@users.noreply.github.com> Date: Thu, 22 Oct 2026 02:23:00 -0400 Subject: [PATCH 09/13] docs(deviations): leading-zero decimal integers and BMP-only identifier letters are recorded deviations; the numeric twin pins 01 by the ledger, not the allowlist (refs #2300) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two DEVIATION-by-default surfaces, RECORDED by ruling (R-HT, R-HW); #2300 stays open. Refusing leading-zero integers was rejected and is not done. 1. leading-zero-decimal-integer (docs/deviations.yaml; category syntax, audience python, severity none). integer_literals.md gains "Leading zeros are digits (C#'s rule)" with an output-asserted spec block. Measured (python3 3.12.13; sharpyc run @ e7e0a0ef7, unchanged here): print(01, 0_1, 007) Sharpy: 1 1 7 print(01) / print(007) / print(0_0_7) CPython: SyntaxError: leading zeros in decimal integer literals are not permitted; use an 0o prefix for octal integers print(00, 0_0) both: 0 0 (all-zero: CPython accepts) print(01.5, 01e3) CPython: 1.5 1000.0 (a FLOAT's leading zero is fine) print(0__1) CPython: invalid decimal literal (Sharpy refuses too) The twin: the two allowlist rows 01 / 0_1 (numeric-literal-twin-allowlist.txt) are deleted; NumericLiteralTwinTests.SharpyOnly gains the row "leading-zero decimal integer" (predicate ^(?=[0-9_]*[1-9])0(?:_?[0-9])+$, the spec sentence + ledger id) and each SharpyOnly row now carries the Sharpy verdict it pins (SharpyAccepts): a trailing-dot float is refused by Sharpy and accepted by CPython, a leading-zero integer the reverse; the OVER-BROAD check runs in the matching direction. The predicate excludes floats (01.5, 01e3), all-zero integers (00, 0_0) and the malformed 0__1 / 01_, which stay compared — anchored by literal ExclusionOf asserts. SharpyOnlyRows_LexAsTheSpecSays gains 01, 0_1, 007, 0_0_7, 01L. The allowlist positive-control row that used the key "01" now uses "10" (01 is no longer a compared spelling, which would add a 5th violation to a control that counts 4) — the control's input, not an expected value. 2. identifier-letters-bmp-only (category syntax, audience python, severity error, code SPY0015). identifiers.md: "letters of the Basic Multilingual Plane — C#'s identifier alphabet (Axiom 1)". Measured: 𝒳 = 1 (U+1D4B3) CPython: binds, print(𝒳) -> 1 Sharpy: SPY0015 Unexpected character: '' — the message holds the lone high surrogate U+D835 (bytes EF BF BD on the terminal), once per line using 𝒳 été = 1 Sharpy: runs, prints 1 (control) The full-code-point message (optional courtesy) is not done. build_tools/cpython_oracle/ledger.yaml regenerated (ledger --write; --check clean: 104 deviations). Mutation (Rule 12; NumericLiteralTwin filter): M3 the lexer refuses a leading-zero decimal integer -> 347/6/0: the 5 new SharpyOnlyRows (01 0_1 007 0_0_7 01L) + SharedGrammar_AgreesWithCPython (SHARPY-ONLY "01", "0_1": the spec accepts it, Sharpy gives refused) M4 the SharpyOnly row's SharpyAccepts flipped to false -> 352/1/0: SharedGrammar red with SHARPY-ONLY + OVER-BROAD on "01" and "0_1" restored (cp + touch, rebuilt) -> green (inside the run below) Spec block positive control: "# 7 1 7" -> "# 7 1 8" -> RED integer_literals.md:20 output mismatch (expected 7 1 8, actual 7 1 7); restored -> clean. Filtered runs @ this commit's tree (measured): Compiler.Tests NumericLiteralTwin 353/0/0 Compiler.Tests Lexer|Conformance|Census|Roster 20197/0/5 check_spec_blocks.sh clean (output-asserted 2, failing 0) pytest test_cpython_oracle_ledger + test_allowlist_issue_state + test_cpython_oracle 82 passed python -m build_tools.cpython_oracle ledger --check up to date check_allowlist_issue_state.sh exit 0 Expected values changed: none (the control row key above is an input). Co-Authored-By: Claude Opus 5.5 (1M context) Claude-Session: https://claude.ai/code/session_0126pNZqmMsSbn4WFFPhpnxW --- build_tools/cpython_oracle/ledger.yaml | 16 ++++ docs/deviations.yaml | 38 +++++++++ docs/language_specification/identifiers.md | 5 +- .../integer_literals.md | 10 +++ .../Lexer/NumericLiteralTwinTests.cs | 79 ++++++++++++++----- .../Lexer/numeric-literal-twin-allowlist.txt | 3 - 6 files changed, 129 insertions(+), 22 deletions(-) diff --git a/build_tools/cpython_oracle/ledger.yaml b/build_tools/cpython_oracle/ledger.yaml index 7f9f2b5de..b9655bb9e 100644 --- a/build_tools/cpython_oracle/ledger.yaml +++ b/build_tools/cpython_oracle/ledger.yaml @@ -277,6 +277,14 @@ deviations: spec_ref: docs/language_specification/function_variadic_arguments.md existing_diagnostic: SPY0220 planned_diagnostic: SPY0472 +- id: identifier-letters-bmp-only + category: syntax + audience: python + severity: error + code: SPY0015 + spec_ref: docs/language_specification/identifiers.md + existing_diagnostic: SPY0015 + planned_diagnostic: null - id: identity-with-value-types category: operators audience: python @@ -381,6 +389,14 @@ deviations: spec_ref: docs/language_specification/function_default_parameters.md existing_diagnostic: SPY0433 planned_diagnostic: null +- id: leading-zero-decimal-integer + category: syntax + audience: python + severity: none + code: null + spec_ref: docs/language_specification/integer_literals.md + existing_diagnostic: null + planned_diagnostic: null - id: lru-cache-maxsize-saturates category: stdlib audience: python diff --git a/docs/deviations.yaml b/docs/deviations.yaml index 21bc37fa9..7739e06b4 100644 --- a/docs/deviations.yaml +++ b/docs/deviations.yaml @@ -2211,3 +2211,41 @@ deviations: t.start() t.join() print(t.ident > 0) # True + + - id: leading-zero-decimal-integer + code: null + category: syntax + audience: python + severity: none + python_behavior: "A decimal integer literal with a leading zero is a SyntaxError: `01`, `0_1` and `007` are `leading zeros in decimal integer literals are not permitted; use an 0o prefix for octal integers`. An all-zero literal (`00`, `0_0`) is accepted." + sharpy_behavior: "Leading zeros are digits (C#'s rule — C# accepts `007` as 7): `01` and `0_1` lex as the integer 1 and `007` as 7, silently. An octal literal is `0o7` in both languages, so no Python program changes meaning — a Python refusal becomes a Sharpy acceptance. Ruled R-HT (#2300, P68 numeric twin; DEVIATION-by-default under python-fidelity-scope D5 a); refusing them was rejected." + spec_ref: docs/language_specification/integer_literals.md + existing_diagnostic: null + planned_diagnostic: null + example: + python: | + print(01) # SyntaxError: leading zeros in decimal integer literals are not permitted; use an 0o prefix for octal integers + print(00, 0_0) # 0 0 + sharpy: | + def main(): + print(01, 0_1, 007) # 1 1 7 + print(00, 0_0) # 0 0 + + - id: identifier-letters-bmp-only + code: SPY0015 + category: syntax + audience: python + severity: error + python_behavior: "An identifier's letters are any Unicode code points with the XID_Start/XID_Continue property, including letters outside the Basic Multilingual Plane: `𝒳 = 1` (U+1D4B3 MATHEMATICAL SCRIPT CAPITAL X) binds `𝒳`." + sharpy_behavior: "Identifier letters are the letters of the Basic Multilingual Plane — C#'s identifier alphabet, whose letter test is per UTF-16 code unit (Axiom 1). `été = 1` is an identifier; `𝒳 = 1` is refused with SPY0015 `Unexpected character: '\\uD835'`, the message naming the lone high surrogate of the pair (it prints as U+FFFD `'�'` in a terminal). Ruled R-HW (#2300, P68 record round)." + spec_ref: docs/language_specification/identifiers.md + existing_diagnostic: SPY0015 + planned_diagnostic: null + example: + python: | + 𝒳 = 1 + print(𝒳) # 1 + sharpy: | + def main(): + 𝒳 = 1 # SPY0015: Unexpected character: '�' (the lone high surrogate U+D835) + # control — a BMP letter: `été = 1` then `print(été)` prints 1 diff --git a/docs/language_specification/identifiers.md b/docs/language_specification/identifiers.md index 0f465a94d..5d9be7675 100644 --- a/docs/language_specification/identifiers.md +++ b/docs/language_specification/identifiers.md @@ -16,7 +16,10 @@ digit ::= '0'..'9' - Can contain letters, digits, and underscores - Case-sensitive: `myVar`, `myvar`, and `MYVAR` are different identifiers - Cannot be a keyword -- Unicode letters are allowed but discouraged for interop +- Unicode letters are allowed but discouraged for interop. They are the letters of the Basic + Multilingual Plane — C#'s identifier alphabet (Axiom 1): `été` is an identifier, while a letter + outside the BMP, such as `𝒳` (U+1D4B3), is refused with SPY0015 although Python accepts it + (deviation `identifier-letters-bmp-only` in `docs/deviations.yaml`) ## Examples diff --git a/docs/language_specification/integer_literals.md b/docs/language_specification/integer_literals.md index 8da886db9..9a1b772e6 100644 --- a/docs/language_specification/integer_literals.md +++ b/docs/language_specification/integer_literals.md @@ -12,6 +12,16 @@ million = 1_000_000 billion = 1_000_000_000 ``` +Leading zeros are digits (C#'s rule): `007` is the decimal integer 7 and `0_1` is 1. Python refuses +both (`leading zeros in decimal integer literals are not permitted`); an octal literal is spelled +`0o7` in both languages (deviation `leading-zero-decimal-integer` in `docs/deviations.yaml`). + + +```python +def main(): + print(007, 0_1, 0o7) # 7 1 7 +``` + ## Type Inference - Integer literals without a suffix are inferred as `int` (32-bit, `System.Int32`) by default diff --git a/src/Sharpy.Compiler.Tests/Lexer/NumericLiteralTwinTests.cs b/src/Sharpy.Compiler.Tests/Lexer/NumericLiteralTwinTests.cs index fa1c0c47d..d506a5f95 100644 --- a/src/Sharpy.Compiler.Tests/Lexer/NumericLiteralTwinTests.cs +++ b/src/Sharpy.Compiler.Tests/Lexer/NumericLiteralTwinTests.cs @@ -34,9 +34,11 @@ namespace Sharpy.Compiler.Tests.Lexer; /// CPython like any other. /// Sharpy-only syntax is never compared with CPython. The suffix roster and the /// imaginary j are excluded by construction (no generated spelling contains a roster -/// letter after a number); the trailing-dot floats are excluded by , -/// a literal (predicate, spec sentence) table whose cells are still CHECKED — Sharpy must refuse -/// them, and CPython must accept them (an exclusion CPython agrees with is over-broad). Every +/// letter after a number); the trailing-dot floats and the leading-zero decimal integers are +/// excluded by , a literal (predicate, spec sentence, Sharpy verdict) +/// table whose cells are still CHECKED — Sharpy must give the row's verdict (refuse a +/// trailing-dot float, accept 01), and CPython the opposite one (an exclusion CPython +/// agrees with is over-broad). Every /// excluded family is pinned to Sharpy's own token or diagnostic by /// . /// Ratchet. A disagreement fails unless its spelling is a row of @@ -105,14 +107,23 @@ private static readonly (string Name, string Text, TokenType[] Tokens)[] Separat /// /// Shared-grammar spellings whose Sharpy verdict is fixed by the spec, not by CPython. Each - /// cell matching a predicate must be refused by Sharpy AND accepted by CPython (else the - /// exclusion is over-broad); pins the tokens. + /// cell matching a predicate must get the row's Sharpy verdict (SharpyAccepts) AND the + /// opposite verdict from CPython (else the exclusion is over-broad); + /// pins the tokens. /// - private static readonly (string Name, Regex Predicate, string SpecSentence)[] SharpyOnly = + private static readonly (string Name, Regex Predicate, string SpecSentence, bool SharpyAccepts)[] SharpyOnly = { ("trailing-dot float", new Regex(@"^[0-9](?:_?[0-9])*\.(?:[eE][+-]?[0-9](?:_?[0-9])*)?$", RegexOptions.CultureInvariant), - "float_literals.md: \"Invalid - trailing decimal point without digit … # invalid = 5. # ERROR: use 5.0 instead\""), + "float_literals.md: \"Invalid - trailing decimal point without digit … # invalid = 5. # ERROR: use 5.0 instead\"", + false), + // A decimal INTEGER only: CPython accepts leading zeros in a float (01.5, 01e3), and an + // all-zero integer (00, 0_0), so neither is excluded. + ("leading-zero decimal integer", + new Regex(@"^(?=[0-9_]*[1-9])0(?:_?[0-9])+$", RegexOptions.CultureInvariant), + "integer_literals.md: \"Leading zeros are digits (C#'s rule): 007 is the decimal integer 7 and 0_1 is 1\" — " + + "docs/deviations.yaml leading-zero-decimal-integer (R-HT, #2300)", + true), }; private static IEnumerable BaseParts() @@ -216,12 +227,14 @@ private static List Spellings() return all; } - private static string? ExclusionOf(string text) + private static string? ExclusionOf(string text) => SharpyOnlyRowOf(text)?.Name; + + private static (string Name, Regex Predicate, string SpecSentence, bool SharpyAccepts)? SharpyOnlyRowOf(string text) { - foreach (var (name, predicate, _) in SharpyOnly) + foreach (var row in SharpyOnly) { - if (predicate.IsMatch(text)) - return name; + if (row.Predicate.IsMatch(text)) + return row; } return null; } @@ -439,6 +452,17 @@ public void Enumeration_HoldsTheIssuesRepresentativeCells_AndNoSharpyOnlyLetter( Assert.Null(ExclusionOf("1._5")); Assert.Null(ExclusionOf("1.5.")); Assert.Null(ExclusionOf("0xFF.")); + + Assert.Equal("leading-zero decimal integer", ExclusionOf("01")); + Assert.Equal("leading-zero decimal integer", ExclusionOf("0_1")); + Assert.Equal("leading-zero decimal integer", ExclusionOf("007")); + Assert.Null(ExclusionOf("00")); // all zeros: CPython accepts it, so it stays compared + Assert.Null(ExclusionOf("0_0")); + Assert.Null(ExclusionOf("0__1")); // consecutive separators: both refuse, compared + Assert.Null(ExclusionOf("01_")); + Assert.Null(ExclusionOf("01.5")); // a float: CPython accepts its leading zero + Assert.Null(ExclusionOf("01e3")); + Assert.Null(ExclusionOf("10")); } [Fact] @@ -451,7 +475,7 @@ public void Allowlist_EveryRowCitesAnIssue_AndNamesAComparedSpelling() // Positive control: the rules fire on rows built to break them. var broken = AllowlistShapeViolations(new[] { - new AllowlistRow(1, "01", "int:1", "# no cite here"), + new AllowlistRow(1, "10", "int:10", "# no cite here"), new AllowlistRow(2, "0x1p3", "int:8", "# #2300 hex float is not in the matrix"), new AllowlistRow(3, "12.", "refused", "# #2300 Sharpy-only spelling"), new AllowlistRow(4, "00", "", "# #2300 no pinned verdict"), @@ -511,14 +535,24 @@ public void SharedGrammar_AgreesWithCPython() failures.Add($"FOLLOW {Show(s.Text)} + {name}: bare {sharpy.Show()} but followed {followed.Show()} [{s.Origin}]"); } - var exclusion = ExclusionOf(s.Text); - if (exclusion is not null) + if (SharpyOnlyRowOf(s.Text) is { } only) { + string exclusion = only.Name; sharpyOnly++; - if (sharpy.Ok) - failures.Add($"SHARPY-ONLY {Show(s.Text)} ({exclusion}): the spec refuses it, Sharpy lexes {sharpy.Show()}"); - if (!cpython.Ok) - failures.Add($"OVER-BROAD {Show(s.Text)} ({exclusion}): CPython refuses it too ({cpython.Detail}) — compare it instead of excluding it"); + if (only.SharpyAccepts) + { + if (!sharpy.Ok) + failures.Add($"SHARPY-ONLY {Show(s.Text)} ({exclusion}): the spec accepts it, Sharpy gives {sharpy.Show()}"); + if (cpython.Ok) + failures.Add($"OVER-BROAD {Show(s.Text)} ({exclusion}): CPython accepts it too ({cpython.Detail}) — compare it instead of excluding it"); + } + else + { + if (sharpy.Ok) + failures.Add($"SHARPY-ONLY {Show(s.Text)} ({exclusion}): the spec refuses it, Sharpy lexes {sharpy.Show()}"); + if (!cpython.Ok) + failures.Add($"OVER-BROAD {Show(s.Text)} ({exclusion}): CPython refuses it too ({cpython.Detail}) — compare it instead of excluding it"); + } continue; } @@ -661,6 +695,15 @@ SpellingKind.Hex when s is not ("m" or "M") => $"Integer '{value}{s}'", Add("1.e5", "Integer '1', Dot '.', Identifier 'e5'"); foreach (var s in IntegerSuffixes.Concat(FloatSuffixes).Concat(InvalidSuffixes)) Add("5." + s, $"Integer '5', Dot '.', Identifier '{s}'"); + + // integer_literals.md: "Leading zeros are digits (C#'s rule): 007 is the decimal integer 7 and + // 0_1 is 1" — docs/deviations.yaml leading-zero-decimal-integer (R-HT, #2300); CPython refuses + // them, so they left the twin's allowlist for this table. + Add("01", "Integer '01'"); + Add("0_1", "Integer '01'"); + Add("007", "Integer '007'"); + Add("0_0_7", "Integer '007'"); + Add("01L", "Integer '01L'"); return data; } diff --git a/src/Sharpy.Compiler.Tests/Lexer/numeric-literal-twin-allowlist.txt b/src/Sharpy.Compiler.Tests/Lexer/numeric-literal-twin-allowlist.txt index 5688ba16b..b6e5d8484 100644 --- a/src/Sharpy.Compiler.Tests/Lexer/numeric-literal-twin-allowlist.txt +++ b/src/Sharpy.Compiler.Tests/Lexer/numeric-literal-twin-allowlist.txt @@ -12,6 +12,3 @@ # fails; a row whose spelling now agrees fails as stale (drain on fix). A row leaves only by an # owner ruling: a deviation-ledger entry (the spelling moves to the twin's Sharpy-only table) # or a promoted lexer-defect issue (the row re-cites it and drains when the fix lands). - -01 => int:1 # #2300 leading-zero decimal integer — CPython refuses ("leading zeros in decimal integer literals are not permitted"), Sharpy lexes Integer 1; spec silent; DEVIATION-by-default (python-fidelity-scope D5 a) until ruled at /batch-issues -0_1 => int:1 # #2300 the same leading-zero decimal integer cell spelled with a digit separator From fed8a7c81f61303f1e05270c6b62b341b865903c Mon Sep 17 00:00:00 2001 From: antonsynd <2648730+antonsynd@users.noreply.github.com> Date: Thu, 22 Oct 2026 03:32:52 -0400 Subject: [PATCH 10/13] =?UTF-8?q?test(infra):=20a=20python=20twin=20that?= =?UTF-8?q?=20cannot=20reach=20its=20oracle=20reports=20Skipped,=20never?= =?UTF-8?q?=20Passed=20=E2=80=94=20one=20RequireOrSkip=20gate=20+=20a=20SK?= =?UTF-8?q?IP:=20census=20(refs=20#2342)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Contract (verification contract §2): a test that cannot reach its oracle reports SKIPPED, never PASSED. Ruling R-HZ (a). - Xunit.SkippableFact 1.5.85 (published 2026-08-24, nuget.org; deps Validation 2.6.68 + xunit.extensibility.execution >= 2.4.0) added to Sharpy.TestInfrastructure (SkipException) and Sharpy.Compiler.Tests (the attributes) — the repo pins versions per csproj, no central file. - ONE helper, PythonInterpreter.RequireOrSkip(minMinor: 12) (+ an overload with the candidates and the required flag explicit): located -> returned; absent -> SkipException; absent + SHARPY_REQUIRE_PYTHON_ORACLE=1 -> PythonOracleMissingException naming the env var and the candidates. - The eight write-SKIP:-and-return sites (re-grepped: exactly these eight in every src/*Tests* and TestInfrastructure .cs) now open with RequireOrSkip under [SkippableFact]/[SkippableTheory]: PythonInterpreterTests x2, NumericLiteralTwinTests.SharedGrammar, FormatSpecDifferentialTests x2, DifferentialExecutionTests x1, CPythonDifferentialParseTests x2 (one a Theory, 14 rows). A site whose python is present but whose in-repo oracle script is missing now FAILS (FileNotFoundException) — a broken checkout is not a skip. - Census guard PythonOracleSkipCensusTests (Roslyn syntax scan of every .cs under src/*.Tests and src/Sharpy.TestInfrastructure, minus bin/obj/ TestFixtures): no string starting "SKIP:"; a [Fact]/[Theory] calling RequireOrSkip (outside a lambda) must be [SkippableFact]/[SkippableTheory] (under a plain [Fact] the SkipException FAILS); >= 8 skippable callers (literal floor) and > 500 files scanned (anti-vacuity). Positive control Census_FiresOnEachForbiddenShape: a built source with a SKIP: write, an interpolated $"SKIP: {why}", a [Theory] calling RequireOrSkip -> lines 9/15/18 flagged; the [SkippableFact]/[SkippableTheory] and the Assert.Throws(() => RequireOrSkip(...)) shapes pass. - PythonInterpreterTests gains the two verdict tests RequireOrSkip_RequiredAndMissing_Fails_NamingTheOracle and RequireOrSkip_NotRequiredAndMissing_Skips (unresolvable candidate list, flag explicit — never an in-process env override). - Sibling seam found by the filtered run: CiFilterCoverageConformanceTests enumerated tests by attribute NAME ("FactAttribute"/"TheoryAttribute"), so a [SkippableFact] dropped out of its universe — FormatSpecDifferential- Tests (both its tests now skippable) went red as "FQN filter substring matches no test", and the CI-selection check would have silently stopped covering every skippable test. It now matches by derivation (typeof(FactAttribute).IsAssignableFrom). Direction: red (EveryFilterSubstring_MatchesAtLeastOneTest, 8x "FormatSpecDifferential") before the fix, green after (run below). Mutation / direction record — the eight sites (24 test cases at base, 26 after: + the 2 verdict tests) run in a CHILD `dotnet test` whose PATH is scrubbed to a dotnet shim + /usr/bin:/bin:/usr/sbin:/sbin (python3 there is 3.9.6 < 3.12); the env var is set only in the child (measured): base @ a60d16bc5 this commit python present 24/0/0 26/0/0 (positive control: all 21 site cases pass both) PATH scrubbed 24/0/0 (vacuous) 5/0/21 (every site Skipped, reason names the env var) scrubbed + env=1 21/3/0 (only the 3 5/21/0 (every site fails: interpreter sites) PythonOracleMissingException) restored (python present again) = the 26/0/0 row. Census mutation: RunBatch's [SkippableFact] -> [Fact] and a `Output.WriteLine("SKIP: mutant")` at both FormatSpec sites -> 1/1/0, the census red listing exactly 3 violations (PythonInterpreterTests.cs:82 under [Fact], FormatSpecDifferentialTests.cs:118 and :267 SKIP: strings), the positive control green; restored (cp + touch, rebuilt) -> green. Filtered runs @ this commit's tree (measured): Compiler.Tests Lexer|Conformance|Census|Roster|Differential before the CiFilterCoverage fix 20384/1/5 (the red above) Compiler.Tests Conformance|PythonInterpreterTests after it 17496/0/0 dotnet format whitespace --verify-no-changes clean (measured before the 4-line CiFilterCoverage edit) Expected values changed: none. Co-Authored-By: Claude Opus 5.5 (1M context) Claude-Session: https://claude.ai/code/session_0126pNZqmMsSbn4WFFPhpnxW --- .../CiFilterCoverageConformanceTests.cs | 5 +- .../PythonOracleSkipCensusTests.cs | 182 ++++++++++++++++++ .../Lexer/NumericLiteralTwinTests.cs | 18 +- .../Lexer/PythonInterpreterTests.cs | 40 ++-- .../DifferentialExecutionTests.cs | 12 +- .../FormatSpecDifferentialTests.cs | 23 +-- .../Parser/CPythonDifferentialParseTests.cs | 23 +-- .../Sharpy.Compiler.Tests.csproj | 2 + .../Oracles/PythonInterpreter.cs | 27 ++- .../Sharpy.TestInfrastructure.csproj | 2 + 10 files changed, 272 insertions(+), 62 deletions(-) create mode 100644 src/Sharpy.Compiler.Tests/Conformance/PythonOracleSkipCensusTests.cs diff --git a/src/Sharpy.Compiler.Tests/Conformance/CiFilterCoverageConformanceTests.cs b/src/Sharpy.Compiler.Tests/Conformance/CiFilterCoverageConformanceTests.cs index 5c8437463..9373cdf37 100644 --- a/src/Sharpy.Compiler.Tests/Conformance/CiFilterCoverageConformanceTests.cs +++ b/src/Sharpy.Compiler.Tests/Conformance/CiFilterCoverageConformanceTests.cs @@ -460,8 +460,11 @@ private static List ReflectTests() var classTraits = CollectTraits(type); foreach (var method in type.GetMethods(BindingFlags.Public | BindingFlags.Instance)) { + // Every xunit test attribute derives from FactAttribute ([Theory], and the + // [SkippableFact]/[SkippableTheory] of the python twins, #2342) — match by + // derivation, never by name, or a subclassed test drops out of the universe. if (!method.GetCustomAttributesData().Any(a => - a.AttributeType.Name is "FactAttribute" or "TheoryAttribute")) + typeof(FactAttribute).IsAssignableFrom(a.AttributeType))) continue; var merged = MergeTraits(classTraits, CollectTraits(method)); diff --git a/src/Sharpy.Compiler.Tests/Conformance/PythonOracleSkipCensusTests.cs b/src/Sharpy.Compiler.Tests/Conformance/PythonOracleSkipCensusTests.cs new file mode 100644 index 000000000..137e47345 --- /dev/null +++ b/src/Sharpy.Compiler.Tests/Conformance/PythonOracleSkipCensusTests.cs @@ -0,0 +1,182 @@ +using Microsoft.CodeAnalysis; +using Microsoft.CodeAnalysis.CSharp; +using Microsoft.CodeAnalysis.CSharp.Syntax; +using Sharpy.TestInfrastructure.Oracles; +using Xunit; + +namespace Sharpy.Compiler.Tests.Conformance; + +/// +/// The census behind #2342 (R-HZ; verification contract §2): a test that cannot reach its python +/// oracle reports SKIPPED, never PASSED. Two rules over every C# source of every test project +/// (src/*.Tests/ and src/Sharpy.TestInfrastructure/): +/// +/// no string (plain or interpolated) starts with SKIP: — the write-and-return idiom +/// that xunit counts Passed; +/// a [Fact]/[Theory] that calls +/// is a [SkippableFact]/[SkippableTheory] — under a plain [Fact] the +/// helper's SkipException is a FAILURE, not a skip. +/// +/// Syntax-only (Roslyn parse, no compilation), so the scan cannot be blinded by a build setting. +/// +public class PythonOracleSkipCensusTests +{ + /// + /// The python-twin tests that open with RequireOrSkip at #2342 (the eight former + /// SKIP: sites plus the helper's own located-verdict test): a floor, anchored as a + /// literal so a scan that finds no file cannot pass vacuously. + /// + private const int RequireOrSkipCallerFloor = 8; + + /// Spelled in two pieces so this file's own literals do not match the rule they state. + private const string SkipMarker = "SKIP" + ":"; + + private sealed record Violation(string File, int Line, string Rule); + + private static IEnumerable TestSourceFiles(string repoRoot) + { + var src = Path.Combine(repoRoot, "src"); + var roots = Directory.GetDirectories(src) + .Where(d => + { + var name = Path.GetFileName(d); + return name.EndsWith(".Tests", StringComparison.Ordinal) || name == "Sharpy.TestInfrastructure"; + }); + foreach (var root in roots) + { + foreach (var file in Directory.EnumerateFiles(root, "*.cs", SearchOption.AllDirectories)) + { + var rel = Path.GetRelativePath(root, file).Replace('\\', '/'); + // Build output and the fixtures' generated C# snapshots are not test sources. + if (rel.StartsWith("bin/", StringComparison.Ordinal) || rel.StartsWith("obj/", StringComparison.Ordinal) + || rel.Contains("/TestFixtures/", StringComparison.Ordinal) || rel.StartsWith("Integration/TestFixtures/", StringComparison.Ordinal)) + { + continue; + } + yield return file; + } + } + } + + private static string StringText(SyntaxNode node) => node switch + { + LiteralExpressionSyntax lit when lit.IsKind(SyntaxKind.StringLiteralExpression) => lit.Token.ValueText, + // The text before the first hole: `$"SKIP: {why}"` starts with "SKIP: ". + InterpolatedStringExpressionSyntax interp => string.Concat( + interp.Contents.TakeWhile(c => c is InterpolatedStringTextSyntax) + .Cast().Select(t => t.TextToken.ValueText)), + _ => "", + }; + + private static bool HasAttribute(MethodDeclarationSyntax method, params string[] names) => + method.AttributeLists.SelectMany(l => l.Attributes).Any(a => + { + var name = a.Name.ToString(); + var simple = name.Contains('.') ? name[(name.LastIndexOf('.') + 1)..] : name; + return names.Any(n => simple == n || simple == n + "Attribute"); + }); + + /// Scans one source; returns the violations and how many test methods call RequireOrSkip. + private static (List Violations, int RequireOrSkipCallers) Scan(string path, string source) + { + var violations = new List(); + var root = CSharpSyntaxTree.ParseText(source, path: path).GetRoot(); + + foreach (var node in root.DescendantNodes().Where(n => n is LiteralExpressionSyntax or InterpolatedStringExpressionSyntax)) + { + if (StringText(node).TrimStart().StartsWith(SkipMarker, StringComparison.Ordinal)) + { + int line = node.GetLocation().GetLineSpan().StartLinePosition.Line + 1; + violations.Add(new Violation(path, line, "a 'SKIP:' string — write-and-return counts Passed; call PythonInterpreter.RequireOrSkip")); + } + } + + int callers = 0; + foreach (var method in root.DescendantNodes().OfType()) + { + // The gate is a statement of the test body; a call inside a lambda (Assert.Throws(() => + // RequireOrSkip(...)), the helper's own verdict tests) runs under the lambda's caller. + bool callsHelper = method.DescendantNodes().OfType().Any(inv => + (inv.Expression is MemberAccessExpressionSyntax { Name.Identifier.ValueText: "RequireOrSkip" } + or IdentifierNameSyntax { Identifier.ValueText: "RequireOrSkip" }) + && !inv.Ancestors().TakeWhile(a => a != method).Any(a => a is AnonymousFunctionExpressionSyntax)); + if (!callsHelper) + continue; + if (HasAttribute(method, "SkippableFact", "SkippableTheory")) + { + callers++; + } + else if (HasAttribute(method, "Fact", "Theory")) + { + int line = method.Identifier.GetLocation().GetLineSpan().StartLinePosition.Line + 1; + violations.Add(new Violation(path, line, + $"{method.Identifier.ValueText} calls RequireOrSkip under [Fact]/[Theory] — its SkipException FAILS there; use [SkippableFact]/[SkippableTheory]")); + } + } + return (violations, callers); + } + + [Fact] + public void NoTestWritesSkipAndReturns_AndEveryRequireOrSkipCallerIsSkippable() + { + var sln = PythonInterpreter.FindRepoFile("sharpy.sln") + ?? throw new FileNotFoundException("sharpy.sln not found above the test binary."); + var repoRoot = Path.GetDirectoryName(sln)!; + + var violations = new List(); + int files = 0, callers = 0; + foreach (var file in TestSourceFiles(repoRoot)) + { + files++; + var (v, c) = Scan(Path.GetRelativePath(repoRoot, file), File.ReadAllText(file)); + violations.AddRange(v); + callers += c; + } + + Assert.True(files > 500, $"the census scanned only {files} test source files — the repository walk is broken."); + Assert.True(violations.Count == 0, + $"{violations.Count} python-oracle skip violation(s) (#2342):\n" + + string.Join("\n", violations.Select(v => $" {v.File}:{v.Line}: {v.Rule}"))); + Assert.True(callers >= RequireOrSkipCallerFloor, + $"only {callers} [SkippableFact]/[SkippableTheory] tests call RequireOrSkip (floor {RequireOrSkipCallerFloor}): a python twin lost its gate, or the scan went blind."); + } + + [Fact] + public void Census_FiresOnEachForbiddenShape() + { + // Positive control: each rule fires on a source built to break it, and the allowed shape passes. + const string Broken = """ + public class T + { + [Fact] + public void A() + { + var python = Locate(); + if (python is null) + { + _output.WriteLine("SKIP: no python"); + return; + } + } + + [Fact] + public void B() { Output.WriteLine($"SKIP: {why}; nothing ran"); } + + [Theory] + public void C(string s) { PythonInterpreter.RequireOrSkip(minMinor: 12); } + + [SkippableFact] + public void D() { PythonInterpreter.RequireOrSkip(minMinor: 12); } + + [SkippableTheory] + public void E(string s) { RequireOrSkip(); } + + [Fact] + public void F() { Assert.Throws(() => PythonInterpreter.RequireOrSkip(none, required: false)); } + } + """; + var (violations, callers) = Scan("control.cs", Broken); + Assert.Equal(new[] { 9, 15, 18 }, violations.Select(v => v.Line).ToArray()); + Assert.Equal(2, callers); + } +} diff --git a/src/Sharpy.Compiler.Tests/Lexer/NumericLiteralTwinTests.cs b/src/Sharpy.Compiler.Tests/Lexer/NumericLiteralTwinTests.cs index d506a5f95..0be98d105 100644 --- a/src/Sharpy.Compiler.Tests/Lexer/NumericLiteralTwinTests.cs +++ b/src/Sharpy.Compiler.Tests/Lexer/NumericLiteralTwinTests.cs @@ -46,7 +46,7 @@ namespace Sharpy.Compiler.Tests.Lexer; /// disagreement whose Sharpy verdict differs from the row's pin fails. A row without a /// #NNNN cite fails, a row naming a spelling the matrix does not compare fails, and a row /// whose spelling now agrees fails as stale (drain on fix). -/// Python absent: a loud skip, unless SHARPY_REQUIRE_PYTHON_ORACLE=1 (CI's +/// Python absent: reported Skipped (, #2342), unless SHARPY_REQUIRE_PYTHON_ORACLE=1 (CI's /// Compiler shards), where it fails. /// public class NumericLiteralTwinTests @@ -483,7 +483,7 @@ public void Allowlist_EveryRowCitesAnIssue_AndNamesAComparedSpelling() Assert.Equal(4, broken.Count); } - [Fact] + [SkippableFact] public void SharedGrammar_AgreesWithCPython() { var stopwatch = Stopwatch.StartNew(); @@ -493,16 +493,10 @@ public void SharedGrammar_AgreesWithCPython() var shapeViolations = AllowlistShapeViolations(allowlist, compared); Assert.True(shapeViolations.Count == 0, string.Join("\n", shapeViolations)); - var python = PythonInterpreter.Locate(); - var script = PythonInterpreter.FindRepoFile("build_tools", "differential_lex", "number_oracle.py"); - if (python is null || script is null) - { - string why = python is null ? "no Python 3.12+ interpreter (python3/python) on PATH" : "build_tools/differential_lex/number_oracle.py not found"; - if (PythonInterpreter.Required) - Assert.Fail($"{why}, and {PythonInterpreter.RequireEnvironmentVariable}=1 makes the numeric twin's oracle required."); - _output.WriteLine($"SKIP: {why}; the numeric twin did not run. Set {PythonInterpreter.RequireEnvironmentVariable}=1 to make this a failure."); - return; - } + var python = PythonInterpreter.RequireOrSkip(minMinor: 12); + // The oracle script is IN the repository: its absence is a broken checkout, never a skip. + var script = PythonInterpreter.FindRepoFile("build_tools", "differential_lex", "number_oracle.py") + ?? throw new FileNotFoundException("build_tools/differential_lex/number_oracle.py not found under the repository root."); var requests = spellings.Select((s, i) => JsonSerializer.Serialize(new { id = i, source = s.Text })); var replies = python.RunBatch(script, requests); diff --git a/src/Sharpy.Compiler.Tests/Lexer/PythonInterpreterTests.cs b/src/Sharpy.Compiler.Tests/Lexer/PythonInterpreterTests.cs index 853022a31..7d8ef5ec6 100644 --- a/src/Sharpy.Compiler.Tests/Lexer/PythonInterpreterTests.cs +++ b/src/Sharpy.Compiler.Tests/Lexer/PythonInterpreterTests.cs @@ -41,14 +41,33 @@ public void Locate_NotRequiredAndMissing_ReturnsNull() } [Fact] + public void RequireOrSkip_RequiredAndMissing_Fails_NamingTheOracle() + { + var ex = Assert.Throws( + () => PythonInterpreter.RequireOrSkip(NoSuchInterpreter, required: true)); + Assert.Contains(PythonInterpreter.RequireEnvironmentVariable, ex.Message); + Assert.Contains(NoSuchInterpreter[0], ex.Message); + } + + [Fact] + public void RequireOrSkip_NotRequiredAndMissing_Skips() + { + // The skip verdict is a SkipException, which [SkippableFact] reports as Skipped — never a + // silent return that xunit counts Passed (#2342). + var ex = Assert.Throws( + () => PythonInterpreter.RequireOrSkip(NoSuchInterpreter, required: false)); + Assert.Contains(NoSuchInterpreter[0], ex.Message); + Assert.Contains(PythonInterpreter.RequireEnvironmentVariable, ex.Message); + } + + [SkippableFact] public void TryLocate_FindsAPinnedInterpreter_AndRefusesAnOlderMinimum() { - var python = PythonInterpreter.Locate(PythonInterpreter.DefaultCandidates, PythonInterpreter.Required); - if (python is null) - { - _output.WriteLine("SKIP: no Python 3.12+ interpreter (python3/python) on PATH; set SHARPY_REQUIRE_PYTHON_ORACLE=1 to make this a failure."); - return; - } + var python = PythonInterpreter.RequireOrSkip(minMinor: 12); + + // The located verdict: the same interpreter, with the required flag either way, is returned. + Assert.Equal(python.Executable, PythonInterpreter.RequireOrSkip(new[] { python.Executable }, required: true).Executable); + Assert.Equal(python.Executable, PythonInterpreter.RequireOrSkip(new[] { python.Executable }, required: false).Executable); _output.WriteLine($"python oracle: {python.Executable} → {python.Version}"); Assert.Matches(@"^Python \d+\.\d+", python.Version); @@ -59,15 +78,10 @@ public void TryLocate_FindsAPinnedInterpreter_AndRefusesAnOlderMinimum() Assert.NotNull(PythonInterpreter.TryLocate(new[] { python.Executable }, minMinor: python.Minor)); } - [Fact] + [SkippableFact] public void RunBatch_EchoesStdinLines_AndReportsStderrOnFailure() { - var python = PythonInterpreter.Locate(PythonInterpreter.DefaultCandidates, PythonInterpreter.Required); - if (python is null) - { - _output.WriteLine("SKIP: no Python 3.12+ interpreter (python3/python) on PATH."); - return; - } + var python = PythonInterpreter.RequireOrSkip(minMinor: 12); string dir = Path.Combine(Path.GetTempPath(), $"sharpy-p68-pyinterp-{Guid.NewGuid():N}"); Directory.CreateDirectory(dir); diff --git a/src/Sharpy.Compiler.Tests/Properties/Differential/DifferentialExecutionTests.cs b/src/Sharpy.Compiler.Tests/Properties/Differential/DifferentialExecutionTests.cs index 2607fab70..5cb4adc9a 100644 --- a/src/Sharpy.Compiler.Tests/Properties/Differential/DifferentialExecutionTests.cs +++ b/src/Sharpy.Compiler.Tests/Properties/Differential/DifferentialExecutionTests.cs @@ -9,6 +9,7 @@ using Sharpy.Compiler.Tests.Properties.Generators; using Sharpy.Compiler.Tests.Properties.Generators.Typed; using Sharpy.TestInfrastructure.Integration; +using Sharpy.TestInfrastructure.Oracles; using Xunit; using Xunit.Abstractions; @@ -151,15 +152,12 @@ private static Module ParseForSubset(string source) return module; } - [Fact] + [SkippableFact] public void DifferentialExecution_SharedSubset_MatchesCPython() { - var oracle = PythonExecOracle.TryLocate(); - if (oracle is null) - { - Output.WriteLine("SKIP: python3 >= 3.12 and run_programs.py not both available."); - return; - } + PythonInterpreter.RequireOrSkip(minMinor: 12); + var oracle = PythonExecOracle.TryLocate() + ?? throw new FileNotFoundException("python3 >= 3.12 is present but build_tools/differential_exec/run_programs.py was not found."); var sw = Stopwatch.StartNew(); int limit = ReadIntEnv("DIFF_EXEC_LIMIT", FullPool); diff --git a/src/Sharpy.Compiler.Tests/Properties/Differential/FormatSpecDifferentialTests.cs b/src/Sharpy.Compiler.Tests/Properties/Differential/FormatSpecDifferentialTests.cs index 7a004e752..ccf52c7ab 100644 --- a/src/Sharpy.Compiler.Tests/Properties/Differential/FormatSpecDifferentialTests.cs +++ b/src/Sharpy.Compiler.Tests/Properties/Differential/FormatSpecDifferentialTests.cs @@ -5,6 +5,7 @@ using System.Text.RegularExpressions; using Sharpy.Compiler.Diagnostics; using Sharpy.TestInfrastructure.Integration; +using Sharpy.TestInfrastructure.Oracles; using Xunit; using Xunit.Abstractions; @@ -110,15 +111,12 @@ private enum Column private static string ColumnName(Column column) => column == Column.Engine ? "engine" : "static"; - [Fact] + [SkippableFact] public void FormatSpec_DifferentialSweep_MatchesCPython() { - var oracle = PyOracle.TryLocate(); - if (oracle is null) - { - Output.WriteLine("SKIP: python3 >= 3.12 and run_programs.py not both available."); - return; - } + PythonInterpreter.RequireOrSkip(minMinor: 12); + var oracle = PyOracle.TryLocate() + ?? throw new FileNotFoundException("python3 >= 3.12 is present but build_tools/differential_exec/run_programs.py was not found."); var sw = Stopwatch.StartNew(); var cells = GenerateCells(TargetCells); @@ -261,15 +259,12 @@ public void FormatSpec_DifferentialSweep_MatchesCPython() /// rendering (template_strings.md), so its python program is the f-string's. No allowlist row may /// name a grammar:: key: the stratum is total. /// - [Fact] + [SkippableFact] public void FormatSpec_GrammarTotalityStratum_MatchesCPython() { - var oracle = PyOracle.TryLocate(); - if (oracle is null) - { - Output.WriteLine("SKIP: python3 >= 3.12 and run_programs.py not both available."); - return; - } + PythonInterpreter.RequireOrSkip(minMinor: 12); + var oracle = PyOracle.TryLocate() + ?? throw new FileNotFoundException("python3 >= 3.12 is present but build_tools/differential_exec/run_programs.py was not found."); var cells = GrammarTotalityStratum().ToList(); // Anchor to literals: 22 spec cells x 4 routes + 17 field cells, each in two columns. diff --git a/src/Sharpy.Compiler.Tests/Properties/Parser/CPythonDifferentialParseTests.cs b/src/Sharpy.Compiler.Tests/Properties/Parser/CPythonDifferentialParseTests.cs index 32f840ac1..f34bc6233 100644 --- a/src/Sharpy.Compiler.Tests/Properties/Parser/CPythonDifferentialParseTests.cs +++ b/src/Sharpy.Compiler.Tests/Properties/Parser/CPythonDifferentialParseTests.cs @@ -6,6 +6,7 @@ using CsCheck; using Sharpy.Compiler.Parser.Ast; using Sharpy.Compiler.Tests.Properties.Generators; +using Sharpy.TestInfrastructure.Oracles; using Xunit; using Xunit.Abstractions; @@ -58,15 +59,12 @@ public CPythonDifferentialParseTests(ITestOutputHelper output) }); }); - [Fact] + [SkippableFact] public void SharedSubset_SharpyAndCPythonAgree() { - var python = PythonOracle.TryLocate(); - if (python is null) - { - _output.WriteLine("SKIP: python3 >= 3.12 and compare_ast.py not both available."); - return; - } + PythonInterpreter.RequireOrSkip(minMinor: 12); + var python = PythonOracle.TryLocate() + ?? throw new FileNotFoundException("python3 >= 3.12 is present but build_tools/differential_parse/compare_ast.py was not found."); List samples = CollectSubsetSamples(targetCount: 150, maxCandidates: 1500); Assert.True(samples.Count > 50, @@ -143,7 +141,7 @@ public void SharedSubset_SharpyAndCPythonAgree() /// skeleton. Pins the exact cases the generator covers randomly so a regression fails loudly, /// and doubles as a fast oracle smoke test independent of generation randomness. /// - [Theory] + [SkippableTheory] [InlineData("f(lambda x: x[0], (a, b))")] [InlineData("first(lambda t: t[0], xs)")] [InlineData("x[a, b:c]")] @@ -161,12 +159,9 @@ public void SharedSubset_SharpyAndCPythonAgree() [InlineData("x[lambda a: t : b]")] public void AnchoredSharedSubset_DualParsesWithMatchingSkeleton(string source) { - var python = PythonOracle.TryLocate(); - if (python is null) - { - _output.WriteLine("SKIP: python3 >= 3.12 and compare_ast.py not both available."); - return; - } + PythonInterpreter.RequireOrSkip(minMinor: 12); + var python = PythonOracle.TryLocate() + ?? throw new FileNotFoundException("python3 >= 3.12 is present but build_tools/differential_parse/compare_ast.py was not found."); // Sharpy side. var tokens = new Sharpy.Compiler.Lexer.Lexer(source).TokenizeAll(); diff --git a/src/Sharpy.Compiler.Tests/Sharpy.Compiler.Tests.csproj b/src/Sharpy.Compiler.Tests/Sharpy.Compiler.Tests.csproj index 36ebc3dc9..1f6114c66 100644 --- a/src/Sharpy.Compiler.Tests/Sharpy.Compiler.Tests.csproj +++ b/src/Sharpy.Compiler.Tests/Sharpy.Compiler.Tests.csproj @@ -21,6 +21,8 @@ + + diff --git a/src/Sharpy.TestInfrastructure/Oracles/PythonInterpreter.cs b/src/Sharpy.TestInfrastructure/Oracles/PythonInterpreter.cs index 63d3e059c..da47c27e1 100644 --- a/src/Sharpy.TestInfrastructure/Oracles/PythonInterpreter.cs +++ b/src/Sharpy.TestInfrastructure/Oracles/PythonInterpreter.cs @@ -1,6 +1,7 @@ using System.Diagnostics; using System.Text; using System.Text.RegularExpressions; +using Xunit; namespace Sharpy.TestInfrastructure.Oracles; @@ -11,7 +12,8 @@ namespace Sharpy.TestInfrastructure.Oracles; /// one today; retrofitting them is a chore note on #2300, not this helper's seam). /// /// -/// Absence is a SKIP on a developer machine and a FAIL where the oracle is required: +/// Absence is a SKIP on a developer machine (xunit's skipped count, through +/// , #2342) and a FAIL where the oracle is required: /// reads SHARPY_REQUIRE_PYTHON_ORACLE=1, which CI sets on the /// Compiler shards, so a runner that lost its setup-python step cannot pass the twin /// vacuously. The overload takes the @@ -93,6 +95,29 @@ private PythonInterpreter(string executable, string version, int major, int mino return located; } + /// + /// THE gate every python twin opens with (#2342, R-HZ): the located interpreter, or the test + /// reports SKIPPED — never PASSED — when none answers. Three verdicts: located → returned; + /// absent and not → (the test must be a + /// [SkippableFact]/[SkippableTheory], which xunit then counts under + /// skipped); absent and required (SHARPY_REQUIRE_PYTHON_ORACLE=1, CI) → + /// , a failure naming the oracle. + /// + public static PythonInterpreter RequireOrSkip(int minMinor = 12) => RequireOrSkip(DefaultCandidates, Required, minMinor); + + /// + /// over with the required flag + /// explicit, so a test can exercise each verdict without setting the process-global + /// environment variable. + /// + public static PythonInterpreter RequireOrSkip(IReadOnlyList candidates, bool required, int minMinor = 12) + { + return Locate(candidates, required, minMinor) + ?? throw new SkipException( + $"No Python 3.{minMinor}+ interpreter among [{string.Join(", ", candidates)}] on PATH: the python twin " + + $"compared nothing and is SKIPPED. Set {RequireEnvironmentVariable}=1 to make this a failure."); + } + /// /// The absolute path of under the repository root (the first /// ancestor of the test binary holding sharpy.sln), or null when the root or the file diff --git a/src/Sharpy.TestInfrastructure/Sharpy.TestInfrastructure.csproj b/src/Sharpy.TestInfrastructure/Sharpy.TestInfrastructure.csproj index 66715c649..e75e94e08 100644 --- a/src/Sharpy.TestInfrastructure/Sharpy.TestInfrastructure.csproj +++ b/src/Sharpy.TestInfrastructure/Sharpy.TestInfrastructure.csproj @@ -14,6 +14,8 @@ + + From ff76b8d409364f9a8e53ceba47dcb795ab0b6f9e Mon Sep 17 00:00:00 2001 From: antonsynd <2648730+antonsynd@users.noreply.github.com> Date: Thu, 22 Oct 2026 04:49:36 -0400 Subject: [PATCH 11/13] =?UTF-8?q?test(conformance):=20SharpyInstanceFormat?= =?UTF-8?q?tingTests=20joins=20the=20ConsoleCapture=20run-alone=20roster?= =?UTF-8?q?=20=E2=80=94=20it=20swaps=20Console.Out=20like=20Print=5FTests?= =?UTF-8?q?=20(refs=20#2057)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The #2057 chore (2f2c38fc0) added Sharpy.Core.Tests.SharpyInstanceFormattingTests to [Collection("ConsoleCapture")] (its print site captures stdout through Console.SetOut) without a SerialCollectionRoster row. ProcessGlobalStateConformanceTests caught it in the band's gate @ fed8a7c81: EveryMemberOfARunAloneCollection_IsRostered ("+ Sharpy.Core.Tests.SharpyInstanceFormattingTests in \"ConsoleCapture\"") and PositiveControl_TheRosterIsFoundInFull (members 54 expected, 55 found). The roster row gains the member; the hand-counted ExpectedSerialMembers goes 54 -> 55. No guard is new, so no mutation is owed: the two reds above are the existing guard firing on this exact omission. ProcessGlobalStateConformanceTests|InProcessIsolationSweepTests: 38/0/0 (measured, this tree). Co-Authored-By: Claude Opus 5.5 (1M context) Claude-Session: https://claude.ai/code/session_0126pNZqmMsSbn4WFFPhpnxW --- .../Conformance/ProcessGlobalStateConformanceTests.cs | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/Sharpy.Compiler.Tests/Conformance/ProcessGlobalStateConformanceTests.cs b/src/Sharpy.Compiler.Tests/Conformance/ProcessGlobalStateConformanceTests.cs index ce29d7e1b..0632c37c8 100644 --- a/src/Sharpy.Compiler.Tests/Conformance/ProcessGlobalStateConformanceTests.cs +++ b/src/Sharpy.Compiler.Tests/Conformance/ProcessGlobalStateConformanceTests.cs @@ -560,14 +560,14 @@ private static readonly (string Project, string Collection, string Reason, strin new[] { "Sharpy.Stdlib.Tests.StrftimeHostTimeZoneTests" }), ("Sharpy.Core.Tests", "ConsoleCapture", "Console.SetOut swaps a process-global writer while Core.Tests runs collections in parallel", - new[] { "Sharpy.Core.Tests.Print_Tests" }), + new[] { "Sharpy.Core.Tests.Print_Tests", "Sharpy.Core.Tests.SharpyInstanceFormattingTests" }), }; /// Positive control: roster size counted by hand from the P2/P4/P5 audit (not from the array). private const int ExpectedSerialCollections = 8; /// 1 + 1 + 44 + 1 (Compiler; LoadContextRetention #2250) + 1 + 3 + 1 (Stdlib) + 1 (Core), counted by hand; the WallClockWindow row (#2179 P5.1) drained with #2251. - private const int ExpectedSerialMembers = 54; + private const int ExpectedSerialMembers = 55; [Fact] public void EveryRunAloneCollection_IsRostered_AndEveryRosterRowIsDefined() From a7cd29e276864ebddb4adb44c0e6fd8f0526f0d4 Mon Sep 17 00:00:00 2001 From: antonsynd <2648730+antonsynd@users.noreply.github.com> Date: Thu, 22 Oct 2026 04:49:51 -0400 Subject: [PATCH 12/13] =?UTF-8?q?test(conformance):=20the=20in-process=20i?= =?UTF-8?q?solation=20sweep=20pins=2098=20drift=20cells=20under=20#2343=20?= =?UTF-8?q?=E2=80=94=20a=20reused=20Compiler=20reports=20the=20assemblies?= =?UTF-8?q?=20an=20earlier=20compile=20used=20(refs=20#2343)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The band's gate @ fed8a7c81 failed InProcessIsolationSweepTests.Sweep_EightThreads_EightRepetitions: 98 cells, all `compiler-per-thread 8 {analyze,compile} drift`, each reproduced 8/8, each the same shape: solo `used=[]`, observed `used=[.../Sharpy.Core.dll]` for a program that imports nothing. Attribution (measured, verification-contract §5–§6): - The sweep alone on this tree: 98 red (not load). - Each chore branch alone (same base e7e0a0ef7): semantic, lexer, stdlib and parser/cli 0 red; core (a7456f5b4) 98 red. - Control: this tree with the core chore's three Core sources reverted to 96d7a2792 and its new fixture kept: still 98 red. So the cause is the fixture's PRESENCE, not the Core change. The corpus sample is every 16th single-file fixture by sorted name; inserting builtins/iterable_instance_str_2057 moved 194 of the 213 sampled members. The newly sampled builtins_qualified_value_position_1463 imports `builtins`, a Core module. - Mechanism (read): Compiler builds one ModuleRegistry and reuses it; RecordModuleUsage fills _usedAssemblyPaths and nothing clears it. Filed as #2343 (class-tracker, lane:silent-wrong, reach:probe) with the cell matrix and acceptance. One row per red cell, each citing #2343 with `reproduced 8/8 @ fed8a7c81`. The rows drain when #2343's cure lands (the ratchet fails a stale row). No guard is new; the sweep and its stale-row check are the existing instrument. ProcessGlobalStateConformanceTests|InProcessIsolationSweepTests: 38/0/0 (measured, this tree). Co-Authored-By: Claude Opus 5.5 (1M context) Claude-Session: https://claude.ai/code/session_0126pNZqmMsSbn4WFFPhpnxW --- .../in-process-isolation-allowlist.txt | 98 +++++++++++++++++++ 1 file changed, 98 insertions(+) diff --git a/src/Sharpy.Compiler.Tests/Conformance/in-process-isolation-allowlist.txt b/src/Sharpy.Compiler.Tests/Conformance/in-process-isolation-allowlist.txt index e3c1820c2..8618db0a7 100644 --- a/src/Sharpy.Compiler.Tests/Conformance/in-process-isolation-allowlist.txt +++ b/src/Sharpy.Compiler.Tests/Conformance/in-process-isolation-allowlist.txt @@ -12,3 +12,101 @@ # as stale — drain it when the fix lands — unless it is marked `probabilistic` (reported, not judged). # Every row cites an OPEN issue by number (check_allowlist_issue_state.sh gates the state); the k/R is the measured # reproduction count at the named sha. +algorithms/binary_search_tree compiler-per-thread 8 compile drift # #2343 a reused Compiler reports Sharpy.Core.dll used after a builtins importer on the same instance (solo used=[]); reproduced 8/8 @ fed8a7c81 +builtins/iterable_sources_matrix compiler-per-thread 8 analyze drift # #2343 a reused Compiler reports Sharpy.Core.dll used after a builtins importer on the same instance (solo used=[]); reproduced 8/8 @ fed8a7c81 +builtins/zip_basic compiler-per-thread 8 compile drift # #2343 a reused Compiler reports Sharpy.Core.dll used after a builtins importer on the same instance (solo used=[]); reproduced 8/8 @ fed8a7c81 +classes/indexer_setitem_value_param_1529 compiler-per-thread 8 analyze drift # #2343 a reused Compiler reports Sharpy.Core.dll used after a builtins importer on the same instance (solo used=[]); reproduced 8/8 @ fed8a7c81 +collections/dict_key_widening_ok_1620 compiler-per-thread 8 analyze drift # #2343 a reused Compiler reports Sharpy.Core.dll used after a builtins importer on the same instance (solo used=[]); reproduced 8/8 @ fed8a7c81 +control_flow/logical_operators_simple compiler-per-thread 8 compile drift # #2343 a reused Compiler reports Sharpy.Core.dll used after a builtins importer on the same instance (solo used=[]); reproduced 8/8 @ fed8a7c81 +decorators/bracket_attr_import_used_1429 compiler-per-thread 8 analyze drift # #2343 a reused Compiler reports Sharpy.Core.dll used after a builtins importer on the same instance (solo used=[]); reproduced 8/8 @ fed8a7c81 +decorators/bracket_attr_import_used_1429 compiler-per-thread 8 compile drift # #2343 a reused Compiler reports Sharpy.Core.dll used after a builtins importer on the same instance (solo used=[]); reproduced 8/8 @ fed8a7c81 +decorators/deprecated_self_init compiler-per-thread 8 compile drift # #2343 a reused Compiler reports Sharpy.Core.dll used after a builtins importer on the same instance (solo used=[]); reproduced 8/8 @ fed8a7c81 +deferred_overload_annotated_pin_1589 compiler-per-thread 8 analyze drift # #2343 a reused Compiler reports Sharpy.Core.dll used after a builtins importer on the same instance (solo used=[]); reproduced 8/8 @ fed8a7c81 +delegates/delegate_generic_invoke_optional_params_0001 compiler-per-thread 8 compile drift # #2343 a reused Compiler reports Sharpy.Core.dll used after a builtins importer on the same instance (solo used=[]); reproduced 8/8 @ fed8a7c81 +enums/enum_hex_values compiler-per-thread 8 analyze drift # #2343 a reused Compiler reports Sharpy.Core.dll used after a builtins importer on the same instance (solo used=[]); reproduced 8/8 @ fed8a7c81 +errors/parser_recovery_header_with_body compiler-per-thread 8 compile drift # #2343 a reused Compiler reports Sharpy.Core.dll used after a builtins importer on the same instance (solo used=[]); reproduced 8/8 @ fed8a7c81 +expressions/nested_list_literal_0002 compiler-per-thread 8 compile drift # #2343 a reused Compiler reports Sharpy.Core.dll used after a builtins importer on the same instance (solo used=[]); reproduced 8/8 @ fed8a7c81 +fstrings/format_iformattable_class_1988 compiler-per-thread 8 compile drift # #2343 a reused Compiler reports Sharpy.Core.dll used after a builtins importer on the same instance (solo used=[]); reproduced 8/8 @ fed8a7c81 +fstrings/format_no_dunder_format_refused_1988 compiler-per-thread 8 analyze drift # #2343 a reused Compiler reports Sharpy.Core.dll used after a builtins importer on the same instance (solo used=[]); reproduced 8/8 @ fed8a7c81 +fstrings/format_no_dunder_format_refused_1988 compiler-per-thread 8 compile drift # #2343 a reused Compiler reports Sharpy.Core.dll used after a builtins importer on the same instance (solo used=[]); reproduced 8/8 @ fed8a7c81 +fuzz/seed2025_iter18 compiler-per-thread 8 analyze drift # #2343 a reused Compiler reports Sharpy.Core.dll used after a builtins importer on the same instance (solo used=[]); reproduced 8/8 @ fed8a7c81 +generators/tuple_unpacking_in_generator compiler-per-thread 8 analyze drift # #2343 a reused Compiler reports Sharpy.Core.dll used after a builtins importer on the same instance (solo used=[]); reproduced 8/8 @ fed8a7c81 +generics/bcl_extension_method_explicit_args compiler-per-thread 8 compile drift # #2343 a reused Compiler reports Sharpy.Core.dll used after a builtins importer on the same instance (solo used=[]); reproduced 8/8 @ fed8a7c81 +generics/iter_open_bare_T_1518 compiler-per-thread 8 analyze drift # #2343 a reused Compiler reports Sharpy.Core.dll used after a builtins importer on the same instance (solo used=[]); reproduced 8/8 @ fed8a7c81 +generics/parenthesized_callee_narrowed_optional_1147 compiler-per-thread 8 analyze drift # #2343 a reused Compiler reports Sharpy.Core.dll used after a builtins importer on the same instance (solo used=[]); reproduced 8/8 @ fed8a7c81 +if_elif_else_variable_scoping compiler-per-thread 8 compile drift # #2343 a reused Compiler reports Sharpy.Core.dll used after a builtins importer on the same instance (solo used=[]); reproduced 8/8 @ fed8a7c81 +interop/clr_action_int_arity_1613 compiler-per-thread 8 compile drift # #2343 a reused Compiler reports Sharpy.Core.dll used after a builtins importer on the same instance (solo used=[]); reproduced 8/8 @ fed8a7c81 +interop/clr_char_argument_computed_refused_1402 compiler-per-thread 8 analyze drift # #2343 a reused Compiler reports Sharpy.Core.dll used after a builtins importer on the same instance (solo used=[]); reproduced 8/8 @ fed8a7c81 +optional_result/optional_some_function_arg compiler-per-thread 8 analyze drift # #2343 a reused Compiler reports Sharpy.Core.dll used after a builtins importer on the same instance (solo used=[]); reproduced 8/8 @ fed8a7c81 +probe/builtins_ok compiler-per-thread 8 analyze drift # #2343 a reused Compiler reports Sharpy.Core.dll used after a builtins importer on the same instance (solo used=[]); reproduced 8/8 @ fed8a7c81 +probe/builtins_ok compiler-per-thread 8 compile drift # #2343 a reused Compiler reports Sharpy.Core.dll used after a builtins importer on the same instance (solo used=[]); reproduced 8/8 @ fed8a7c81 +probe/dataclass_user compiler-per-thread 8 analyze drift # #2343 a reused Compiler reports Sharpy.Core.dll used after a builtins importer on the same instance (solo used=[]); reproduced 8/8 @ fed8a7c81 +probe/dataclass_user compiler-per-thread 8 compile drift # #2343 a reused Compiler reports Sharpy.Core.dll used after a builtins importer on the same instance (solo used=[]); reproduced 8/8 @ fed8a7c81 +probe/exception_typed compiler-per-thread 8 analyze drift # #2343 a reused Compiler reports Sharpy.Core.dll used after a builtins importer on the same instance (solo used=[]); reproduced 8/8 @ fed8a7c81 +probe/exception_typed compiler-per-thread 8 compile drift # #2343 a reused Compiler reports Sharpy.Core.dll used after a builtins importer on the same instance (solo used=[]); reproduced 8/8 @ fed8a7c81 +probe/format_typed compiler-per-thread 8 analyze drift # #2343 a reused Compiler reports Sharpy.Core.dll used after a builtins importer on the same instance (solo used=[]); reproduced 8/8 @ fed8a7c81 +probe/format_typed compiler-per-thread 8 compile drift # #2343 a reused Compiler reports Sharpy.Core.dll used after a builtins importer on the same instance (solo used=[]); reproduced 8/8 @ fed8a7c81 +probe/len_typed compiler-per-thread 8 analyze drift # #2343 a reused Compiler reports Sharpy.Core.dll used after a builtins importer on the same instance (solo used=[]); reproduced 8/8 @ fed8a7c81 +probe/len_typed compiler-per-thread 8 compile drift # #2343 a reused Compiler reports Sharpy.Core.dll used after a builtins importer on the same instance (solo used=[]); reproduced 8/8 @ fed8a7c81 +probe/list_no_member compiler-per-thread 8 analyze drift # #2343 a reused Compiler reports Sharpy.Core.dll used after a builtins importer on the same instance (solo used=[]); reproduced 8/8 @ fed8a7c81 +probe/list_no_member compiler-per-thread 8 compile drift # #2343 a reused Compiler reports Sharpy.Core.dll used after a builtins importer on the same instance (solo used=[]); reproduced 8/8 @ fed8a7c81 +probe/str_no_member compiler-per-thread 8 analyze drift # #2343 a reused Compiler reports Sharpy.Core.dll used after a builtins importer on the same instance (solo used=[]); reproduced 8/8 @ fed8a7c81 +probe/str_upper_control compiler-per-thread 8 analyze drift # #2343 a reused Compiler reports Sharpy.Core.dll used after a builtins importer on the same instance (solo used=[]); reproduced 8/8 @ fed8a7c81 +probe/str_upper_control compiler-per-thread 8 compile drift # #2343 a reused Compiler reports Sharpy.Core.dll used after a builtins importer on the same instance (solo used=[]); reproduced 8/8 @ fed8a7c81 +self_type/self_type_struct compiler-per-thread 8 analyze drift # #2343 a reused Compiler reports Sharpy.Core.dll used after a builtins importer on the same instance (solo used=[]); reproduced 8/8 @ fed8a7c81 +tpa/drive_info compiler-per-thread 8 analyze drift # #2343 a reused Compiler reports Sharpy.Core.dll used after a builtins importer on the same instance (solo used=[]); reproduced 8/8 @ fed8a7c81 +trigger/class-str-in-if-in-def compiler-per-thread 8 analyze drift # #2343 a reused Compiler reports Sharpy.Core.dll used after a builtins importer on the same instance (solo used=[]); reproduced 8/8 @ fed8a7c81 +trigger/class_format_def-body compiler-per-thread 8 analyze drift # #2343 a reused Compiler reports Sharpy.Core.dll used after a builtins importer on the same instance (solo used=[]); reproduced 8/8 @ fed8a7c81 +trigger/class_format_mod-try compiler-per-thread 8 analyze drift # #2343 a reused Compiler reports Sharpy.Core.dll used after a builtins importer on the same instance (solo used=[]); reproduced 8/8 @ fed8a7c81 +trigger/class_len_def-body compiler-per-thread 8 analyze drift # #2343 a reused Compiler reports Sharpy.Core.dll used after a builtins importer on the same instance (solo used=[]); reproduced 8/8 @ fed8a7c81 +trigger/class_len_mod-try compiler-per-thread 8 analyze drift # #2343 a reused Compiler reports Sharpy.Core.dll used after a builtins importer on the same instance (solo used=[]); reproduced 8/8 @ fed8a7c81 +trigger/class_list_mod-try compiler-per-thread 8 analyze drift # #2343 a reused Compiler reports Sharpy.Core.dll used after a builtins importer on the same instance (solo used=[]); reproduced 8/8 @ fed8a7c81 +trigger/class_str_def-body compiler-per-thread 8 analyze drift # #2343 a reused Compiler reports Sharpy.Core.dll used after a builtins importer on the same instance (solo used=[]); reproduced 8/8 @ fed8a7c81 +trigger/dataclass-str-in-module-try compiler-per-thread 8 analyze drift # #2343 a reused Compiler reports Sharpy.Core.dll used after a builtins importer on the same instance (solo used=[]); reproduced 8/8 @ fed8a7c81 +trigger/dataclass_format_def-body compiler-per-thread 8 analyze drift # #2343 a reused Compiler reports Sharpy.Core.dll used after a builtins importer on the same instance (solo used=[]); reproduced 8/8 @ fed8a7c81 +trigger/dataclass_format_mod-try compiler-per-thread 8 analyze drift # #2343 a reused Compiler reports Sharpy.Core.dll used after a builtins importer on the same instance (solo used=[]); reproduced 8/8 @ fed8a7c81 +trigger/dataclass_len_def-body compiler-per-thread 8 analyze drift # #2343 a reused Compiler reports Sharpy.Core.dll used after a builtins importer on the same instance (solo used=[]); reproduced 8/8 @ fed8a7c81 +trigger/dataclass_len_mod-try compiler-per-thread 8 analyze drift # #2343 a reused Compiler reports Sharpy.Core.dll used after a builtins importer on the same instance (solo used=[]); reproduced 8/8 @ fed8a7c81 +trigger/dataclass_list_def-body compiler-per-thread 8 analyze drift # #2343 a reused Compiler reports Sharpy.Core.dll used after a builtins importer on the same instance (solo used=[]); reproduced 8/8 @ fed8a7c81 +trigger/dataclass_str_def-body compiler-per-thread 8 analyze drift # #2343 a reused Compiler reports Sharpy.Core.dll used after a builtins importer on the same instance (solo used=[]); reproduced 8/8 @ fed8a7c81 +trigger/dataclass_str_mod-try compiler-per-thread 8 analyze drift # #2343 a reused Compiler reports Sharpy.Core.dll used after a builtins importer on the same instance (solo used=[]); reproduced 8/8 @ fed8a7c81 +trigger/def_format_def-body compiler-per-thread 8 analyze drift # #2343 a reused Compiler reports Sharpy.Core.dll used after a builtins importer on the same instance (solo used=[]); reproduced 8/8 @ fed8a7c81 +trigger/def_format_mod-try compiler-per-thread 8 analyze drift # #2343 a reused Compiler reports Sharpy.Core.dll used after a builtins importer on the same instance (solo used=[]); reproduced 8/8 @ fed8a7c81 +trigger/def_len_def-body compiler-per-thread 8 analyze drift # #2343 a reused Compiler reports Sharpy.Core.dll used after a builtins importer on the same instance (solo used=[]); reproduced 8/8 @ fed8a7c81 +trigger/def_list_mod-try compiler-per-thread 8 analyze drift # #2343 a reused Compiler reports Sharpy.Core.dll used after a builtins importer on the same instance (solo used=[]); reproduced 8/8 @ fed8a7c81 +trigger/def_str_def-body compiler-per-thread 8 analyze drift # #2343 a reused Compiler reports Sharpy.Core.dll used after a builtins importer on the same instance (solo used=[]); reproduced 8/8 @ fed8a7c81 +trigger/def_str_mod-try compiler-per-thread 8 analyze drift # #2343 a reused Compiler reports Sharpy.Core.dll used after a builtins importer on the same instance (solo used=[]); reproduced 8/8 @ fed8a7c81 +trigger/delegate-str-in-def compiler-per-thread 8 analyze drift # #2343 a reused Compiler reports Sharpy.Core.dll used after a builtins importer on the same instance (solo used=[]); reproduced 8/8 @ fed8a7c81 +trigger/delegate_format_def-body compiler-per-thread 8 analyze drift # #2343 a reused Compiler reports Sharpy.Core.dll used after a builtins importer on the same instance (solo used=[]); reproduced 8/8 @ fed8a7c81 +trigger/delegate_format_mod-try compiler-per-thread 8 analyze drift # #2343 a reused Compiler reports Sharpy.Core.dll used after a builtins importer on the same instance (solo used=[]); reproduced 8/8 @ fed8a7c81 +trigger/delegate_len_def-body compiler-per-thread 8 analyze drift # #2343 a reused Compiler reports Sharpy.Core.dll used after a builtins importer on the same instance (solo used=[]); reproduced 8/8 @ fed8a7c81 +trigger/delegate_len_mod-try compiler-per-thread 8 analyze drift # #2343 a reused Compiler reports Sharpy.Core.dll used after a builtins importer on the same instance (solo used=[]); reproduced 8/8 @ fed8a7c81 +trigger/delegate_list_def-body compiler-per-thread 8 analyze drift # #2343 a reused Compiler reports Sharpy.Core.dll used after a builtins importer on the same instance (solo used=[]); reproduced 8/8 @ fed8a7c81 +trigger/delegate_list_mod-try compiler-per-thread 8 analyze drift # #2343 a reused Compiler reports Sharpy.Core.dll used after a builtins importer on the same instance (solo used=[]); reproduced 8/8 @ fed8a7c81 +trigger/delegate_str_mod-try compiler-per-thread 8 analyze drift # #2343 a reused Compiler reports Sharpy.Core.dll used after a builtins importer on the same instance (solo used=[]); reproduced 8/8 @ fed8a7c81 +trigger/interface-str-in-for-in-def compiler-per-thread 8 analyze drift # #2343 a reused Compiler reports Sharpy.Core.dll used after a builtins importer on the same instance (solo used=[]); reproduced 8/8 @ fed8a7c81 +trigger/interface_format_def-body compiler-per-thread 8 analyze drift # #2343 a reused Compiler reports Sharpy.Core.dll used after a builtins importer on the same instance (solo used=[]); reproduced 8/8 @ fed8a7c81 +trigger/interface_format_mod-try compiler-per-thread 8 analyze drift # #2343 a reused Compiler reports Sharpy.Core.dll used after a builtins importer on the same instance (solo used=[]); reproduced 8/8 @ fed8a7c81 +trigger/interface_len_def-body compiler-per-thread 8 analyze drift # #2343 a reused Compiler reports Sharpy.Core.dll used after a builtins importer on the same instance (solo used=[]); reproduced 8/8 @ fed8a7c81 +trigger/interface_len_mod-try compiler-per-thread 8 analyze drift # #2343 a reused Compiler reports Sharpy.Core.dll used after a builtins importer on the same instance (solo used=[]); reproduced 8/8 @ fed8a7c81 +trigger/interface_list_mod-try compiler-per-thread 8 analyze drift # #2343 a reused Compiler reports Sharpy.Core.dll used after a builtins importer on the same instance (solo used=[]); reproduced 8/8 @ fed8a7c81 +trigger/interface_str_def-body compiler-per-thread 8 analyze drift # #2343 a reused Compiler reports Sharpy.Core.dll used after a builtins importer on the same instance (solo used=[]); reproduced 8/8 @ fed8a7c81 +trigger/interface_str_mod-try compiler-per-thread 8 analyze drift # #2343 a reused Compiler reports Sharpy.Core.dll used after a builtins importer on the same instance (solo used=[]); reproduced 8/8 @ fed8a7c81 +trigger/struct-str-in-def compiler-per-thread 8 analyze drift # #2343 a reused Compiler reports Sharpy.Core.dll used after a builtins importer on the same instance (solo used=[]); reproduced 8/8 @ fed8a7c81 +trigger/struct_format_def-body compiler-per-thread 8 analyze drift # #2343 a reused Compiler reports Sharpy.Core.dll used after a builtins importer on the same instance (solo used=[]); reproduced 8/8 @ fed8a7c81 +trigger/struct_format_mod-try compiler-per-thread 8 analyze drift # #2343 a reused Compiler reports Sharpy.Core.dll used after a builtins importer on the same instance (solo used=[]); reproduced 8/8 @ fed8a7c81 +trigger/struct_len_def-body compiler-per-thread 8 analyze drift # #2343 a reused Compiler reports Sharpy.Core.dll used after a builtins importer on the same instance (solo used=[]); reproduced 8/8 @ fed8a7c81 +trigger/struct_len_mod-try compiler-per-thread 8 analyze drift # #2343 a reused Compiler reports Sharpy.Core.dll used after a builtins importer on the same instance (solo used=[]); reproduced 8/8 @ fed8a7c81 +trigger/struct_list_def-body compiler-per-thread 8 analyze drift # #2343 a reused Compiler reports Sharpy.Core.dll used after a builtins importer on the same instance (solo used=[]); reproduced 8/8 @ fed8a7c81 +trigger/struct_list_mod-try compiler-per-thread 8 analyze drift # #2343 a reused Compiler reports Sharpy.Core.dll used after a builtins importer on the same instance (solo used=[]); reproduced 8/8 @ fed8a7c81 +trigger/struct_str_def-body compiler-per-thread 8 analyze drift # #2343 a reused Compiler reports Sharpy.Core.dll used after a builtins importer on the same instance (solo used=[]); reproduced 8/8 @ fed8a7c81 +trigger/union_format_mod-try compiler-per-thread 8 analyze drift # #2343 a reused Compiler reports Sharpy.Core.dll used after a builtins importer on the same instance (solo used=[]); reproduced 8/8 @ fed8a7c81 +trigger/union_len_def-body compiler-per-thread 8 analyze drift # #2343 a reused Compiler reports Sharpy.Core.dll used after a builtins importer on the same instance (solo used=[]); reproduced 8/8 @ fed8a7c81 +trigger/union_len_mod-try compiler-per-thread 8 analyze drift # #2343 a reused Compiler reports Sharpy.Core.dll used after a builtins importer on the same instance (solo used=[]); reproduced 8/8 @ fed8a7c81 +trigger/union_list_def-body compiler-per-thread 8 analyze drift # #2343 a reused Compiler reports Sharpy.Core.dll used after a builtins importer on the same instance (solo used=[]); reproduced 8/8 @ fed8a7c81 +trigger/union_str_mod-try compiler-per-thread 8 analyze drift # #2343 a reused Compiler reports Sharpy.Core.dll used after a builtins importer on the same instance (solo used=[]); reproduced 8/8 @ fed8a7c81 +trigger/wrap-try-method-format compiler-per-thread 8 analyze drift # #2343 a reused Compiler reports Sharpy.Core.dll used after a builtins importer on the same instance (solo used=[]); reproduced 8/8 @ fed8a7c81 +trigger/wrap-try-shadowing-len compiler-per-thread 8 analyze drift # #2343 a reused Compiler reports Sharpy.Core.dll used after a builtins importer on the same instance (solo used=[]); reproduced 8/8 @ fed8a7c81 +tuple_unpacking/sole_star_group_1845 compiler-per-thread 8 compile drift # #2343 a reused Compiler reports Sharpy.Core.dll used after a builtins importer on the same instance (solo used=[]); reproduced 8/8 @ fed8a7c81 +variable_name_collision_class/main compiler-per-thread 8 compile drift # #2343 a reused Compiler reports Sharpy.Core.dll used after a builtins importer on the same instance (solo used=[]); reproduced 8/8 @ fed8a7c81 +with_statement/as_member_unused_control_1710 compiler-per-thread 8 analyze drift # #2343 a reused Compiler reports Sharpy.Core.dll used after a builtins importer on the same instance (solo used=[]); reproduced 8/8 @ fed8a7c81 From 4d222124271b65b12feaeec797398c3443d1d56a Mon Sep 17 00:00:00 2001 From: antonsynd <2648730+antonsynd@users.noreply.github.com> Date: Thu, 22 Oct 2026 05:02:05 -0400 Subject: [PATCH 13/13] =?UTF-8?q?test(lsp):=20the=20code-action=20shape=20?= =?UTF-8?q?sweep's=20extract-method=20and=20wrap-try=20diagnostics=20pins?= =?UTF-8?q?=20count=20the=20#2057=20fixture=20=E2=80=94=20one=20new=20corp?= =?UTF-8?q?us=20stem=20in=20two=20existing=20classes=20(refs=20#2294,=20re?= =?UTF-8?q?fs=20#2296)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The band's gate @ fed8a7c81 failed CodeActionShapeSweepTests.Identity_EveryCodeActionKeepsTheSourceShape: regular `wrap-try identity diagnostics #2296` pinned 374, actual 375. The one added stem is builtins/iterable_instance_str_2057 (the #2057 chore's new run fixture): wrapping its statements in try changes its diagnostics, the cell the #2296 pin already holds for 374 other stems. Re-pinned from a FULL run (SHARPY_CODEACTION_SWEEP_FULL=1, DOP 8; the allowlist header's regeneration rule) on this tree. Exactly two keys moved in each of the four twins, each by +1, and in every one the added stem is that fixture (checked in -full-mismatched-stems.txt): - extract-method {identity,crlf,cr,comment} diagnostics #2294: full 2870 -> 2871 (regular unchanged) - wrap-try {identity,crlf,cr,comment} diagnostics #2296: full 2311 -> 2312; identity regular 374 -> 375 Every other row of the four actual-pins files equals the committed row. No guard is new; the sweep's pin check is the existing instrument, and its red named this stem. Co-Authored-By: Claude Opus 5.5 (1M context) Claude-Session: https://claude.ai/code/session_0126pNZqmMsSbn4WFFPhpnxW --- .../Conformance/code-action-shape-allowlist.txt | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/src/Sharpy.Lsp.Tests/Conformance/code-action-shape-allowlist.txt b/src/Sharpy.Lsp.Tests/Conformance/code-action-shape-allowlist.txt index 8a98079b6..111ac1dc1 100644 --- a/src/Sharpy.Lsp.Tests/Conformance/code-action-shape-allowlist.txt +++ b/src/Sharpy.Lsp.Tests/Conformance/code-action-shape-allowlist.txt @@ -14,7 +14,7 @@ # #2294 synthesized type spellings (Extract method parameter inference, Extract variable annotation); # #2296 code actions that are not meaning-preserving (diagnostics of every other action). # Regenerate from a FULL run: .claude/tmp/code-action-sweep/-actual-pins.txt. A drained pin is deleted. -extract-method identity diagnostics 2870 abc897324d0ad46e 830 5a51b33adb11239c # #2294 adds SPY0202 (and 5588 other cells) +extract-method identity diagnostics 2871 e40824855c586391 830 5a51b33adb11239c # #2294 adds SPY0202 (and 5588 other cells) extract-method identity parse 532 65f86a1b0a274bfd 532 65f86a1b0a274bfd # #2294 does not parse (SPY0104 Expected RightBracket, got Comma) (and 387 other cells) extract-variable identity diagnostics 220 88ce9452651014a6 41 92370514d76502bc # #2294 adds SPY0220 (and 257 other cells) extract-variable identity parse 56 8009d0bb498adc30 56 8009d0bb498adc30 # #2294 does not parse (SPY0114 List type shorthand requires an element type:) (and 32 other cells) @@ -27,8 +27,8 @@ remove-annotation identity diagnostics 518 48e0dd41eb5701ae 91 0589405297fbebe0 remove-import identity diagnostics 18 b5484e02da1684c7 4 c64f809e53e1ecee # #2296 removes an error rename identity diagnostics 38 bcd7354d59346fed 4 02aeed9aec22452e # #2296 adds SPY0200 (and 33 other cells) to-if identity diagnostics 4 35d1e4876190ef73 1 96a879de4c764cb2 # #2296 adds SPY0402 (and 4 other cells) -wrap-try identity diagnostics 2311 86867ecb61a8c639 374 935e7cd9b8e0ee4f # #2296 adds SPY0200 (and 770 other cells) -extract-method crlf diagnostics 2870 abc897324d0ad46e 103 9382790fe30236c2 # #2294 adds SPY0202 (and 5588 other cells) +wrap-try identity diagnostics 2312 f8096961c78d453e 375 b1d513d5f7fd9b3f # #2296 adds SPY0200 (and 770 other cells) +extract-method crlf diagnostics 2871 e40824855c586391 103 9382790fe30236c2 # #2294 adds SPY0202 (and 5588 other cells) extract-method crlf parse 532 65f86a1b0a274bfd 68 794d7bfe2cabf6f3 # #2294 does not parse (SPY0104 Expected RightBracket, got Comma) (and 387 other cells) extract-variable crlf diagnostics 220 88ce9452651014a6 9 579d0c58dbf30210 # #2294 adds SPY0220 (and 257 other cells) extract-variable crlf parse 56 8009d0bb498adc30 5 fb240c4c4a9247fd # #2294 does not parse (SPY0114 List type shorthand requires an element type:) (and 32 other cells) @@ -41,8 +41,8 @@ remove-annotation crlf diagnostics 518 48e0dd41eb5701ae 9 589fc84f1e1bef7d # #22 remove-import crlf diagnostics 18 b5484e02da1684c7 0 - # #2296 removes an error rename crlf diagnostics 38 bcd7354d59346fed 0 - # #2296 adds SPY0200 (and 33 other cells) to-if crlf diagnostics 4 35d1e4876190ef73 0 - # #2296 adds SPY0402 (and 4 other cells) -wrap-try crlf diagnostics 2311 86867ecb61a8c639 51 dc33e60edb181a8d # #2296 adds SPY0200 (and 770 other cells) -extract-method cr diagnostics 2870 abc897324d0ad46e 106 84f086ef82d71563 # #2294 adds SPY0202 (and 5588 other cells) +wrap-try crlf diagnostics 2312 f8096961c78d453e 51 dc33e60edb181a8d # #2296 adds SPY0200 (and 770 other cells) +extract-method cr diagnostics 2871 e40824855c586391 106 84f086ef82d71563 # #2294 adds SPY0202 (and 5588 other cells) extract-method cr parse 532 65f86a1b0a274bfd 68 794d7bfe2cabf6f3 # #2294 does not parse (SPY0104 Expected RightBracket, got Comma) (and 387 other cells) extract-variable cr diagnostics 220 88ce9452651014a6 4 b1013f49688ce6db # #2294 adds SPY0220 (and 257 other cells) extract-variable cr parse 56 8009d0bb498adc30 5 fb240c4c4a9247fd # #2294 does not parse (SPY0114 List type shorthand requires an element type:) (and 32 other cells) @@ -55,8 +55,8 @@ remove-annotation cr diagnostics 518 48e0dd41eb5701ae 27 4ea9b045a28530f6 # #229 remove-import cr diagnostics 18 b5484e02da1684c7 0 - # #2296 removes an error rename cr diagnostics 38 bcd7354d59346fed 0 - # #2296 adds SPY0200 (and 33 other cells) to-if cr diagnostics 4 35d1e4876190ef73 0 - # #2296 adds SPY0402 (and 4 other cells) -wrap-try cr diagnostics 2311 86867ecb61a8c639 50 751a80830ccf7134 # #2296 adds SPY0200 (and 770 other cells) -extract-method comment diagnostics 2870 abc897324d0ad46e 119 ff6550044212717d # #2294 adds SPY0202 (and 5590 other cells) +wrap-try cr diagnostics 2312 f8096961c78d453e 50 751a80830ccf7134 # #2296 adds SPY0200 (and 770 other cells) +extract-method comment diagnostics 2871 e40824855c586391 119 ff6550044212717d # #2294 adds SPY0202 (and 5590 other cells) extract-method comment parse 532 65f86a1b0a274bfd 68 794d7bfe2cabf6f3 # #2294 does not parse (SPY0104 Expected RightBracket, got Comma) (and 387 other cells) extract-variable comment diagnostics 220 88ce9452651014a6 5 8d44ec83ca2ddff1 # #2294 adds SPY0220 (and 257 other cells) extract-variable comment parse 56 8009d0bb498adc30 5 fb240c4c4a9247fd # #2294 does not parse (SPY0114 List type shorthand requires an element type:) (and 32 other cells) @@ -68,4 +68,4 @@ remove-annotation comment diagnostics 334 1bf1d889bd4e3457 28 fea4201873be93cd # remove-import comment diagnostics 18 b5484e02da1684c7 0 - # #2296 removes an error rename comment diagnostics 38 bcd7354d59346fed 0 - # #2296 adds SPY0200 (and 33 other cells) to-if comment diagnostics 4 35d1e4876190ef73 0 - # #2296 adds SPY0222 (and 4 other cells) -wrap-try comment diagnostics 2311 5cc4e89979d9f814 34 1b2d395153d3ce0f # #2296 adds SPY0200 (and 772 other cells) +wrap-try comment diagnostics 2312 912a1027945e8b4c 34 1b2d395153d3ce0f # #2296 adds SPY0200 (and 772 other cells)