From c5f38848a58e37fe3d0dc4e4775b5d828a0f87b4 Mon Sep 17 00:00:00 2001 From: Richard Feldman Date: Tue, 27 Oct 2026 09:45:46 -0400 Subject: [PATCH 1/5] Settle every recursive type digest through one complete reduction Monotype type digests, Lambda Mono type digests, procedure identities and recursive layout keys each carried their own copy of the recursive-type reduction, and only the layout store's paired a cyclic component against a settled group it refers into. A program whose solved types tie a copy of a recursive type whose references lead into a separately tied copy therefore rendered a different procedure identity than a program drawing a single knot, under the same specialization key, and a cached procedure from the first program was spliced into the second for the wrong identity. base.recursive_digest is now the one reduction every consumer uses: acyclic copies fold onto settled positions, a cyclic component referring into a settled group is paired coinductively against its member shapes, and every other component is reduced by bisimulation as before. --- design.md | 70 ++- src/base/mod.zig | 2 + src/base/recursive_digest.zig | 846 +++++++++++++++++++++++++++++ src/layout/store.zig | 820 +++------------------------- src/lir/lir_image.zig | 14 +- src/postcheck/lambda_mono/type.zig | 22 +- src/postcheck/monotype/type.zig | 663 +++------------------- src/postcheck/proc_identity.zig | 513 ++++------------- src/postcheck/type_digest.zig | 340 ------------ 9 files changed, 1197 insertions(+), 2093 deletions(-) create mode 100644 src/base/recursive_digest.zig delete mode 100644 src/postcheck/type_digest.zig diff --git a/design.md b/design.md index 5631b5fc794..ef4214b2608 100644 --- a/design.md +++ b/design.md @@ -398,6 +398,41 @@ requires structural identity, name it `...Key` only after establishing why a dense ID cannot represent it; changing the name solely to silence the lint violates this invariant. +### Recursive Type Digests + +Every type identity derived from a graph of type nodes is settled by one +reduction, `base.recursive_digest`: Monotype type digests, Lambda Mono type +digests, procedure identities (`postcheck/proc_identity.zig`), and recursive +layout keys. Each consumer only discovers its graph and encodes each node as +its content (its encoding with every child as a bare slot) and its ordered +children; how a recursive type is digested lives in one place. + +A digest is a function of the infinite type alone, never of the graph a +program happened to draw for it. Programs draw one recursive type as a single +knot, as a knot rolled out several levels, or as a copy whose own knot also +references a second, separately tied copy, depending on which parts of the +type each program's solving touched. All of them digest alike, in one +reduction or across reductions that share a `Settled` store. The reduction +resolves strongly connected components in reverse topological order: + +- An acyclic node digests its one-step unfolding (content and child digests). + When that unfolding is a settled recursive position's, the node is a + rolled-out copy of that position and takes its digest. +- A cyclic component that refers into a settled recursive group is paired + coinductively against the group's recorded member shapes. Every member of a + component reaches every other, so either every member is equivalent to a + position of the group or none is; on success the members take the group's + digests. +- Any other cyclic component is refined to its coarsest bisimulation + partition and rendered once with its positions in label order. No child + outside it is equivalent to one of its members, so the refinement sees + exactly the reduced group whatever graph spelled it. + +Completeness is required, not an optimization. A cache entry found under a +specialization key is spliced only when its procedure identity is the one the +consuming program lowers for that key, so two programs that agree on a +specialization key must agree on every identity derived from the same types. + ### Native relocation identity Every linked function relocation carries the exact call encoding selected by @@ -474,15 +509,12 @@ Exhaustion therefore retains the ordinary exact IR; it is never cached as representation. Lambda Mono type digests consume the explicit capture-type graph, including -erased callable members. Iterative discovery encodes each reachable node once; -acyclic nodes combine their scalar data with completed child digests. Cyclic -components are reduced by bisimulation and encode their positions in label order, -never store-local type ids. Sharing, duplication, and recursive unrolling do -not change type identity. The store owns reusable discovery, component, and -partition buffers. Every request discards its graph and results on success or -allocation failure while retaining capacity: no identity survives mutation of -the source store. Allocation failure propagates to the caller. The versioned -Lambda Mono digest domain separates this encoding from the former flat walk. +erased callable members. Iterative discovery encodes each reachable node once +and the shared recursive-type reduction ("Recursive Type Digests") settles the graph, never +store-local type ids. The store owns reusable discovery and reduction buffers. +Every request discards its graph and results on success or allocation failure +while retaining capacity: no identity survives mutation of the source store. +Allocation failure propagates to the caller. Representation finiteness is different from proof-query termination. Monotype bounds minted iterator identities at the single graph-owned construction choke @@ -17914,9 +17946,9 @@ committed recursive node different slot boxing. Instead, every layout commit returns the recursive-graph digest each node settled to, the builder records it per type, and a cached child re-enters a later graph as a `committed` leaf that carries its layout and that digest. The analysis digests the leaf exactly as it -would digest a re-expansion of the same subgraph, and the store persists the -one-step unfolding of every committed recursive member, so an unrolled copy -committed later resolves to the recursive layout it unrolls. A type committed +would digest a re-expansion of the same subgraph, and the store persists every +settled recursive group, so an unrolled copy committed later resolves to the +recursive layout it unrolls. A type committed without a digest resolved to a store-interned layout ref (a primitive or builtin layout through nominals) and is expanded again directly. The shared layout store owns recursive graph reduction and interning. @@ -19736,14 +19768,12 @@ lifetime and is not the shared-memory IPC transport. ### Layout Selection Recursive layout commitment interns unrolled copies of recursive nodes -before selecting boxed slots. Once a recursive component has exact structural -keys, its nodes also record their one-step encodings with settled child digests, -and the store keeps the one-step shape of every member of the settled group. -An acyclic node with that same encoding inherits the recursive node's key and -representation. A cyclic component that refers into a settled group (a copy of -a recursive type unrolled through one of its own recursive references) is -paired coinductively against the group's member shapes and inherits the -group's keys when every member pairs. Thus an unrolled record and its recursive +before selecting boxed slots. Recursive layout keys are settled by the shared +recursive-type reduction ("Recursive Type Digests") against groups the store +keeps across commits: an acyclic copy of a recursive position inherits that +position's key and representation, and a cyclic component that refers into a +settled group (a copy of a recursive type unrolled through one of its own +recursive references) inherits the group's keys. Thus an unrolled record and its recursive counterpart commit the same field storage, and two spellings of one recursive type commit the same layout whether they meet in one graph or in separate commits; graph sharing cannot make one inline and the other boxed. Procedure reuse by solved type identity consumes this consistent layout diff --git a/src/base/mod.zig b/src/base/mod.zig index 4bad242b632..11eb1c964f9 100644 --- a/src/base/mod.zig +++ b/src/base/mod.zig @@ -47,6 +47,7 @@ pub const SerialStringInterner = @import("SerialStringInterner.zig"); pub const InternedBytes = @import("InternedBytes.zig"); pub const SpecializationStrategy = @import("SpecializationStrategy.zig").SpecializationStrategy; pub const TypeDigestHasher = @import("TypeDigestHasher.zig"); +pub const recursive_digest = @import("recursive_digest.zig"); /// SHA-256 on the CPU's SHA-256 instructions, interchangeable with `std.crypto.hash.sha2.Sha256`. pub const Sha256 = @import("sha256.zig").Sha256; pub const ModuleIdentity = @import("module_identity.zig"); @@ -118,6 +119,7 @@ test { std.testing.refAllDecls(cpu_count); std.testing.refAllDecls(dev_diagnostics); std.testing.refAllDecls(TypeDigestHasher); + std.testing.refAllDecls(recursive_digest); std.testing.refAllDecls(@import("sha256.zig")); std.testing.refAllDecls(module_path_mod); std.testing.refAllDecls(@import("roc_version.zig")); diff --git a/src/base/recursive_digest.zig b/src/base/recursive_digest.zig new file mode 100644 index 00000000000..b00ab0a2365 --- /dev/null +++ b/src/base/recursive_digest.zig @@ -0,0 +1,846 @@ +//! Canonical digests of possibly recursive type graphs. +//! +//! Every type identity the compiler derives from a graph of type nodes +//! (Monotype and Lambda Mono type digests, procedure identities, recursive +//! layout keys) is reduced here, so each one digests a type as its infinite +//! unfolding rather than as whichever graph a program happened to draw for +//! it. Two programs draw the same recursive type as different graphs: one +//! may tie a single knot, another may roll the knot out several levels, or +//! tie one copy whose references lead into a second, separately tied copy. +//! The digest must not see any of that. +//! +//! A producer describes each node by its content (a digest of its +//! encoding with every child as a bare slot) and its ordered children, each +//! either another node of the graph or a digest settled earlier. The +//! reduction decomposes the graph into strongly connected components, which +//! resolve in reverse topological order, so every child outside a component +//! has its digest when the component resolves: +//! +//! - An acyclic node digests its one-step unfolding: its content with its +//! children's digests. When that unfolding is the unfolding of a position +//! of a recursive group settled earlier, the node is a rolled-out copy of +//! that position and takes its digest. +//! - A cyclic component that refers into a settled recursive group is first +//! paired coinductively against the group's recorded member shapes. Every +//! member of a component reaches every other, so either every member is +//! equivalent to a position of that group or none is; when the pairing +//! succeeds, the component spells the group's unfolding and its members +//! take the group's digests. +//! - Any other cyclic component is refined to its coarsest bisimulation +//! partition, its reduced positions are ordered by their final labels, the +//! group is rendered once with in-group references written as ranks, and +//! every member digests as its rank against that rendering. No child +//! outside such a component is equivalent to one of its members (that +//! child would lie in a settled group the component pairs with), so the +//! refinement sees exactly the reduced group whatever graph spelled it, +//! and the labels, their order, and the rendering are functions of the +//! infinite type alone. +//! +//! `Settled` keeps every resolved group's member shapes and unfoldings, so +//! these guarantees hold across separate reductions that share it. + +const std = @import("std"); +const TypeDigestHasher = @import("TypeDigestHasher.zig"); +const invariant = @import("invariant.zig").invariant; +const Allocator = std.mem.Allocator; + +/// Canonical digest of one type. +pub const Identity = [TypeDigestHasher.digest_length]u8; + +/// One ordered child of a node. +pub const Child = union(enum) { + /// Another node of the same graph. + node: u32, + /// A type whose digest was settled before this reduction. + settled: Identity, +}; + +/// How a node's digest was settled. +pub const Resolution = enum(u8) { + /// Not part of a cycle, and not a copy of a recursive position. + acyclic, + /// Not part of a cycle in this graph, but a rolled-out copy of a + /// position of a settled recursive group. + folded, + /// A member of a cyclic component. + cyclic, +}; + +/// Work counters for one reduction. +pub const Stats = struct { + /// Node encodings replayed while reducing cyclic components: labels, + /// the group rendering, and the unfolding index. + group_encodings: u64 = 0, +}; + +const no_position = std.math.maxInt(u32); +const unvisited = std.math.maxInt(u32); + +/// Everything reductions sharing one digest domain have settled: the +/// unfolding of every recursive position, the member shapes of every +/// recursive group, and the result of every component shape reduced. All of +/// it is content addressed, so it never needs rollback. +pub const Settled = struct { + /// Owns everything settled. + gpa: Allocator, + /// Separates this digest family from every other. + domain: []const u8, + /// One-step unfolding of every recursive position, mapped to the + /// position's digest. + unfoldings: std.AutoHashMapUnmanaged(Identity, Identity) = .empty, + /// Group and rank of every recursive position's digest. + member_of: std.AutoHashMapUnmanaged(Identity, MemberRef) = .empty, + groups: std.ArrayList(Group) = .empty, + ranks: std.ArrayList(Rank) = .empty, + rank_children: std.ArrayList(RankChild) = .empty, + /// Digests of every cyclic component reduced so far, keyed by its shape: + /// its members' labels and in-component references in member order. A + /// component's resolution is a function of exactly that shape. + shapes: std.AutoHashMapUnmanaged(Identity, ShapeDigests) = .empty, + shape_digests: std.ArrayList(Identity) = .empty, + + const MemberRef = struct { + group: u32, + rank: u32, + }; + + const Group = struct { + rank_start: u32, + rank_count: u32, + }; + + /// One reduced position of a settled group. + const Rank = struct { + digest: Identity, + content: Identity, + child_start: u32, + child_len: u32, + }; + + const RankChild = union(enum) { + /// A position of the same group. + rank: u32, + /// A type outside the group. + settled: Identity, + }; + + const ShapeDigests = struct { + start: u32, + len: u32, + }; + + pub fn init(gpa: Allocator, domain: []const u8) Settled { + return .{ .gpa = gpa, .domain = domain }; + } + + pub fn deinit(self: *Settled) void { + self.unfoldings.deinit(self.gpa); + self.member_of.deinit(self.gpa); + self.groups.deinit(self.gpa); + self.ranks.deinit(self.gpa); + self.rank_children.deinit(self.gpa); + self.shapes.deinit(self.gpa); + self.shape_digests.deinit(self.gpa); + } + + /// Forget everything settled, keeping capacity. + pub fn clear(self: *Settled) void { + self.unfoldings.clearRetainingCapacity(); + self.member_of.clearRetainingCapacity(); + self.groups.clearRetainingCapacity(); + self.ranks.clearRetainingCapacity(); + self.rank_children.clearRetainingCapacity(); + self.shapes.clearRetainingCapacity(); + self.shape_digests.clearRetainingCapacity(); + } + + /// A copy owned by `gpa`. + pub fn clone(self: *const Settled, gpa: Allocator) Allocator.Error!Settled { + var result = Settled.init(gpa, self.domain); + errdefer result.deinit(); + result.unfoldings = try self.unfoldings.clone(gpa); + result.member_of = try self.member_of.clone(gpa); + result.groups = try self.groups.clone(gpa); + result.ranks = try self.ranks.clone(gpa); + result.rank_children = try self.rank_children.clone(gpa); + result.shapes = try self.shapes.clone(gpa); + result.shape_digests = try self.shape_digests.clone(gpa); + return result; + } + + fn rank(self: *const Settled, group: u32, member_rank: u32) Rank { + const entry = self.groups.items[group]; + std.debug.assert(member_rank < entry.rank_count); + return self.ranks.items[entry.rank_start + member_rank]; + } + + fn rankChildren(self: *const Settled, entry: Rank) []const RankChild { + return self.rank_children.items[entry.child_start..][0..entry.child_len]; + } +}; + +/// One graph to reduce. Reset discards every node but keeps all capacity. +pub const Graph = struct { + gpa: Allocator, + nodes: std.ArrayList(Node) = .empty, + children: std.ArrayList(Child) = .empty, + frames: std.ArrayList(Frame) = .empty, + scc_stack: std.ArrayList(u32) = .empty, + members: std.ArrayList(u32) = .empty, + /// Member position of every node of the component being resolved; + /// `no_position` everywhere else. + positions: std.ArrayList(u32) = .empty, + labels: std.ArrayList(Identity) = .empty, + next_labels: std.ArrayList(Identity) = .empty, + sorted_labels: std.ArrayList(Identity) = .empty, + distinct_labels: std.AutoHashMapUnmanaged(Identity, u32) = .empty, + member_ranks: std.ArrayList(u32) = .empty, + representatives: std.ArrayList(u32) = .empty, + block_digests: std.ArrayList(Identity) = .empty, + unfolding_keys: std.ArrayList(Identity) = .empty, + candidate_groups: std.ArrayList(u32) = .empty, + paired: std.ArrayList(u32) = .empty, + pending_pairs: std.ArrayList(Pair) = .empty, + + const Node = struct { + content: Identity = undefined, + child_start: u32 = 0, + child_len: u32 = 0, + index: u32 = unvisited, + lowlink: u32 = unvisited, + on_stack: bool = false, + resolution: Resolution = undefined, + digest: ?Identity = null, + }; + + const Frame = struct { node: u32, next_child: u32 }; + + const Pair = struct { position: u32, rank: u32 }; + + pub fn init(gpa: Allocator) Graph { + return .{ .gpa = gpa }; + } + + pub fn deinit(self: *Graph) void { + inline for (list_fields) |field| @field(self, field).deinit(self.gpa); + self.distinct_labels.deinit(self.gpa); + } + + /// Start a new graph, including after allocation failure. + pub fn reset(self: *Graph) void { + inline for (list_fields) |field| @field(self, field).clearRetainingCapacity(); + self.distinct_labels.clearRetainingCapacity(); + } + + const list_fields = .{ "nodes", "children", "frames", "scc_stack", "members", "positions", "labels", "next_labels", "sorted_labels", "member_ranks", "representatives", "block_digests", "unfolding_keys", "candidate_groups", "paired", "pending_pairs" }; + + pub fn nodeCount(self: *const Graph) u32 { + return @intCast(self.nodes.items.len); + } + + /// Reserve a node before encoding it. + pub fn addNode(self: *Graph) Allocator.Error!u32 { + const node: u32 = @intCast(self.nodes.items.len); + try self.nodes.append(self.gpa, .{}); + return node; + } + + /// Begin a reserved node's ordered children. The children of one node + /// are appended before the next node begins. + pub fn beginNode(self: *Graph, node: u32) void { + self.nodes.items[node].child_start = @intCast(self.children.items.len); + } + + pub fn putChild(self: *Graph, child: Child) Allocator.Error!void { + try self.children.append(self.gpa, child); + } + + /// Finish a node with its content: the digest of its encoding with + /// every child as a bare slot. + pub fn endNode(self: *Graph, node: u32, content: Identity) void { + const entry = &self.nodes.items[node]; + entry.child_len = @intCast(self.children.items.len - entry.child_start); + entry.content = content; + } + + /// The digest of a node of a resolved graph. + pub fn digest(self: *const Graph, node: u32) Identity { + return self.nodes.items[node].digest orelse + invariant("recursive digest read a node of an unresolved graph", .{}); + } + + /// How a node of a resolved graph was settled. + pub fn resolution(self: *const Graph, node: u32) Resolution { + std.debug.assert(self.nodes.items[node].digest != null); + return self.nodes.items[node].resolution; + } + + /// Settle the digest of every node. + pub fn resolve(self: *Graph, settled: *Settled, stats: ?*Stats) Allocator.Error!void { + try self.positions.resize(self.gpa, self.nodes.items.len); + @memset(self.positions.items, no_position); + var next_index: u32 = 0; + for (0..self.nodes.items.len) |start_index| { + const start: u32 = @intCast(start_index); + if (self.nodes.items[start].index != unvisited) continue; + try self.visit(start, &next_index); + try self.frames.append(self.gpa, .{ .node = start, .next_child = 0 }); + while (self.frames.items.len != 0) { + const frame = &self.frames.items[self.frames.items.len - 1]; + const node = frame.node; + const children = self.childrenOf(node); + if (frame.next_child < children.len) { + const child = children[frame.next_child]; + frame.next_child += 1; + const target = switch (child) { + .settled => continue, + .node => |target| target, + }; + const target_entry = &self.nodes.items[target]; + if (target_entry.index == unvisited) { + try self.visit(target, &next_index); + try self.frames.append(self.gpa, .{ .node = target, .next_child = 0 }); + } else if (target_entry.on_stack) { + const entry = &self.nodes.items[node]; + entry.lowlink = @min(entry.lowlink, target_entry.index); + } + continue; + } + _ = self.frames.pop(); + const entry = self.nodes.items[node]; + if (self.frames.items.len != 0) { + const parent = &self.nodes.items[self.frames.items[self.frames.items.len - 1].node]; + parent.lowlink = @min(parent.lowlink, entry.lowlink); + } + if (entry.lowlink != entry.index) continue; + self.members.clearRetainingCapacity(); + while (true) { + const member = self.scc_stack.pop() orelse + invariant("recursive digest popped past its component root", .{}); + self.nodes.items[member].on_stack = false; + try self.members.append(self.gpa, member); + if (member == node) break; + } + try self.resolveComponent(settled, stats); + } + } + } + + fn visit(self: *Graph, node: u32, next_index: *u32) Allocator.Error!void { + const entry = &self.nodes.items[node]; + entry.index = next_index.*; + entry.lowlink = next_index.*; + entry.on_stack = true; + next_index.* += 1; + try self.scc_stack.append(self.gpa, node); + } + + fn childrenOf(self: *const Graph, node: u32) []const Child { + const entry = self.nodes.items[node]; + return self.children.items[entry.child_start..][0..entry.child_len]; + } + + /// The node a child names inside the component being resolved, if any. + fn memberChild(self: *const Graph, child: Child) ?u32 { + return switch (child) { + .settled => null, + .node => |target| if (self.positions.items[target] == no_position) null else target, + }; + } + + /// The digest of a child outside the component being resolved. + fn childDigest(self: *const Graph, child: Child) Identity { + return switch (child) { + .settled => |settled_digest| settled_digest, + .node => |target| self.nodes.items[target].digest orelse + invariant("recursive digest read a child before resolving it", .{}), + }; + } + + fn finish(self: *Graph, node: u32, node_digest: Identity, how: Resolution) void { + const entry = &self.nodes.items[node]; + if (entry.digest != null) invariant("recursive digest resolved a node twice", .{}); + entry.digest = node_digest; + entry.resolution = how; + } + + /// A node's content with every child as its digest. + fn unfoldingKey(self: *const Graph, domain: []const u8, node: u32) Identity { + var hasher = TypeDigestHasher.init(); + writeBytes(&hasher, domain); + writeBytes(&hasher, "unfolding"); + hasher.update(&self.nodes.items[node].content); + const children = self.childrenOf(node); + writeU32(&hasher, @intCast(children.len)); + for (children) |child| hasher.update(&self.childDigest(child)); + return hasher.finalResult(); + } + + fn resolveComponent(self: *Graph, settled: *Settled, stats: ?*Stats) Allocator.Error!void { + const members = self.members.items; + if (members.len == 1) { + const node = members[0]; + const self_edge = for (self.childrenOf(node)) |child| { + if (child == .node and child.node == node) break true; + } else false; + if (!self_edge) { + const unfolding = self.unfoldingKey(settled.domain, node); + if (settled.unfoldings.get(unfolding)) |position_digest| { + self.finish(node, position_digest, .folded); + } else { + self.finish(node, unfolding, .acyclic); + } + return; + } + } + for (members, 0..) |node, pos| self.positions.items[node] = @intCast(pos); + defer for (members) |node| { + self.positions.items[node] = no_position; + }; + try self.resolveCyclicComponent(settled, stats); + } + + fn resolveCyclicComponent(self: *Graph, settled: *Settled, stats: ?*Stats) Allocator.Error!void { + const members = self.members.items; + const member_count = members.len; + + // Every member starts from its label: its content with every child + // outside the component as its digest and every child inside it as + // a bare marker. + try self.labels.resize(self.gpa, member_count); + try self.next_labels.resize(self.gpa, member_count); + var labels = self.labels.items; + var next_labels = self.next_labels.items; + for (members, 0..) |node, pos| { + var hasher = TypeDigestHasher.init(); + writeBytes(&hasher, "label"); + hasher.update(&self.nodes.items[node].content); + for (self.childrenOf(node)) |child| { + if (self.memberChild(child) != null) { + writeBytes(&hasher, "member"); + } else { + writeBytes(&hasher, "child"); + hasher.update(&self.childDigest(child)); + } + } + labels[pos] = hasher.finalResult(); + } + if (stats) |s| s.group_encodings += member_count; + + // Labels and in-component references determine everything below, + // so a component of a shape reduced before takes that result. + const shape = shape: { + var hasher = TypeDigestHasher.init(); + writeBytes(&hasher, "shape"); + writeU32(&hasher, @intCast(member_count)); + for (members, 0..) |node, pos| { + hasher.update(&labels[pos]); + for (self.childrenOf(node)) |child| { + const target = self.memberChild(child) orelse continue; + writeU32(&hasher, self.positions.items[target]); + } + writeBytes(&hasher, "end-member"); + } + break :shape hasher.finalResult(); + }; + if (settled.shapes.get(shape)) |recorded| { + if (recorded.len != member_count) invariant("recursive digest shape recorded a different member count", .{}); + for (members, settled.shape_digests.items[recorded.start..][0..recorded.len]) |node, member_digest| { + self.finish(node, member_digest, .cyclic); + } + return; + } + + try self.paired.resize(self.gpa, member_count); + if (try self.foldOntoSettledGroup(settled)) { + try reserveShape(settled, member_count); + for (members, self.paired.items) |node, member_rank| { + self.finish(node, settled.rank(self.candidate_groups.items[0], member_rank).digest, .cyclic); + } + recordShape(settled, shape, self, members); + return; + } + + // Refine to the coarsest stable partition: each round relabels a + // member by its label followed by the labels of its ordered + // in-component children, until a round separates no members. + const distinct_labels = &self.distinct_labels; + distinct_labels.clearRetainingCapacity(); + for (labels) |label| try distinct_labels.put(self.gpa, label, no_position); + var label_count: u32 = distinct_labels.count(); + while (true) { + distinct_labels.clearRetainingCapacity(); + for (members, 0..) |node, pos| { + var hasher = TypeDigestHasher.init(); + hasher.update(&labels[pos]); + for (self.childrenOf(node)) |child| { + const target = self.memberChild(child) orelse continue; + hasher.update(&labels[self.positions.items[target]]); + } + next_labels[pos] = hasher.finalResult(); + try distinct_labels.put(self.gpa, next_labels[pos], no_position); + } + const next_count: u32 = distinct_labels.count(); + const stable = next_count == label_count; + std.mem.swap([]Identity, &labels, &next_labels); + label_count = next_count; + if (stable) break; + } + + // Group order: the reduced positions sorted by label. + const block_count = label_count; + try self.sorted_labels.resize(self.gpa, block_count); + const sorted_labels = self.sorted_labels.items; + { + var it = distinct_labels.keyIterator(); + var next: usize = 0; + while (it.next()) |label| : (next += 1) sorted_labels[next] = label.*; + std.debug.assert(next == block_count); + } + std.mem.sort(Identity, sorted_labels, {}, identityLessThan); + for (sorted_labels, 0..) |label, block_rank| { + const slot = distinct_labels.getPtr(label) orelse unreachable; // sorted_labels holds exactly the keys of distinct_labels + slot.* = @intCast(block_rank); + } + try self.member_ranks.resize(self.gpa, member_count); + const member_ranks = self.member_ranks.items; + try self.representatives.resize(self.gpa, block_count); + const representatives = self.representatives.items; + @memset(representatives, no_position); + for (members, 0..) |node, pos| { + const block_rank = distinct_labels.get(labels[pos]) orelse unreachable; // the final round put every member's label into distinct_labels + member_ranks[pos] = block_rank; + // The partition is stable, so any member represents its position. + if (representatives[block_rank] == no_position) representatives[block_rank] = node; + } + + // One group rendering: every reduced position in rank order with + // in-group references written as ranks. + var group_hasher = TypeDigestHasher.init(); + writeBytes(&group_hasher, settled.domain); + writeBytes(&group_hasher, "group"); + writeU32(&group_hasher, block_count); + for (representatives) |rep| { + writeBytes(&group_hasher, "position"); + group_hasher.update(&self.nodes.items[rep].content); + for (self.childrenOf(rep)) |child| { + if (self.memberChild(child)) |target| { + writeBytes(&group_hasher, "rank"); + writeU32(&group_hasher, member_ranks[self.positions.items[target]]); + } else { + writeBytes(&group_hasher, "child"); + group_hasher.update(&self.childDigest(child)); + } + } + } + if (stats) |s| s.group_encodings += block_count; + const group_digest = group_hasher.finalResult(); + try self.block_digests.resize(self.gpa, block_count); + const block_digests = self.block_digests.items; + for (block_digests, 0..) |*block_digest, block_rank| { + var hasher = TypeDigestHasher.init(); + writeBytes(&hasher, settled.domain); + writeBytes(&hasher, "group-member"); + hasher.update(&group_digest); + writeU32(&hasher, @intCast(block_rank)); + block_digest.* = hasher.finalResult(); + } + for (members, member_ranks) |node, block_rank| self.finish(node, block_digests[block_rank], .cyclic); + + // A rolled-out copy of a position unfolds to the position's one-step + // encoding, so it folds to the position's digest. + try self.unfolding_keys.resize(self.gpa, block_count); + for (representatives, self.unfolding_keys.items) |rep, *key| key.* = self.unfoldingKey(settled.domain, rep); + if (stats) |s| s.group_encodings += block_count; + + // Reserve everything before committing anything, so allocation + // failure leaves `settled` as it was. + const record_group = !settled.member_of.contains(block_digests[0]); + try settled.unfoldings.ensureUnusedCapacity(settled.gpa, block_count); + if (record_group) { + try settled.member_of.ensureUnusedCapacity(settled.gpa, block_count); + try settled.groups.ensureUnusedCapacity(settled.gpa, 1); + try settled.ranks.ensureUnusedCapacity(settled.gpa, block_count); + var child_count: usize = 0; + for (representatives) |rep| child_count += self.childrenOf(rep).len; + try settled.rank_children.ensureUnusedCapacity(settled.gpa, child_count); + } + try reserveShape(settled, member_count); + + for (self.unfolding_keys.items, block_digests) |key, block_digest| { + const gop = settled.unfoldings.getOrPutAssumeCapacity(key); + if (gop.found_existing) { + // Position digests are intrinsic, so an equivalent position + // settled earlier must agree. + std.debug.assert(std.mem.eql(u8, gop.value_ptr, &block_digest)); + } else { + gop.value_ptr.* = block_digest; + } + } + if (record_group) { + const group: u32 = @intCast(settled.groups.items.len); + settled.groups.appendAssumeCapacity(.{ .rank_start = @intCast(settled.ranks.items.len), .rank_count = block_count }); + for (representatives, block_digests, 0..) |rep, block_digest, block_rank| { + const child_start: u32 = @intCast(settled.rank_children.items.len); + for (self.childrenOf(rep)) |child| { + settled.rank_children.appendAssumeCapacity(if (self.memberChild(child)) |target| + .{ .rank = member_ranks[self.positions.items[target]] } + else + .{ .settled = self.childDigest(child) }); + } + settled.ranks.appendAssumeCapacity(.{ + .digest = block_digest, + .content = self.nodes.items[rep].content, + .child_start = child_start, + .child_len = @intCast(settled.rank_children.items.len - child_start), + }); + settled.member_of.putAssumeCapacityNoClobber(block_digest, .{ .group = group, .rank = @intCast(block_rank) }); + } + } + recordShape(settled, shape, self, members); + } + + /// Pair the component coinductively against each settled group it + /// refers into. On success the pairing group is `candidate_groups[0]` + /// and `paired` holds each member's rank in it. + fn foldOntoSettledGroup(self: *Graph, settled: *const Settled) Allocator.Error!bool { + const members = self.members.items; + self.candidate_groups.clearRetainingCapacity(); + for (members) |node| { + for (self.childrenOf(node)) |child| { + if (self.memberChild(child) != null) continue; + const member_ref = settled.member_of.get(self.childDigest(child)) orelse continue; + if (std.mem.indexOfScalar(u32, self.candidate_groups.items, member_ref.group) == null) { + try self.candidate_groups.append(self.gpa, member_ref.group); + } + } + } + for (self.candidate_groups.items, 0..) |group, candidate| { + const rank_count = settled.groups.items[group].rank_count; + for (0..rank_count) |start_rank| { + if (!try self.pairWithGroup(settled, group, @intCast(start_rank))) continue; + self.candidate_groups.items[0] = self.candidate_groups.items[candidate]; + return true; + } + } + return false; + } + + fn pairWithGroup(self: *Graph, settled: *const Settled, group: u32, start_rank: u32) Allocator.Error!bool { + const members = self.members.items; + const paired = self.paired.items; + @memset(paired, no_position); + self.pending_pairs.clearRetainingCapacity(); + try self.pending_pairs.append(self.gpa, .{ .position = 0, .rank = start_rank }); + while (self.pending_pairs.pop()) |pair| { + if (paired[pair.position] != no_position) { + if (paired[pair.position] != pair.rank) return false; + continue; + } + paired[pair.position] = pair.rank; + const node = members[pair.position]; + const expected = settled.rank(group, pair.rank); + if (!std.mem.eql(u8, &self.nodes.items[node].content, &expected.content)) return false; + const actual_children = self.childrenOf(node); + const expected_children = settled.rankChildren(expected); + if (actual_children.len != expected_children.len) return false; + for (actual_children, expected_children) |actual, expected_child| { + if (self.memberChild(actual)) |target| { + switch (expected_child) { + .rank => |child_rank| try self.pending_pairs.append(self.gpa, .{ .position = self.positions.items[target], .rank = child_rank }), + .settled => return false, + } + } else { + const actual_digest = self.childDigest(actual); + const expected_digest = switch (expected_child) { + .rank => |child_rank| settled.rank(group, child_rank).digest, + .settled => |settled_digest| settled_digest, + }; + if (!std.mem.eql(u8, &actual_digest, &expected_digest)) return false; + } + } + } + return std.mem.indexOfScalar(u32, paired, no_position) == null; + } +}; + +fn reserveShape(settled: *Settled, member_count: usize) Allocator.Error!void { + try settled.shapes.ensureUnusedCapacity(settled.gpa, 1); + try settled.shape_digests.ensureUnusedCapacity(settled.gpa, member_count); +} + +fn recordShape(settled: *Settled, shape: Identity, graph: *const Graph, members: []const u32) void { + settled.shapes.putAssumeCapacityNoClobber(shape, .{ + .start = @intCast(settled.shape_digests.items.len), + .len = @intCast(members.len), + }); + for (members) |node| settled.shape_digests.appendAssumeCapacity(graph.digest(node)); +} + +fn identityLessThan(_: void, lhs: Identity, rhs: Identity) bool { + return std.mem.order(u8, &lhs, &rhs) == .lt; +} + +/// Hash a length-prefixed byte string. +pub fn writeBytes(hasher: *TypeDigestHasher, bytes: []const u8) void { + writeU32(hasher, @intCast(bytes.len)); + hasher.update(bytes); +} + +/// Hash a little-endian u32. +pub fn writeU32(hasher: *TypeDigestHasher, value: u32) void { + const bytes: [4]u8 = .{ @truncate(value), @truncate(value >> 8), @truncate(value >> 16), @truncate(value >> 24) }; + hasher.update(&bytes); +} + +fn testContent(tag: []const u8) Identity { + var hasher = TypeDigestHasher.init(); + writeBytes(&hasher, tag); + return hasher.finalResult(); +} + +/// Adds a node whose children are all nodes of `graph`. +fn testNode(graph: *Graph, node: u32, tag: []const u8, children: []const Child) Allocator.Error!void { + graph.beginNode(node); + for (children) |child| try graph.putChild(child); + graph.endNode(node, testContent(tag)); +} + +test "a knot whose references lead into a separately tied copy digests as that copy" { + const gpa = std.testing.allocator; + var settled = Settled.init(gpa, "test"); + defer settled.deinit(); + var graph = Graph.init(gpa); + defer graph.deinit(); + + // c = pair(c, c); b1 = pair(b2, b1); b2 = pair(b1, c). All three are the + // same infinite type, but {b1, b2} is a component of its own whose + // references lead into c's. + const c = try graph.addNode(); + const b1 = try graph.addNode(); + const b2 = try graph.addNode(); + try testNode(&graph, b1, "pair", &.{ .{ .node = b2 }, .{ .node = b1 } }); + try testNode(&graph, b2, "pair", &.{ .{ .node = b1 }, .{ .node = c } }); + try testNode(&graph, c, "pair", &.{ .{ .node = c }, .{ .node = c } }); + try graph.resolve(&settled, null); + try std.testing.expectEqual(graph.digest(c), graph.digest(b1)); + try std.testing.expectEqual(graph.digest(c), graph.digest(b2)); + + // The same type alone in a fresh reduction, with nothing settled. + var fresh_settled = Settled.init(gpa, "test"); + defer fresh_settled.deinit(); + var fresh = Graph.init(gpa); + defer fresh.deinit(); + const solo = try fresh.addNode(); + try testNode(&fresh, solo, "pair", &.{ .{ .node = solo }, .{ .node = solo } }); + try fresh.resolve(&fresh_settled, null); + try std.testing.expectEqual(fresh.digest(solo), graph.digest(b1)); +} + +test "a rolled-out copy ending in a settled knot digests as the knot" { + const gpa = std.testing.allocator; + var settled = Settled.init(gpa, "test"); + defer settled.deinit(); + var graph = Graph.init(gpa); + defer graph.deinit(); + + // list = List(node); node = [Leaf, Branch(list)]. + const node = try graph.addNode(); + const list = try graph.addNode(); + try testNode(&graph, node, "union", &.{.{ .node = list }}); + try testNode(&graph, list, "list", &.{.{ .node = node }}); + try graph.resolve(&settled, null); + const node_digest = graph.digest(node); + const list_digest = graph.digest(list); + + // A later reduction rolls the knot out twice before reaching the + // settled copy. + graph.reset(); + const outer = try graph.addNode(); + const outer_list = try graph.addNode(); + const inner = try graph.addNode(); + try testNode(&graph, outer, "union", &.{.{ .node = outer_list }}); + try testNode(&graph, outer_list, "list", &.{.{ .node = inner }}); + try testNode(&graph, inner, "union", &.{.{ .settled = list_digest }}); + try graph.resolve(&settled, null); + try std.testing.expectEqual(node_digest, graph.digest(outer)); + try std.testing.expectEqual(list_digest, graph.digest(outer_list)); + try std.testing.expectEqual(node_digest, graph.digest(inner)); + try std.testing.expectEqual(Resolution.folded, graph.resolution(inner)); +} + +test "a cyclic component spelling a settled group through some references pairs with it" { + const gpa = std.testing.allocator; + var settled = Settled.init(gpa, "test"); + defer settled.deinit(); + var graph = Graph.init(gpa); + defer graph.deinit(); + + // tree = [Leaf, Node(record)]; record = { left: tree, right: tree }. + const tree = try graph.addNode(); + const record = try graph.addNode(); + try testNode(&graph, tree, "tree", &.{.{ .node = record }}); + try testNode(&graph, record, "record", &.{ .{ .node = tree }, .{ .node = tree } }); + try graph.resolve(&settled, null); + const tree_digest = graph.digest(tree); + const record_digest = graph.digest(record); + + // Its own knot through `left`, the settled copy through `right`. + graph.reset(); + const copy_tree = try graph.addNode(); + const copy_record = try graph.addNode(); + try testNode(&graph, copy_tree, "tree", &.{.{ .node = copy_record }}); + try testNode(&graph, copy_record, "record", &.{ .{ .node = copy_tree }, .{ .settled = tree_digest } }); + try graph.resolve(&settled, null); + try std.testing.expectEqual(tree_digest, graph.digest(copy_tree)); + try std.testing.expectEqual(record_digest, graph.digest(copy_record)); + try std.testing.expectEqual(Resolution.cyclic, graph.resolution(copy_tree)); +} + +test "a cyclic component that refers into an inequivalent settled group stays its own type" { + const gpa = std.testing.allocator; + var settled = Settled.init(gpa, "test"); + defer settled.deinit(); + var graph = Graph.init(gpa); + defer graph.deinit(); + + const tree = try graph.addNode(); + const record = try graph.addNode(); + try testNode(&graph, tree, "tree", &.{.{ .node = record }}); + try testNode(&graph, record, "record", &.{ .{ .node = tree }, .{ .node = tree } }); + try graph.resolve(&settled, null); + const tree_digest = graph.digest(tree); + + // forest = [Empty, More(forest, tree)]: a recursive type that holds the + // settled tree without being one. + graph.reset(); + const forest = try graph.addNode(); + try testNode(&graph, forest, "tree", &.{ .{ .node = forest }, .{ .settled = tree_digest } }); + try graph.resolve(&settled, null); + try std.testing.expect(!std.mem.eql(u8, &tree_digest, &graph.digest(forest))); + try std.testing.expect(settled.member_of.get(graph.digest(forest)).?.group != settled.member_of.get(tree_digest).?.group); +} + +test "a knot digests the same however far it is rolled out" { + const gpa = std.testing.allocator; + var digests: [3]Identity = undefined; + for (&digests, 0..) |*out, rolls| { + var settled = Settled.init(gpa, "test"); + defer settled.deinit(); + var graph = Graph.init(gpa); + defer graph.deinit(); + // A cycle of `rolls + 1` list nodes over one union node per roll. + const count: u32 = @intCast(rolls + 1); + var nodes: [6]u32 = undefined; + for (0..2 * count) |i| nodes[i] = try graph.addNode(); + for (0..count) |i| { + const next = nodes[2 * ((i + 1) % count)]; + try testNode(&graph, nodes[2 * i], "union", &.{.{ .node = nodes[2 * i + 1] }}); + try testNode(&graph, nodes[2 * i + 1], "list", &.{.{ .node = next }}); + } + try graph.resolve(&settled, null); + out.* = graph.digest(nodes[0]); + } + try std.testing.expectEqual(digests[0], digests[1]); + try std.testing.expectEqual(digests[0], digests[2]); +} diff --git a/src/layout/store.zig b/src/layout/store.zig index 67f0160dd20..6340def9103 100644 --- a/src/layout/store.zig +++ b/src/layout/store.zig @@ -2,6 +2,7 @@ const std = @import("std"); const TypeDigestHasher = @import("base").TypeDigestHasher; +const recursive_digest = @import("base").recursive_digest; const Allocator = std.mem.Allocator; const tracy = @import("tracy"); const base = @import("base"); @@ -147,15 +148,10 @@ pub const Store = struct { recursive_layout_keys: collections.SafeList(RecursiveLayoutKey), /// Position in `recursive_layout_keys` of each key, looked up by key. interned_recursive_graphs: RecursiveGraphIndex, - // One-step unfolding of every committed recursive node, mapped to that - // node's recursive key. Persisting these across commits lets an acyclic - // unrolled copy committed later (possibly against `committed` leaves) - // resolve to the recursive layout it unrolls. - recursive_unfoldings: RecursiveGraphUnfoldings, - // One-step shape of every member of every settled recursive group, so a - // cyclic component committed later that spells one of those groups with - // references back into it resolves to that group's identities. - recursive_groups: RecursiveGroupShapes, + // Every recursive group settled by a commit, so a graph committed later + // that unrolls one (possibly against `committed` leaves) or ties a copy + // whose references lead back into one resolves to that group's keys. + recursive_digests: recursive_digest.Settled, // The target's usize type (32-bit or 64-bit) - used for layout calculations // This is critical for cross-compilation (e.g., compiling for wasm32 on a 64-bit host) @@ -203,8 +199,7 @@ pub const Store = struct { .scratch_intern_key = .empty, .recursive_layout_keys = .{}, .interned_recursive_graphs = .empty, - .recursive_unfoldings = .empty, - .recursive_groups = .{}, + .recursive_digests = recursive_digest.Settled.init(allocator, recursive_digest_domain), .target_usize = target_usize, .digest_cache = try digest_mod.DigestCache.create(allocator), }; @@ -386,8 +381,7 @@ pub const Store = struct { self.scratch_field_offsets.deinit(self.allocator); self.recursive_layout_keys.deinit(self.allocator); self.interned_recursive_graphs.deinit(self.allocator); - self.recursive_unfoldings.deinit(self.allocator); - self.recursive_groups.deinit(self.allocator); + self.recursive_digests.deinit(); self.digest_cache.destroy(self.allocator); self.commit_scratch.promote(self.allocator).deinit(); } @@ -1000,29 +994,20 @@ pub const Store = struct { pub fn recursiveLayoutKeys(self: *const Self) []const RecursiveLayoutKey { return self.recursive_layout_keys.items.items; } - /// Recursive key per one-step unfolding of a committed recursive node. - pub const RecursiveGraphUnfoldings = std.AutoHashMapUnmanaged(RecursiveGraphAnalysis.RecursiveKey, RecursiveGraphAnalysis.RecursiveKey); - /// One-step shapes of the members of settled recursive groups. - pub const RecursiveGroupShapes = RecursiveGraphAnalysis.GroupShapes; + /// Digest domain of recursive layout keys. + pub const recursive_digest_domain = "roc.layout.recursive-graph.v3"; - /// Canonical identity of every cyclic node of a temporary layout graph. + /// Canonical identity of every recursive node of a temporary layout + /// graph. /// - /// The graph's condensation is discovered by Tarjan's algorithm, which - /// pops components children-first, so every reference leaving a popped - /// component already has a settled digest. An acyclic node digests its - /// encoding with those child digests. A cyclic component is reduced by - /// bisimulation refinement over content labels, its reduced positions are - /// ordered by their final labels, the whole group is rendered once with - /// group-relative back-references, and every member is identified by its - /// position against that rendering's digest. Nominal nodes are transparent: - /// references resolve through them and they carry no identity of their - /// own. A `committed` leaf carries the digest its node settled to when it - /// was committed, so it digests exactly like a re-expansion of that - /// subgraph; one-step unfoldings of recursive members persist in the store - /// across commits for the same reason. A cyclic component that refers into - /// a settled recursive group is first paired coinductively against that - /// group's stored member shapes, and takes the group's identities when it - /// spells the same unfolding. + /// Every non-nominal node of the graph is reduced by the shared + /// recursive-type reduction (`base.recursive_digest`) against the store's + /// settled recursive groups. Nominal nodes are transparent: references + /// resolve through them and they carry no identity of their own. A + /// `committed` leaf carries the digest its node settled to when it was + /// committed, so it digests exactly like a re-expansion of that + /// subgraph, and a reference to a canonical layout is that layout's + /// index. const RecursiveGraphAnalysis = struct { /// Identity per recursive node or its unrolled copy; null otherwise. keys: []?RecursiveKey, @@ -1032,46 +1017,58 @@ pub const Store = struct { pub const RecursiveKey = graph_mod.Digest; - const no_component = std.math.maxInt(u32); - /// Component marker for `committed` leaves, whose digest is given. - const committed_component = std.math.maxInt(u32) - 1; - const unvisited = std.math.maxInt(u32); - const domain = "roc.layout.recursive-graph.v2"; + const no_node = std.math.maxInt(u32); /// Analyze `graph` with temporaries and results in `scratch`; the - /// store's `unfoldings` grow in `unfoldings_allocator`. + /// store's `settled` groups grow in their own allocator. fn init( scratch: Allocator, - unfoldings_allocator: Allocator, graph: *const LayoutGraph, - unfoldings: *RecursiveGraphUnfoldings, - groups: *GroupShapes, + settled: *recursive_digest.Settled, ) Allocator.Error!RecursiveGraphAnalysis { const node_count = graph.nodes.items.len; const keys = try scratch.alloc(?RecursiveKey, node_count); @memset(keys, null); const node_digests = try scratch.alloc(?RecursiveKey, node_count); - var engine = try Engine.init(scratch, unfoldings_allocator, graph, keys, unfoldings, groups); - for (graph.nodes.items, 0..) |node, i| { - switch (node) { - .committed => |committed| { - engine.visit_index[i] = 0; - engine.component[i] = committed_component; - engine.digests[i] = graph.committedDigest(committed); - }, - .pending, .nominal, .box, .list, .closure, .erased_callable, .struct_, .tag_union => {}, - } + // The reduction node of every node that is neither nominal nor + // committed. + const reduction_node = try scratch.alloc(u32, node_count); + var reduction = recursive_digest.Graph.init(scratch); + for (graph.nodes.items, reduction_node) |node, *index| { + index.* = switch (node) { + .nominal, .committed => no_node, + .pending, .box, .list, .closure, .erased_callable, .struct_, .tag_union => try reduction.addNode(), + }; + } + for (reduction_node, 0..) |index, i| { + if (index == no_node) continue; + reduction.beginNode(index); + var hasher = TypeDigestHasher.init(); + try encodeNode(graph, @intCast(i), ContentSink{ + .graph = graph, + .reduction = &reduction, + .reduction_node = reduction_node, + .hasher = &hasher, + }); + reduction.endNode(index, hasher.finalResult()); } - for (graph.nodes.items, 0..) |node, i| { - if (node == .nominal) continue; - if (engine.visit_index[i] != unvisited) continue; - try engine.strongConnect(@intCast(i)); + try reduction.resolve(settled, null); + + for (reduction_node, keys) |index, *key| { + if (index == no_node) continue; + switch (reduction.resolution(index)) { + .acyclic => {}, + .folded, .cyclic => key.* = reduction.digest(index), + } } for (node_digests, 0..) |*digest, i| { digest.* = switch (resolveNominalRef(graph, .{ .local = @fromBackingInt(@intCast(i)) })) { .canonical => null, - .local => |node_id| engine.digests[@backingInt(node_id)], + .local => |node_id| switch (graph.getNode(node_id)) { + .committed => |committed| graph.committedDigest(committed), + .pending, .nominal, .box, .list, .closure, .erased_callable, .struct_, .tag_union => reduction.digest(reduction_node[@backingInt(node_id)]), + }, }; } @@ -1096,65 +1093,12 @@ pub const Store = struct { base.invariant("layout.Store invariant violated: logical layout graph contained a nominal-only cycle", .{}); } - /// A child reference in a node's one-step shape. - const ShapeChild = union(enum) { - canonical: Idx, - settled: RecursiveKey, - /// Rank of a member of the same recursive group. - member: u32, - /// Node index of a member of the component being resolved. - component_node: u32, - }; - - /// One-step shapes of every member of every settled recursive group, - /// owned by the store so they outlive the commit that settled them. - pub const GroupShapes = struct { - /// Group and rank of every settled recursive member digest. - member_of: std.AutoHashMapUnmanaged(RecursiveKey, MemberRef) = .empty, - groups: std.ArrayList(Group) = .empty, - ranks: std.ArrayList(Rank) = .empty, - content: std.ArrayList(u8) = .empty, - children: std.ArrayList(ShapeChild) = .empty, - - const MemberRef = struct { - group: u32, - rank: u32, - }; - - const Group = struct { - rank_start: u32, - rank_count: u32, - }; - - const Rank = struct { - digest: RecursiveKey, - content_start: u32, - content_len: u32, - child_start: u32, - child_len: u32, - }; - - pub fn deinit(self_shapes: *GroupShapes, allocator: Allocator) void { - self_shapes.children.deinit(allocator); - self_shapes.content.deinit(allocator); - self_shapes.ranks.deinit(allocator); - self_shapes.groups.deinit(allocator); - self_shapes.member_of.deinit(allocator); - } - - fn rank(self_shapes: *const GroupShapes, group: u32, member_rank: u32) Rank { - const entry = self_shapes.groups.items[group]; - std.debug.assert(member_rank < entry.rank_count); - return self_shapes.ranks.items[entry.rank_start + member_rank]; - } - }; - /// The single encoding of one non-nominal node: a content-kind /// discriminator, every non-reference value, and its ordered child /// references resolved through nominals via `sink.child`. fn encodeNode(graph: *const LayoutGraph, node_index: u32, sink: anytype) Allocator.Error!void { switch (graph.getNode(@fromBackingInt(@intCast(node_index)))) { - .nominal, .committed => unreachable, // the walk resolves children through nominals and pre-settles committed leaves + .nominal, .committed => unreachable, // the analysis resolves children through nominals and settles committed leaves without encoding them .pending => base.invariant("layout.Store invariant violated: commitGraph received a reserved node that was never filled", .{}), .box => |child| { try sink.writeByte(0); @@ -1191,632 +1135,44 @@ pub const Store = struct { } } - fn littleEndianBytes(value: u32) [4]u8 { - return .{ @truncate(value), @truncate(value >> 8), @truncate(value >> 16), @truncate(value >> 24) }; - } - - fn hashU32(hasher: *TypeDigestHasher, value: u32) void { - const bytes = littleEndianBytes(value); - hasher.update(&bytes); - } - - fn digestLessThan(_: void, lhs: RecursiveKey, rhs: RecursiveKey) bool { - for (lhs, rhs) |lhs_byte, rhs_byte| { - if (lhs_byte != rhs_byte) return lhs_byte < rhs_byte; - } - return false; - } - - const Engine = struct { - /// Allocator of this analysis's temporaries. - allocator: Allocator, - unfoldings_allocator: Allocator, + /// Hashes a node's encoding with every child as a bare slot and + /// hands its ordered children to the reduction. + const ContentSink = struct { graph: *const LayoutGraph, - keys: []?RecursiveKey, - /// Settled digest per node, valid once `component[node]` is set. - digests: []RecursiveKey, - /// Popped component per node, `no_component` until popped. - component: []u32, - /// Member position within its popped component. - position: []u32, - component_count: u32 = 0, - visit_index: []u32, - low_link: []u32, - on_stack: []bool, - next_visit: u32 = 0, - stack: std.ArrayList(u32) = .empty, - members: std.ArrayList(u32) = .empty, - /// In-component child node indices of the component being - /// resolved, contiguous per member. - edges: std.ArrayList(u32) = .empty, - edge_start: std.ArrayList(u32) = .empty, - edge_len: std.ArrayList(u32) = .empty, - render_buf: std.ArrayList(u8) = .empty, - /// Exact one-step encodings of settled recursive nodes, owned by - /// the store so they outlive this commit. - unfoldings: *RecursiveGraphUnfoldings, - /// Settled recursive group shapes, owned by the store. - groups: *GroupShapes, - /// Scratch one-step shape of the node being paired. - shape_content: std.ArrayList(u8) = .empty, - shape_children: std.ArrayList(ShapeChild) = .empty, - - fn init( - allocator: Allocator, - unfoldings_allocator: Allocator, - graph: *const LayoutGraph, - keys: []?RecursiveKey, - unfoldings: *RecursiveGraphUnfoldings, - groups: *GroupShapes, - ) Allocator.Error!Engine { - const node_count = graph.nodes.items.len; - const digests = try allocator.alloc(RecursiveKey, node_count); - const component = try allocator.alloc(u32, node_count); - @memset(component, no_component); - const position = try allocator.alloc(u32, node_count); - const visit_index = try allocator.alloc(u32, node_count); - @memset(visit_index, unvisited); - const low_link = try allocator.alloc(u32, node_count); - const on_stack = try allocator.alloc(bool, node_count); - @memset(on_stack, false); - return .{ - .allocator = allocator, - .unfoldings_allocator = unfoldings_allocator, - .graph = graph, - .keys = keys, - .digests = digests, - .component = component, - .position = position, - .visit_index = visit_index, - .low_link = low_link, - .on_stack = on_stack, - .unfoldings = unfoldings, - .groups = groups, - }; - } - - /// Discovery sink: lists every local child for Tarjan's walk, in - /// encoding order. - const VisitSink = struct { - children: *std.ArrayList(u32), - allocator: Allocator, - - fn writeByte(_: VisitSink, _: u8) Allocator.Error!void {} - - fn writeU32(_: VisitSink, _: u32) Allocator.Error!void {} - - fn child(self_sink: VisitSink, ref: GraphRef) Allocator.Error!void { - switch (ref) { - .canonical => {}, - .local => |node_id| try self_sink.children.append(self_sink.allocator, @backingInt(node_id)), - } - } - }; - - /// Edge sink: collects the ordered children that lie inside the - /// component being resolved. - const EdgeSink = struct { - engine: *Engine, - component_id: u32, - - fn writeByte(_: EdgeSink, _: u8) Allocator.Error!void {} - - fn writeU32(_: EdgeSink, _: u32) Allocator.Error!void {} - - fn child(self_sink: EdgeSink, ref: GraphRef) Allocator.Error!void { - const child_index: u32 = switch (ref) { - .canonical => return, - .local => |node_id| @backingInt(node_id), - }; - const engine = self_sink.engine; - if (engine.component[child_index] != self_sink.component_id) return; - try engine.edges.append(engine.allocator, child_index); - } - }; - - /// Label sink: hashes a node's encoding with every child outside - /// the component rendered as its settled digest and every child - /// inside it as a bare positional marker. - const LabelSink = struct { - engine: *Engine, - hasher: *TypeDigestHasher, - component_id: u32, - - fn writeByte(self_sink: LabelSink, value: u8) Allocator.Error!void { - self_sink.hasher.update(&.{value}); - } - - fn writeU32(self_sink: LabelSink, value: u32) Allocator.Error!void { - hashU32(self_sink.hasher, value); - } - - fn child(self_sink: LabelSink, ref: GraphRef) Allocator.Error!void { - switch (ref) { - .canonical => |layout_idx| { - try self_sink.writeByte(0); - try self_sink.writeU32(@backingInt(layout_idx)); - }, - .local => |node_id| { - const child_index = @backingInt(node_id); - const engine = self_sink.engine; - if (engine.component[child_index] == self_sink.component_id) { - try self_sink.writeByte(2); - return; - } - std.debug.assert(engine.component[child_index] != no_component); - try self_sink.writeByte(1); - self_sink.hasher.update(&engine.digests[child_index]); - }, - } - } - }; - - /// Group rendering sink: writes a node's encoding with every - /// child outside the component rendered as its settled digest and - /// every child inside it as its canonical position. - const RenderSink = struct { - engine: *Engine, - component_id: u32, - rank_of_member: []const u32, - - fn writeByte(self_sink: RenderSink, value: u8) Allocator.Error!void { - try self_sink.engine.render_buf.append(self_sink.engine.allocator, value); - } - - fn writeU32(self_sink: RenderSink, value: u32) Allocator.Error!void { - const bytes = littleEndianBytes(value); - try self_sink.engine.render_buf.appendSlice(self_sink.engine.allocator, &bytes); - } - - fn child(self_sink: RenderSink, ref: GraphRef) Allocator.Error!void { - switch (ref) { - .canonical => |layout_idx| { - try self_sink.writeByte(0); - try self_sink.writeU32(@backingInt(layout_idx)); - }, - .local => |node_id| { - const child_index = @backingInt(node_id); - const engine = self_sink.engine; - if (engine.component[child_index] == self_sink.component_id) { - try self_sink.writeByte(2); - try self_sink.writeU32(self_sink.rank_of_member[engine.position[child_index]]); - return; - } - std.debug.assert(engine.component[child_index] != no_component); - try self_sink.writeByte(1); - try engine.render_buf.appendSlice(engine.allocator, &engine.digests[child_index]); - }, - } - } - }; - - /// Shape sink: writes a node's non-reference encoding to `content` - /// and classifies each child, naming in-component children by - /// their rank when `rank_of_member` is given and by their node - /// index otherwise. - const ShapeSink = struct { - engine: *Engine, - /// Grows `content` and `children`. - allocator: Allocator, - component_id: u32, - rank_of_member: ?[]const u32, - content: *std.ArrayList(u8), - children: *std.ArrayList(ShapeChild), - - fn writeByte(self_sink: ShapeSink, value: u8) Allocator.Error!void { - try self_sink.content.append(self_sink.allocator, value); - } - - fn writeU32(self_sink: ShapeSink, value: u32) Allocator.Error!void { - const bytes = littleEndianBytes(value); - try self_sink.content.appendSlice(self_sink.allocator, &bytes); - } - - fn child(self_sink: ShapeSink, ref: GraphRef) Allocator.Error!void { - const engine = self_sink.engine; - const shape_child: ShapeChild = switch (ref) { - .canonical => |layout_idx| .{ .canonical = layout_idx }, - .local => |node_id| blk: { - const child_index = @backingInt(node_id); - if (engine.component[child_index] == self_sink.component_id) { - if (self_sink.rank_of_member) |ranks| break :blk .{ .member = ranks[engine.position[child_index]] }; - break :blk .{ .component_node = child_index }; - } - std.debug.assert(engine.component[child_index] != no_component); - break :blk .{ .settled = engine.digests[child_index] }; - }, - }; - try self_sink.children.append(self_sink.allocator, shape_child); - } - }; - - /// A node of Tarjan's walk whose local children are - /// `walk_children[children_start..children_end]`, in encoding - /// order. - const WalkFrame = struct { - node: u32, - children_start: usize, - children_end: usize, - next: usize, - }; + reduction: *recursive_digest.Graph, + reduction_node: []const u32, + hasher: *TypeDigestHasher, - /// Tarjan's walk from `root` on explicit frames, so the depth of - /// the layout graph never becomes call depth. - fn strongConnect(self_engine: *Engine, root: u32) Allocator.Error!void { - const allocator = self_engine.allocator; - var frames = std.ArrayList(WalkFrame).empty; - defer frames.deinit(allocator); - var walk_children = std.ArrayList(u32).empty; - defer walk_children.deinit(allocator); - try self_engine.enterWalkNode(root, &frames, &walk_children); - while (frames.items.len > 0) { - const frame = &frames.items[frames.items.len - 1]; - if (frame.next < frame.children_end) { - const child_index = walk_children.items[frame.next]; - frame.next += 1; - if (self_engine.visit_index[child_index] == unvisited) { - try self_engine.enterWalkNode(child_index, &frames, &walk_children); - } else if (self_engine.on_stack[child_index]) { - self_engine.low_link[frame.node] = @min(self_engine.low_link[frame.node], self_engine.visit_index[child_index]); - } - continue; - } - const finished = frames.pop().?; - walk_children.shrinkRetainingCapacity(finished.children_start); - const node_index = finished.node; - if (self_engine.low_link[node_index] == self_engine.visit_index[node_index]) { - self_engine.members.clearRetainingCapacity(); - while (true) { - const member = self_engine.stack.pop() orelse unreachable; // node_index is still on Tarjan's stack because its low link equals its visit index - self_engine.on_stack[member] = false; - try self_engine.members.append(self_engine.allocator, member); - if (member == node_index) break; - } - try self_engine.resolveComponent(); - } - if (frames.items.len > 0) { - const parent = frames.items[frames.items.len - 1].node; - self_engine.low_link[parent] = @min(self_engine.low_link[parent], self_engine.low_link[node_index]); - } - } + fn writeByte(self_sink: ContentSink, value: u8) Allocator.Error!void { + self_sink.hasher.update(&.{value}); } - fn enterWalkNode( - self_engine: *Engine, - node_index: u32, - frames: *std.ArrayList(WalkFrame), - walk_children: *std.ArrayList(u32), - ) Allocator.Error!void { - self_engine.visit_index[node_index] = self_engine.next_visit; - self_engine.low_link[node_index] = self_engine.next_visit; - self_engine.next_visit += 1; - try self_engine.stack.append(self_engine.allocator, node_index); - self_engine.on_stack[node_index] = true; - - const children_start = walk_children.items.len; - try encodeNode(self_engine.graph, node_index, VisitSink{ .children = walk_children, .allocator = self_engine.allocator }); - try frames.append(self_engine.allocator, .{ - .node = node_index, - .children_start = children_start, - .children_end = walk_children.items.len, - .next = children_start, - }); - } - - /// Settle the digests of the component in `members`, whose every - /// outgoing reference already carries a settled digest. - fn resolveComponent(self_engine: *Engine) Allocator.Error!void { - const members = self_engine.members.items; - const component_id = self_engine.component_count; - self_engine.component_count += 1; - for (members, 0..) |member, pos| { - self_engine.component[member] = component_id; - self_engine.position[member] = @intCast(pos); - } - - self_engine.edges.clearRetainingCapacity(); - self_engine.edge_start.clearRetainingCapacity(); - self_engine.edge_len.clearRetainingCapacity(); - for (members) |member| { - const start: u32 = @intCast(self_engine.edges.items.len); - try encodeNode(self_engine.graph, member, EdgeSink{ .engine = self_engine, .component_id = component_id }); - try self_engine.edge_start.append(self_engine.allocator, start); - try self_engine.edge_len.append(self_engine.allocator, @intCast(self_engine.edges.items.len - start)); - } - - if (members.len == 1 and self_engine.edges.items.len == 0) { - const member = members[0]; - const unfolding = try self_engine.unfoldingKey(member); - if (self_engine.unfoldings.get(unfolding)) |key| { - self_engine.digests[member] = key; - self_engine.keys[member] = key; - } else { - self_engine.digests[member] = unfolding; - } - return; - } - - try self_engine.resolveCyclicComponent(component_id); + fn writeU32(self_sink: ContentSink, value: u32) Allocator.Error!void { + recursive_digest.writeU32(self_sink.hasher, value); } - /// Encode one node using the settled digests of all its children, - /// including children in its own already-resolved component. - fn unfoldingKey(self_engine: *Engine, member: u32) Allocator.Error!RecursiveKey { - var hasher = TypeDigestHasher.init(); - hasher.update(domain); - hasher.update("acyclic"); - try encodeNode(self_engine.graph, member, LabelSink{ - .engine = self_engine, - .hasher = &hasher, - .component_id = no_component, - }); - return hasher.finalResult(); - } - - fn resolveCyclicComponent(self_engine: *Engine, component_id: u32) Allocator.Error!void { - const allocator = self_engine.allocator; - // Temporaries of this component, freed together when it resolves. - var scratch_arena = collections.SingleThreadArena.init(allocator); - defer scratch_arena.deinit(); - const scratch = scratch_arena.allocator(); - const members = self_engine.members.items; - const member_count = members.len; - - if (try self_engine.foldOntoSettledGroup(component_id)) return; - - // Refine to the stable bisimulation partition: every member - // starts from its content label, and each round relabels it - // by its own label followed by the labels reached by every - // ordered in-component reference, until a round stops - // separating members. Labels are pure functions of a - // member's unfolding, so bisimilar members carry equal labels - // and the labels are the same for bisimilar positions of any - // two graphs whatever their node counts or allocation order. - var labels = try scratch.alloc(RecursiveKey, member_count); - var next_labels = try scratch.alloc(RecursiveKey, member_count); - var distinct_labels = std.AutoHashMap(RecursiveKey, u32).init(scratch); - for (members, 0..) |member, pos| { - var hasher = TypeDigestHasher.init(); - try encodeNode(self_engine.graph, member, LabelSink{ - .engine = self_engine, - .hasher = &hasher, - .component_id = component_id, - }); - labels[pos] = hasher.finalResult(); - try distinct_labels.put(labels[pos], no_component); - } - var label_count: u32 = distinct_labels.count(); - while (true) { - distinct_labels.clearRetainingCapacity(); - for (members, 0..) |_, pos| { - var hasher = TypeDigestHasher.init(); - hasher.update(&labels[pos]); - const edges = self_engine.edges.items[self_engine.edge_start.items[pos]..][0..self_engine.edge_len.items[pos]]; - for (edges) |child_index| { - hasher.update(&labels[self_engine.position[child_index]]); - } - next_labels[pos] = hasher.finalResult(); - try distinct_labels.put(next_labels[pos], no_component); - } - const next_count: u32 = distinct_labels.count(); - const stable = next_count == label_count; - const finished_labels = labels; - labels = next_labels; - next_labels = finished_labels; - label_count = next_count; - if (stable) break; - } - - // Canonical order: the reduced positions sorted by label. - const block_count = label_count; - const sorted_labels = try scratch.alloc(RecursiveKey, block_count); - { - var it = distinct_labels.keyIterator(); - var next: usize = 0; - while (it.next()) |label| : (next += 1) { - sorted_labels[next] = label.*; - } - std.debug.assert(next == block_count); - } - std.mem.sort(RecursiveKey, sorted_labels, {}, digestLessThan); - for (sorted_labels, 0..) |label, rank| { - const slot = distinct_labels.getPtr(label) orelse unreachable; // sorted_labels holds exactly distinct_labels' keys - slot.* = @intCast(rank); - } - const rank_of_member = try scratch.alloc(u32, member_count); - const block_rep = try scratch.alloc(u32, block_count); - @memset(block_rep, no_component); - for (members, 0..) |member, pos| { - const rank = distinct_labels.get(labels[pos]) orelse unreachable; // the final refinement round put every member's label - rank_of_member[pos] = rank; - if (block_rep[rank] == no_component) block_rep[rank] = member; - } - - // One group rendering in canonical order; every reduced - // position digests as its rank against the rendering's digest. - self_engine.render_buf.clearRetainingCapacity(); - try self_engine.render_buf.appendSlice(allocator, domain); - try self_engine.render_buf.appendSlice(allocator, "group"); - for (block_rep) |rep| { - try encodeNode(self_engine.graph, rep, RenderSink{ - .engine = self_engine, - .component_id = component_id, - .rank_of_member = rank_of_member, - }); - } - var group_hasher = TypeDigestHasher.init(); - group_hasher.update(self_engine.render_buf.items); - const group_digest = group_hasher.finalResult(); - const block_digest = try scratch.alloc(RecursiveKey, block_count); - for (0..block_count) |rank| { - var hasher = TypeDigestHasher.init(); - hasher.update("recursive-member"); - hashU32(&hasher, @intCast(rank)); - hasher.update(&group_digest); - block_digest[rank] = hasher.finalResult(); - } - for (members, 0..) |member, pos| { - const digest = block_digest[rank_of_member[pos]]; - self_engine.digests[member] = digest; - self_engine.keys[member] = digest; - } - for (members) |member| { - try self_engine.unfoldings.put(self_engine.unfoldings_allocator, try self_engine.unfoldingKey(member), self_engine.digests[member]); - } - - // The store owns the groups past this commit. - const groups = self_engine.groups; - const groups_allocator = self_engine.unfoldings_allocator; - if (groups.member_of.contains(block_digest[0])) return; - const group: u32 = @intCast(groups.groups.items.len); - try groups.groups.append(groups_allocator, .{ - .rank_start = @intCast(groups.ranks.items.len), - .rank_count = block_count, - }); - for (block_rep, 0..) |rep_member, rank| { - const content_start: u32 = @intCast(groups.content.items.len); - const child_start: u32 = @intCast(groups.children.items.len); - try encodeNode(self_engine.graph, rep_member, ShapeSink{ - .engine = self_engine, - .allocator = groups_allocator, - .component_id = component_id, - .rank_of_member = rank_of_member, - .content = &groups.content, - .children = &groups.children, - }); - try groups.ranks.append(groups_allocator, .{ - .digest = block_digest[rank], - .content_start = content_start, - .content_len = @intCast(groups.content.items.len - content_start), - .child_start = child_start, - .child_len = @intCast(groups.children.items.len - child_start), - }); - try groups.member_of.put(groups_allocator, block_digest[rank], .{ .group = group, .rank = @intCast(rank) }); - } - } - - /// Pair the component being resolved coinductively against each - /// settled recursive group it refers into. When every member pairs - /// with a group member whose one-step shape matches it, with - /// in-component children pairing with the shape's in-group - /// children, the component spells that group's unfolding and its - /// members take the paired members' identities. - fn foldOntoSettledGroup(self_engine: *Engine, component_id: u32) Allocator.Error!bool { - const allocator = self_engine.allocator; - const members = self_engine.members.items; - const groups = self_engine.groups; - - var candidate_groups = std.ArrayList(u32).empty; - defer candidate_groups.deinit(allocator); - for (members) |member| { - try self_engine.encodeComponentShape(member, component_id); - for (self_engine.shape_children.items) |shape_child| { - const digest = switch (shape_child) { - .settled => |digest| digest, - .canonical, .member, .component_node => continue, - }; - const member_ref = groups.member_of.get(digest) orelse continue; - const known = for (candidate_groups.items) |group| { - if (group == member_ref.group) break true; - } else false; - if (!known) try candidate_groups.append(allocator, member_ref.group); - } - } - if (candidate_groups.items.len == 0) return false; - - const paired = try allocator.alloc(u32, members.len); - defer allocator.free(paired); - var pending = std.ArrayList(PairedMember).empty; - defer pending.deinit(allocator); - for (candidate_groups.items) |group| { - const rank_count = groups.groups.items[group].rank_count; - for (0..rank_count) |start_rank| { - @memset(paired, no_component); - pending.clearRetainingCapacity(); - try pending.append(allocator, .{ .position = 0, .rank = @intCast(start_rank) }); - if (!try self_engine.pairWithGroup(component_id, group, paired, &pending)) continue; - for (members, paired) |member, rank| { - std.debug.assert(rank != no_component); - const digest = groups.rank(group, rank).digest; - self_engine.digests[member] = digest; - self_engine.keys[member] = digest; - } - return true; - } - } - return false; - } - - const PairedMember = struct { - position: u32, - rank: u32, - }; - - fn pairWithGroup( - self_engine: *Engine, - component_id: u32, - group: u32, - paired: []u32, - pending: *std.ArrayList(PairedMember), - ) Allocator.Error!bool { - const groups = self_engine.groups; - const members = self_engine.members.items; - while (pending.pop()) |pair| { - if (paired[pair.position] != no_component) { - if (paired[pair.position] != pair.rank) return false; - continue; - } - paired[pair.position] = pair.rank; - const shape = groups.rank(group, pair.rank); - try self_engine.encodeComponentShape(members[pair.position], component_id); - const content = groups.content.items[shape.content_start..][0..shape.content_len]; - if (!keyBytesEqual(content, self_engine.shape_content.items)) return false; - const shape_children = groups.children.items[shape.child_start..][0..shape.child_len]; - if (shape_children.len != self_engine.shape_children.items.len) return false; - for (self_engine.shape_children.items, shape_children) |actual, expected| { - switch (actual) { - .canonical => |layout_idx| switch (expected) { - .canonical => |expected_idx| if (expected_idx != layout_idx) return false, - .settled, .member => return false, - .component_node => unreachable, // settled group shapes are encoded with member ranks, so no child of one is a component node - }, - .settled => |digest| switch (expected) { - .settled => |expected_digest| if (!std.meta.eql(expected_digest, digest)) return false, - .member => |expected_rank| if (!std.meta.eql(groups.rank(group, expected_rank).digest, digest)) return false, - .canonical => return false, - .component_node => unreachable, // settled group shapes are encoded with member ranks, so no child of one is a component node - }, - .component_node => |child_index| switch (expected) { - .member => |expected_rank| try pending.append(self_engine.allocator, .{ - .position = self_engine.position[child_index], - .rank = expected_rank, - }), - .canonical, .settled => return false, - .component_node => unreachable, // settled group shapes are encoded with member ranks, so no child of one is a component node - }, - .member => unreachable, // encodeComponentShape passes no member ranks, so no child it encodes is a member - } - } - } - for (paired) |rank| { - if (rank == no_component) return false; - } - return true; - } - - fn encodeComponentShape(self_engine: *Engine, member: u32, component_id: u32) Allocator.Error!void { - self_engine.shape_content.clearRetainingCapacity(); - self_engine.shape_children.clearRetainingCapacity(); - try encodeNode(self_engine.graph, member, ShapeSink{ - .engine = self_engine, - .allocator = self_engine.allocator, - .component_id = component_id, - .rank_of_member = null, - .content = &self_engine.shape_content, - .children = &self_engine.shape_children, + fn child(self_sink: ContentSink, ref: GraphRef) Allocator.Error!void { + self_sink.hasher.update(&.{0xff}); + try self_sink.reduction.putChild(switch (ref) { + .canonical => |layout_idx| .{ .settled = canonicalIdentity(layout_idx) }, + .local => |node_id| switch (self_sink.graph.getNode(node_id)) { + .committed => |committed| .{ .settled = self_sink.graph.committedDigest(committed) }, + .nominal => unreachable, // children resolve through nominals before reaching the sink + .pending, .box, .list, .closure, .erased_callable, .struct_, .tag_union => .{ .node = self_sink.reduction_node[@backingInt(node_id)] }, + }, }); } }; + + /// The identity a reference to an already canonical layout + /// contributes: the layout itself. + fn canonicalIdentity(layout_idx: Idx) RecursiveKey { + var hasher = TypeDigestHasher.init(); + recursive_digest.writeBytes(&hasher, recursive_digest_domain); + recursive_digest.writeBytes(&hasher, "canonical"); + recursive_digest.writeU32(&hasher, @backingInt(layout_idx)); + return hasher.finalResult(); + } }; fn translateGraphRef(mapping: []const GraphRef, ref: GraphRef) GraphRef { @@ -1845,7 +1201,7 @@ pub const Store = struct { } const scratch = scratch_arena.allocator(); - const analysis = try RecursiveGraphAnalysis.init(scratch, self.allocator, graph, &self.recursive_unfoldings, &self.recursive_groups); + const analysis = try RecursiveGraphAnalysis.init(scratch, graph, &self.recursive_digests); const node_count = graph.nodes.items.len; const mapping = try scratch.alloc(GraphRef, node_count); @@ -4318,7 +3674,7 @@ test "commitGraph keeps distinct-field recursive struct payloads apart" { var analysis_arena = collections.SingleThreadArena.init(testing.allocator); defer analysis_arena.deinit(); - const analysis = try Store.RecursiveGraphAnalysis.init(analysis_arena.allocator(), store.allocator, &graph, &store.recursive_unfoldings, &store.recursive_groups); + const analysis = try Store.RecursiveGraphAnalysis.init(analysis_arena.allocator(), &graph, &store.recursive_digests); var seen_keys = std.AutoHashMap(Store.RecursiveGraphAnalysis.RecursiveKey, void).init(testing.allocator); defer seen_keys.deinit(); for (variants) |variant| { @@ -4451,7 +3807,7 @@ test "commitGraph gives an unrolled spelling the layout of the committed union i defer first_commit.deinit(testing.allocator); var first_analysis_arena = collections.SingleThreadArena.init(testing.allocator); defer first_analysis_arena.deinit(); - const first_analysis = try Store.RecursiveGraphAnalysis.init(first_analysis_arena.allocator(), store.allocator, &first, &store.recursive_unfoldings, &store.recursive_groups); + const first_analysis = try Store.RecursiveGraphAnalysis.init(first_analysis_arena.allocator(), &first, &store.recursive_digests); // `A`, the same type spelled with `f0` pointing back to `A` while `f1` // reaches the committed `B`. diff --git a/src/lir/lir_image.zig b/src/lir/lir_image.zig index f4e04312f1a..ae41db6310e 100644 --- a/src/lir/lir_image.zig +++ b/src/lir/lir_image.zig @@ -406,8 +406,7 @@ pub const LayoutStoreImage = extern struct { .scratch_intern_key = .empty, .recursive_layout_keys = try safeListFromRef(layout_mod.Store.RecursiveLayoutKey, base_ptr, image_size, self.recursive_layout_keys), .interned_recursive_graphs = .empty, - .recursive_unfoldings = .empty, - .recursive_groups = .{}, + .recursive_digests = base.recursive_digest.Settled.init(allocator, layout_mod.Store.recursive_digest_domain), .target_usize = target_usize, .digest_cache = try layout_mod.DigestCache.create(allocator), }; @@ -759,8 +758,7 @@ fn serializeSidecarInto( .scratch_intern_key = .empty, .recursive_layout_keys = try cloneSafeList(layout_mod.Store.RecursiveLayoutKey, gpa, lowered.layouts.recursive_layout_keys), .interned_recursive_graphs = .empty, - .recursive_unfoldings = .empty, - .recursive_groups = .{}, + .recursive_digests = base.recursive_digest.Settled.init(gpa, layout_mod.Store.recursive_digest_domain), .target_usize = lowered.layouts.target_usize, // `BoxySidecar.fromStores` reads only the array-backed fields. .digest_cache = undefined, @@ -1017,8 +1015,7 @@ fn serializeCanonicalSidecarInto( // what the code loads. .recursive_layout_keys = .{}, .interned_recursive_graphs = .empty, - .recursive_unfoldings = .empty, - .recursive_groups = .{}, + .recursive_digests = base.recursive_digest.Settled.init(gpa, layout_mod.Store.recursive_digest_domain), .target_usize = source.target_usize, // `BoxySidecar.fromStores` reads only the array-backed fields. .digest_cache = undefined, @@ -1087,7 +1084,7 @@ comptime { // each view starts empty, and its `commit_scratch` and // `scratch_field_offsets` are scratch that views start empty. std.debug.assert(@typeInfo(LirStore).@"struct".field_names.len == 41); - std.debug.assert(@typeInfo(layout_mod.Store).@"struct".field_names.len == 21); + std.debug.assert(@typeInfo(layout_mod.Store).@"struct".field_names.len == 20); std.debug.assert(@typeInfo(base.StringLiteral.Store).@"struct".field_names.len == 1); // The header is authored field by field in image storage and is part of a @@ -2019,8 +2016,7 @@ test "LIR image copies and round-trips every populated store field" { .scratch_intern_key = undefined, .recursive_layout_keys = try h.safeList(layout_mod.Store.RecursiveLayoutKey, source_allocator, 2, 0xa8), .interned_recursive_graphs = undefined, - .recursive_unfoldings = undefined, - .recursive_groups = undefined, + .recursive_digests = undefined, .target_usize = target_usize, .digest_cache = undefined, }; diff --git a/src/postcheck/lambda_mono/type.zig b/src/postcheck/lambda_mono/type.zig index 35635f92668..b674d6081af 100644 --- a/src/postcheck/lambda_mono/type.zig +++ b/src/postcheck/lambda_mono/type.zig @@ -5,7 +5,7 @@ //! callable layout selected by direct LIR lowering. const std = @import("std"); -const DigestGraph = @import("../type_digest.zig").Graph; +const recursive_digest = @import("base").recursive_digest; const TypeDigestHasher = @import("base").TypeDigestHasher; const check = @import("check"); const collections = @import("collections"); @@ -537,7 +537,7 @@ pub const Store = struct { .zst => writeBytes(hasher, "zst"), .hole => writeBytes(hasher, "hole"), } - scratch.graph.setScalar(scratch.node_by_type.get(ty).?, hasher.finalResult()); + scratch.graph.endNode(scratch.node_by_type.get(ty).?, hasher.finalResult()); } fn writeTypeSpanDigest( @@ -559,22 +559,30 @@ pub const Store = struct { /// resolved identity survives a request because Lambda Mono types are mutable. const DigestScratch = struct { allocator: std.mem.Allocator, - graph: DigestGraph, + graph: recursive_digest.Graph, + settled: recursive_digest.Settled, node_by_type: collections.DenseMap(TypeId, u32), pending: std.ArrayList(TypeId) = .empty, fn init(allocator: std.mem.Allocator) DigestScratch { - return .{ .allocator = allocator, .graph = DigestGraph.init(allocator), .node_by_type = collections.DenseMap(TypeId, u32).init(allocator) }; + return .{ + .allocator = allocator, + .graph = recursive_digest.Graph.init(allocator), + .settled = recursive_digest.Settled.init(allocator, "roc.lambda-mono.type.v3"), + .node_by_type = collections.DenseMap(TypeId, u32).init(allocator), + }; } fn deinit(self: *DigestScratch) void { self.graph.deinit(); + self.settled.deinit(); self.node_by_type.deinit(); self.pending.deinit(self.allocator); } fn reset(self: *DigestScratch) void { self.graph.reset(); + self.settled.clear(); self.node_by_type.clearRetainingCapacity(); self.pending.clearRetainingCapacity(); } @@ -592,7 +600,7 @@ const DigestScratch = struct { // Mark the position of every edge among the scalar data, while the // graph separately retains the exact ordered child identities. writeBytes(hasher, "child"); - try self.graph.putChild(try self.discover(ty)); + try self.graph.putChild(.{ .node = try self.discover(ty) }); } fn run(self: *DigestScratch, store: *const Store, name_store: *const names.NameStore, ty: TypeId, targets: ?Store.CallableTargets) std.mem.Allocator.Error!names.TypeDigest { @@ -602,9 +610,9 @@ const DigestScratch = struct { const node = self.node_by_type.get(pending_ty).?; self.graph.beginNode(node); try store.encodeDigestNode(name_store, self, targets, pending_ty); - self.graph.endNode(node); } - return .{ .bytes = try self.graph.resolve(root) }; + try self.graph.resolve(&self.settled, null); + return .{ .bytes = self.graph.digest(root) }; } }; diff --git a/src/postcheck/monotype/type.zig b/src/postcheck/monotype/type.zig index bb794df1e58..d2d48ba693b 100644 --- a/src/postcheck/monotype/type.zig +++ b/src/postcheck/monotype/type.zig @@ -5,6 +5,7 @@ const std = @import("std"); const TypeDigestHasher = @import("base").TypeDigestHasher; +const recursive_digest = @import("base").recursive_digest; const builtin = @import("builtin"); const check = @import("check"); const collections = @import("collections"); @@ -396,27 +397,10 @@ pub const Store = struct { iterator_interface_visited: std.ArrayList(TypeId), iterator_interface_visit_epochs: StoreList(u32, "iterator_interface_visit_epochs"), iterator_interface_visit_epoch: u32, - /// One-step unfoldings of every digested recursive-group position: the - /// hash of a member's node encoding with all children rendered as their - /// finalized digests, mapped to that member's group digest. An acyclic - /// node whose rendering matches an entry is a rolled-out prefix of the - /// same infinite type and adopts the member's digest, which keeps digests - /// independent of how deeply a recursive knot is tied—including across - /// separate digest calls. Keys and values are content-addressed, so - /// entries stay valid across `restore` truncations and need no rollback. - recursive_digest_unfoldings: std.AutoHashMap([32]u8, names.TypeDigest), - /// Every reduced recursive group by its shape: the hash of its members' - /// content labels in discovery order and each member's ordered in-group - /// references by position. A group's reduction is a function of exactly - /// that shape, so a later group of the same shape takes each position's - /// digest from here. Its unfoldings were indexed when the entry was - /// recorded. Keys and values are content-addressed, so entries stay valid - /// across `restore` truncations and need no rollback. - recursive_scc_shapes: std.AutoHashMap([32]u8, RecursiveSccShape), - /// Each recorded shape's per-position rank into its block digests. - recursive_scc_ranks: std.ArrayList(u32), - /// Each recorded shape's block digests, in rank order. - recursive_scc_digests: std.ArrayList(names.TypeDigest), + /// Every recursive group the digests have settled (`base.recursive_digest`), + /// shared by every digest mode. It is content addressed, so it stays valid + /// across `restore` truncations and needs no rollback. + recursive_digests: recursive_digest.Settled, /// Full-content identity for store-level acyclic interning: full digest /// bytes to the one committed representative with that digest. The full /// digest encodes everything `typeEql` compares, so a digest names one @@ -482,10 +466,7 @@ pub const Store = struct { .iterator_interface_visited = .empty, .iterator_interface_visit_epochs = .empty, .iterator_interface_visit_epoch = 0, - .recursive_digest_unfoldings = std.AutoHashMap([32]u8, names.TypeDigest).init(allocator), - .recursive_scc_shapes = std.AutoHashMap([32]u8, RecursiveSccShape).init(allocator), - .recursive_scc_ranks = .empty, - .recursive_scc_digests = .empty, + .recursive_digests = recursive_digest.Settled.init(allocator, "roc.monotype.type.v2"), .full_digest_interned = std.AutoHashMap(DigestBucketKey, TypeId).init(allocator), .spans = .empty, .fields = .empty, @@ -510,10 +491,8 @@ pub const Store = struct { const visit_epochs = try allocator.alloc(u32, result.types.len()); @memset(visit_epochs, 0); result.iterator_interface_visit_epochs = @TypeOf(result.iterator_interface_visit_epochs).fromOwnedSlice(visit_epochs); - var unfoldings = self.recursive_digest_unfoldings.iterator(); - while (unfoldings.next()) |entry| { - try result.recursive_digest_unfoldings.put(entry.key_ptr.*, entry.value_ptr.*); - } + result.recursive_digests.deinit(); + result.recursive_digests = try self.recursive_digests.clone(allocator); try result.full_digest_interned.ensureTotalCapacity(self.full_digest_interned.count()); var interned = self.full_digest_interned.iterator(); while (interned.next()) |entry| { @@ -591,10 +570,7 @@ pub const Store = struct { self.fields.deinit(self.allocator); self.spans.deinit(self.allocator); self.full_digest_interned.deinit(); - self.recursive_digest_unfoldings.deinit(); - self.recursive_scc_shapes.deinit(); - self.recursive_scc_ranks.deinit(self.allocator); - self.recursive_scc_digests.deinit(self.allocator); + self.recursive_digests.deinit(); self.iterator_interface_visit_epochs.deinit(self.allocator); self.iterator_interface_visited.deinit(self.allocator); self.iterator_interface_pending.deinit(self.allocator); @@ -2259,14 +2235,6 @@ pub const Store = struct { return self.equalityDigestCached(name_store, ty, null); } - /// One recorded recursive-group shape, as spans of the store's rank and - /// block-digest pools. - const RecursiveSccShape = struct { - ranks_start: u32, - digests_start: u32, - block_count: u32, - }; - pub const DigestStats = struct { root_requests: u64 = 0, commit_root_requests: u64 = 0, @@ -3253,38 +3221,21 @@ pub const Store = struct { link_len: u32 = 0, scalar_start: usize = 0, scalar_len: usize = 0, - digest: names.TypeDigest = undefined, - resolved: bool = false, }; /// A child reference during digesting: either an already-finalized digest - /// or a node of the currently discovered graph. Whether a child arrived as - /// a cache hit or as a discovered-then-resolved node must never influence - /// digest bytes or partitioning, so every consumer of a `ChildLink` treats - /// a resolved node and an external digest identically. + /// or a node of the currently discovered graph. const ChildLink = union(enum) { external: names.TypeDigest, node: u32, }; - /// Graph-reducing digest computation for one requested type. + /// Digest computation for one requested type. /// /// Discovery walks the uncached reachable subgraph iteratively, pruning - /// at children whose digest for the required mode is already cached. - /// Iterative Tarjan SCC discovery then resolves the condensation in - /// reverse topological order: an acyclic node hashes its encoding with - /// finalized child digests, and each cyclic SCC is reduced by - /// bisimulation refinement over content labels, rendered once with its - /// reduced positions in label order and group-relative back-references, - /// and digested per reduced position. Cyclic member digests and their - /// one-step unfolding index are committed together so OOM retries retain - /// correct folding of later rolled-out prefixes of the same group. - /// - /// Known conservative incompleteness: per-SCC reduction cannot identify - /// an in-SCC position with an equivalent position outside its own SCC, so - /// a knot entangled with a separately tied equivalent knot digests apart - /// from it (see the "entangled equivalent knots" test). Equal digests - /// still always imply structural equality; the miss only costs reuse. + /// at children whose digest for the required mode is already cached, and + /// the shared reduction (`base.recursive_digest`) settles every + /// discovered node against the store's settled recursive groups. const DigestEngine = struct { store: *Store, name_store: *const names.NameStore, @@ -3295,54 +3246,19 @@ pub const Store = struct { link_pool: std.ArrayList(ChildLink), child_offsets: std.ArrayList(usize), scalar_bytes: std.ArrayList(u8), - render_buf: std.ArrayList(u8), - scc: SccScratch, + graph: recursive_digest.Graph, /// Whether this engine runs on the store's kept lists and returns /// them; an engine started while another is running owns its own. uses_store_storage: bool, - /// Tarjan state of one SCC pass, kept between digest requests. - const SccScratch = struct { - visit_index: std.ArrayList(u32) = .empty, - low_link: std.ArrayList(u32) = .empty, - on_stack: std.ArrayList(bool) = .empty, - stack: std.ArrayList(u32) = .empty, - frames: std.ArrayList(SccFrame) = .empty, - members: std.ArrayList(u32) = .empty, - - fn clear(self: *SccScratch) void { - self.visit_index.clearRetainingCapacity(); - self.low_link.clearRetainingCapacity(); - self.on_stack.clearRetainingCapacity(); - self.stack.clearRetainingCapacity(); - self.frames.clearRetainingCapacity(); - self.members.clearRetainingCapacity(); - } - - fn deinit(self: *SccScratch, allocator: std.mem.Allocator) void { - self.visit_index.deinit(allocator); - self.low_link.deinit(allocator); - self.on_stack.deinit(allocator); - self.stack.deinit(allocator); - self.frames.deinit(allocator); - self.members.deinit(allocator); - } - }; - - const SccFrame = struct { node: u32, edge_cursor: u32 }; - - const no_scc_position = std.math.maxInt(u32); - const unvisited = std.math.maxInt(u32); - const Storage = struct { nodes: std.ArrayList(DigestNode) = .empty, node_lookup: ?std.AutoHashMap(u64, u32) = null, link_pool: std.ArrayList(ChildLink) = .empty, child_offsets: std.ArrayList(usize) = .empty, scalar_bytes: std.ArrayList(u8) = .empty, - render_buf: std.ArrayList(u8) = .empty, - scc: SccScratch = .{}, + graph: ?recursive_digest.Graph = null, in_use: bool = false, const retained_lookup_capacity = 64; @@ -3353,8 +3269,7 @@ pub const Store = struct { self.link_pool.deinit(allocator); self.child_offsets.deinit(allocator); self.scalar_bytes.deinit(allocator); - self.render_buf.deinit(allocator); - self.scc.deinit(allocator); + if (self.graph) |*graph| graph.deinit(); self.* = .{}; } }; @@ -3371,8 +3286,7 @@ pub const Store = struct { .link_pool = .empty, .child_offsets = .empty, .scalar_bytes = .empty, - .render_buf = .empty, - .scc = .{}, + .graph = recursive_digest.Graph.init(store.allocator), .uses_store_storage = false, }; kept.in_use = true; @@ -3386,8 +3300,7 @@ pub const Store = struct { .link_pool = kept.link_pool, .child_offsets = kept.child_offsets, .scalar_bytes = kept.scalar_bytes, - .render_buf = kept.render_buf, - .scc = kept.scc, + .graph = kept.graph orelse recursive_digest.Graph.init(store.allocator), .uses_store_storage = true, }; kept.* = .{ .in_use = true }; @@ -3396,16 +3309,14 @@ pub const Store = struct { fn deinit(self: *DigestEngine) void { if (!self.uses_store_storage) { - self.render_buf.deinit(self.gpa); self.link_pool.deinit(self.gpa); self.child_offsets.deinit(self.gpa); self.scalar_bytes.deinit(self.gpa); self.node_lookup.deinit(); self.nodes.deinit(self.gpa); - self.scc.deinit(self.gpa); + self.graph.deinit(); return; } - self.scc.clear(); self.nodes.clearRetainingCapacity(); // Clearing a map touches its whole capacity, so one that a large // type grew is released instead of being cleared before every @@ -3418,15 +3329,14 @@ pub const Store = struct { self.link_pool.clearRetainingCapacity(); self.child_offsets.clearRetainingCapacity(); self.scalar_bytes.clearRetainingCapacity(); - self.render_buf.clearRetainingCapacity(); + self.graph.reset(); self.store.digest_storage = .{ .nodes = self.nodes, .node_lookup = self.node_lookup, .link_pool = self.link_pool, .child_offsets = self.child_offsets, .scalar_bytes = self.scalar_bytes, - .render_buf = self.render_buf, - .scc = self.scc, + .graph = self.graph, }; } @@ -3437,10 +3347,18 @@ pub const Store = struct { while (next < self.nodes.items.len) : (next += 1) { try self.collectNode(@intCast(next)); } - try self.resolveSccs(); - const root_node = self.nodes.items[root]; - std.debug.assert(root_node.resolved); - return root_node.digest; + for (0..self.nodes.items.len) |index| { + const graph_node = try self.graph.addNode(); + std.debug.assert(graph_node == index); + try self.encodeGraphNode(graph_node); + } + var reduction: recursive_digest.Stats = .{}; + try self.graph.resolve(&self.store.recursive_digests, &reduction); + if (self.stats) |s| s.group_encodings += reduction.group_encodings; + for (self.nodes.items, 0..) |node, index| { + try self.store.setCachedDigest(node.ty, node.mode, .{ .bytes = self.graph.digest(@intCast(index)) }); + } + return .{ .bytes = self.graph.digest(root) }; } fn nodeKey(ty: TypeId, mode: NamedDigestMode) u64 { @@ -3485,23 +3403,27 @@ pub const Store = struct { node.scalar_len = self.scalar_bytes.items.len - scalar_start; } - fn linksOf(self: *const DigestEngine, index: u32) []const ChildLink { + /// Hand one discovered node to the reduction: its scalar encoding + /// with a slot at every child's recorded offset, and its ordered + /// children. + fn encodeGraphNode(self: *DigestEngine, index: u32) std.mem.Allocator.Error!void { const node = self.nodes.items[index]; - return self.link_pool.items[node.link_start..][0..node.link_len]; - } - - /// Replay the scalar encoding produced by discovery, inserting each - /// child's finalized digest or SCC reference at its recorded offset. - fn renderNode(self: *DigestEngine, index: u32, sink: anytype) std.mem.Allocator.Error!void { - const node = self.nodes.items[index]; - var cursor: usize = node.scalar_start; + const links = self.link_pool.items[node.link_start..][0..node.link_len]; const offsets = self.child_offsets.items[node.link_start..][0..node.link_len]; - for (self.linksOf(index), offsets) |ref, offset| { - try sink.writeRawBytes(self.scalar_bytes.items[cursor..offset]); - try sink.childReference(ref); + self.graph.beginNode(index); + var hasher = TypeDigestHasher.init(); + var cursor: usize = node.scalar_start; + for (links, offsets) |ref, offset| { + hashBytes(&hasher, self.scalar_bytes.items[cursor..offset]); + hashBytes(&hasher, "child"); + try self.graph.putChild(switch (ref) { + .external => |digest| .{ .settled = digest.bytes }, + .node => |target| .{ .node = target }, + }); cursor = offset; } - try sink.writeRawBytes(self.scalar_bytes.items[cursor .. node.scalar_start + node.scalar_len]); + hashBytes(&hasher, self.scalar_bytes.items[cursor .. node.scalar_start + node.scalar_len]); + self.graph.endNode(index, hasher.finalResult()); } /// Discovery is the only store walk. Scalar bytes use the versioned @@ -3526,415 +3448,6 @@ pub const Store = struct { try self.engine.child_offsets.append(self.engine.gpa, self.engine.scalar_bytes.items.len); } }; - - /// Initial-partition sink for one cyclic SCC: hashes the member's - /// full encoding with every out-of-SCC child rendered as its - /// finalized digest and every in-SCC child as a bare positional - /// marker. In-SCC identities are refined afterwards; folding them - /// into this label would bake discovery order into the partition. - const SccLabelSink = struct { - engine: *DigestEngine, - hasher: *TypeDigestHasher, - member_pos_of_node: []const u32, - - fn writeRawBytes(self: SccLabelSink, bytes: []const u8) std.mem.Allocator.Error!void { - self.hasher.update(bytes); - } - - fn childReference(self: SccLabelSink, ref: ChildLink) std.mem.Allocator.Error!void { - switch (ref) { - .external => |digest| { - hashBytes(self.hasher, "type-digest"); - self.hasher.update(&digest.bytes); - }, - .node => |node_index| { - if (self.member_pos_of_node[node_index] != no_scc_position) { - hashBytes(self.hasher, "in-scc"); - return; - } - const node = self.engine.nodes.items[node_index]; - std.debug.assert(node.resolved); - hashBytes(self.hasher, "type-digest"); - self.hasher.update(&node.digest.bytes); - }, - } - } - }; - - /// Group-relative reference context while rendering one cyclic SCC. - const SccRenderContext = struct { - /// Member position within the SCC per engine node, or - /// `no_scc_position` for nodes outside the SCC. - member_pos_of_node: []const u32, - /// Label-ordered reduced position per SCC member. - rank_of_member: []const u32, - }; - - /// Rendering sink: writes the encoding into a byte buffer, resolving - /// children to their finalized digests, or to group-relative - /// back-references inside a cyclic SCC. - const RenderSink = struct { - engine: *DigestEngine, - out: *std.ArrayList(u8), - scc: ?*const SccRenderContext, - - fn writeRawBytes(self: RenderSink, bytes: []const u8) std.mem.Allocator.Error!void { - try self.out.appendSlice(self.engine.gpa, bytes); - } - - fn childReference(self: RenderSink, ref: ChildLink) std.mem.Allocator.Error!void { - switch (ref) { - .external => |digest| try self.emitDigest(digest), - .node => |node_index| { - if (self.scc) |ctx| { - const member_pos = ctx.member_pos_of_node[node_index]; - if (member_pos != no_scc_position) { - try renderBytes(self.engine.gpa, self.out, "group-ref"); - try renderU32(self.engine.gpa, self.out, ctx.rank_of_member[member_pos]); - return; - } - } - const node = self.engine.nodes.items[node_index]; - std.debug.assert(node.resolved); - try self.emitDigest(node.digest); - }, - } - } - - fn emitDigest(self: RenderSink, digest: names.TypeDigest) std.mem.Allocator.Error!void { - try renderBytes(self.engine.gpa, self.out, "type-digest"); - try renderBytes(self.engine.gpa, self.out, &digest.bytes); - } - }; - - /// Iterative Tarjan SCC discovery. SCCs pop in reverse topological - /// order of the condensation, so every edge leaving a popped SCC - /// already points at finalized digests and it can resolve on the spot. - fn resolveSccs(self: *DigestEngine) std.mem.Allocator.Error!void { - const node_count = self.nodes.items.len; - const scratch = &self.scc; - scratch.clear(); - try scratch.visit_index.resize(self.gpa, node_count); - const visit_index = scratch.visit_index.items; - @memset(visit_index, unvisited); - try scratch.low_link.resize(self.gpa, node_count); - const low_link = scratch.low_link.items; - try scratch.on_stack.resize(self.gpa, node_count); - const on_stack = scratch.on_stack.items; - @memset(on_stack, false); - - const scc_stack = &scratch.stack; - const frames = &scratch.frames; - const scc_members = &scratch.members; - - var next_visit: u32 = 0; - for (0..node_count) |start| { - if (visit_index[start] != unvisited) continue; - visit_index[start] = next_visit; - low_link[start] = next_visit; - next_visit += 1; - try scc_stack.append(self.gpa, @intCast(start)); - on_stack[start] = true; - try frames.append(self.gpa, .{ .node = @intCast(start), .edge_cursor = 0 }); - - while (frames.items.len > 0) { - const frame = &frames.items[frames.items.len - 1]; - const v = frame.node; - const refs = self.linksOf(v); - if (frame.edge_cursor < refs.len) { - const ref = refs[frame.edge_cursor]; - frame.edge_cursor += 1; - const w = switch (ref) { - .external => continue, - .node => |node_index| node_index, - }; - if (visit_index[w] == unvisited) { - visit_index[w] = next_visit; - low_link[w] = next_visit; - next_visit += 1; - try scc_stack.append(self.gpa, w); - on_stack[w] = true; - try frames.append(self.gpa, .{ .node = w, .edge_cursor = 0 }); - } else if (on_stack[w]) { - low_link[v] = @min(low_link[v], visit_index[w]); - } - continue; - } - _ = frames.pop(); - if (frames.items.len > 0) { - const parent = frames.items[frames.items.len - 1].node; - low_link[parent] = @min(low_link[parent], low_link[v]); - } - if (low_link[v] != visit_index[v]) continue; - scc_members.clearRetainingCapacity(); - while (true) { - const w = scc_stack.pop() orelse - Common.invariant("Monotype digest SCC stack underflowed"); - on_stack[w] = false; - try scc_members.append(self.gpa, w); - if (w == v) break; - } - try self.resolveScc(scc_members.items); - } - } - } - - fn resolveScc(self: *DigestEngine, members: []const u32) std.mem.Allocator.Error!void { - if (members.len == 1 and !self.hasSelfEdge(members[0])) { - try self.resolveAcyclicNode(members[0]); - return; - } - try self.resolveCyclicScc(members); - } - - fn hasSelfEdge(self: *const DigestEngine, index: u32) bool { - for (self.linksOf(index)) |ref| { - switch (ref) { - .external => {}, - .node => |target| if (target == index) return true, - } - } - return false; - } - - fn resolveAcyclicNode(self: *DigestEngine, index: u32) std.mem.Allocator.Error!void { - self.render_buf.clearRetainingCapacity(); - const sink = RenderSink{ .engine = self, .out = &self.render_buf, .scc = null }; - try self.renderNode(index, sink); - var digest: names.TypeDigest = .{ .bytes = typeHashOf(self.render_buf.items) }; - // An acyclic store node whose one-step rendering matches a known - // recursive-group unfolding is a rolled-out prefix of the same - // infinite type: same label, and children digest-equal to the - // member's children. It adopts the member's digest so knot depth - // cannot influence identity. - if (self.store.recursive_digest_unfoldings.get(digest.bytes)) |member_digest| { - digest = member_digest; - } - try self.finalizeNode(index, digest); - } - - fn finalizeNode(self: *DigestEngine, index: u32, digest: names.TypeDigest) std.mem.Allocator.Error!void { - const node = &self.nodes.items[index]; - std.debug.assert(!node.resolved); - node.digest = digest; - node.resolved = true; - try self.store.setCachedDigest(node.ty, node.mode, digest); - } - - /// Reduce one cyclic SCC by bisimulation refinement over content - /// labels, order the reduced positions by their final labels, and - /// digest every member as its position in that one group rendering. - fn resolveCyclicScc(self: *DigestEngine, members: []const u32) std.mem.Allocator.Error!void { - const member_count = members.len; - const member_pos_of_node = try self.gpa.alloc(u32, self.nodes.items.len); - defer self.gpa.free(member_pos_of_node); - @memset(member_pos_of_node, no_scc_position); - for (members, 0..) |node_index, pos| { - member_pos_of_node[node_index] = @intCast(pos); - } - - // Refine to the stable bisimulation partition. Every member - // starts from its content label: its full encoding with every - // out-of-SCC child rendered as a finalized digest—computed now, - // not at discovery, so whether a child arrived cached or was - // resolved in this run cannot split bisimilar members—and every - // in-SCC child as a bare positional marker. Each round then - // relabels a member by its own label followed by the labels - // reached by every ordered in-SCC reference, until a round stops - // separating members. A label is a pure function of the - // member's unfolding to the current depth, so two members carry - // equal labels exactly when they are bisimilar, and the labels - // themselves are identical for bisimilar positions of any two - // knots regardless of how many store nodes either knot uses or - // in what order those nodes were allocated. - var labels = try self.gpa.alloc([32]u8, member_count); - defer self.gpa.free(labels); - var next_labels = try self.gpa.alloc([32]u8, member_count); - defer self.gpa.free(next_labels); - var distinct_labels = std.AutoHashMap([32]u8, u32).init(self.gpa); - defer distinct_labels.deinit(); - for (members, 0..) |node_index, pos| { - var hasher = TypeDigestHasher.init(); - const sink = SccLabelSink{ - .engine = self, - .hasher = &hasher, - .member_pos_of_node = member_pos_of_node, - }; - try self.renderNode(node_index, sink); - if (self.stats) |s| s.group_encodings += 1; - labels[pos] = hasher.finalResult(); - try distinct_labels.put(labels[pos], no_scc_position); - } - - // Labels and positional in-group references determine the whole - // reduction below, so a group of a recorded shape is already - // reduced. - const shape = shape: { - var hasher = TypeDigestHasher.init(); - hashBytes(&hasher, "recursive-scc-shape"); - hashU32(&hasher, @intCast(member_count)); - for (members, 0..) |node_index, pos| { - hasher.update(&labels[pos]); - for (self.linksOf(node_index)) |ref| { - const target = switch (ref) { - .external => continue, - .node => |node_target| node_target, - }; - const target_pos = member_pos_of_node[target]; - if (target_pos == no_scc_position) continue; - hashU32(&hasher, target_pos); - } - hashBytes(&hasher, "end-member"); - } - break :shape hasher.finalResult(); - }; - if (self.store.recursive_scc_shapes.get(shape)) |recorded| { - const ranks = self.store.recursive_scc_ranks.items[recorded.ranks_start..][0..member_count]; - const digests = self.store.recursive_scc_digests.items[recorded.digests_start..][0..recorded.block_count]; - for (members, ranks) |node_index, rank| { - const node = &self.nodes.items[node_index]; - std.debug.assert(!node.resolved); - node.digest = digests[rank]; - node.resolved = true; - try self.store.setCachedDigest(node.ty, node.mode, node.digest); - } - return; - } - - var label_count: u32 = distinct_labels.count(); - while (true) { - distinct_labels.clearRetainingCapacity(); - for (members, 0..) |node_index, pos| { - var hasher = TypeDigestHasher.init(); - hasher.update(&labels[pos]); - for (self.linksOf(node_index)) |ref| { - const target = switch (ref) { - .external => continue, - .node => |node_target| node_target, - }; - const target_pos = member_pos_of_node[target]; - if (target_pos == no_scc_position) continue; - hasher.update(&labels[target_pos]); - } - next_labels[pos] = hasher.finalResult(); - try distinct_labels.put(next_labels[pos], no_scc_position); - } - const next_count: u32 = distinct_labels.count(); - const stable = next_count == label_count; - std.mem.swap([][32]u8, &labels, &next_labels); - label_count = next_count; - if (stable) break; - } - - // Group order: the reduced positions sorted by label. Labels are - // intrinsic to the infinite type, so the order is too, and no two - // reduced positions share a label. - const block_count = label_count; - const sorted_labels = try self.gpa.alloc([32]u8, block_count); - defer self.gpa.free(sorted_labels); - { - var it = distinct_labels.keyIterator(); - var next: usize = 0; - while (it.next()) |label| : (next += 1) { - sorted_labels[next] = label.*; - } - std.debug.assert(next == block_count); - } - std.mem.sort([32]u8, sorted_labels, {}, struct { - fn lessThan(_: void, lhs: [32]u8, rhs: [32]u8) bool { - return std.mem.order(u8, &lhs, &rhs) == .lt; - } - }.lessThan); - for (sorted_labels, 0..) |label, rank| { - const slot = distinct_labels.getPtr(label) orelse unreachable; // sorted_labels holds exactly the keys of distinct_labels - slot.* = @intCast(rank); - } - const rank_of_member = try self.gpa.alloc(u32, member_count); - defer self.gpa.free(rank_of_member); - const block_rep = try self.gpa.alloc(u32, block_count); - defer self.gpa.free(block_rep); - @memset(block_rep, no_scc_position); - for (members, 0..) |node_index, pos| { - const rank = distinct_labels.get(labels[pos]) orelse unreachable; // the final refinement round put every labels[pos] into distinct_labels - rank_of_member[pos] = rank; - // The partition is stable, so any member represents its - // position. - if (block_rep[rank] == no_scc_position) block_rep[rank] = node_index; - } - - // One group rendering: every reduced position in label order with - // in-SCC references written as ranks. Every position then - // digests as its rank against the rendering's digest. - self.render_buf.clearRetainingCapacity(); - { - const ctx = SccRenderContext{ - .member_pos_of_node = member_pos_of_node, - .rank_of_member = rank_of_member, - }; - const sink = RenderSink{ .engine = self, .out = &self.render_buf, .scc = &ctx }; - for (block_rep) |rep_index| { - try self.renderNode(rep_index, sink); - if (self.stats) |s| s.group_encodings += 1; - } - } - const group_digest = typeHashOf(self.render_buf.items); - const block_digest = try self.gpa.alloc(names.TypeDigest, block_count); - defer self.gpa.free(block_digest); - for (0..block_count) |rank| { - var hasher = TypeDigestHasher.init(); - hashBytes(&hasher, "recursive-member"); - hashU32(&hasher, @intCast(rank)); - hasher.update(&group_digest); - block_digest[rank] = .{ .bytes = hasher.finalResult() }; - } - for (members, 0..) |node_index, pos| { - const node = &self.nodes.items[node_index]; - std.debug.assert(!node.resolved); - node.digest = block_digest[rank_of_member[pos]]; - node.resolved = true; - } - - // Stage unfoldings using only engine-local member digests. The - // store must commit the whole SCC and its unfolding index - // together: a cached member skips SCC discovery on an OOM retry. - const unfoldings = try self.gpa.alloc([32]u8, block_count); - defer self.gpa.free(unfoldings); - for (block_rep, 0..) |rep_index, rank| { - self.render_buf.clearRetainingCapacity(); - const sink = RenderSink{ .engine = self, .out = &self.render_buf, .scc = null }; - try self.renderNode(rep_index, sink); - if (self.stats) |s| s.group_encodings += 1; - unfoldings[rank] = typeHashOf(self.render_buf.items); - } - try self.store.recursive_digest_unfoldings.ensureUnusedCapacity(block_count); - try self.store.recursive_scc_shapes.ensureUnusedCapacity(1); - try self.store.recursive_scc_ranks.ensureUnusedCapacity(self.store.allocator, member_count); - try self.store.recursive_scc_digests.ensureUnusedCapacity(self.store.allocator, block_count); - - // No fallible work remains; completed earlier SCCs remain reusable - // if any subsequent SCC fails. - for (unfoldings, 0..) |unfolding, rank| { - const gop = self.store.recursive_digest_unfoldings.getOrPutAssumeCapacity(unfolding); - if (gop.found_existing) { - // Reduced-group digests are intrinsic, so an equivalent - // group digested earlier must agree. - std.debug.assert(std.mem.eql(u8, &gop.value_ptr.bytes, &block_digest[rank].bytes)); - } else { - gop.value_ptr.* = block_digest[rank]; - } - } - self.store.recursive_scc_shapes.putAssumeCapacityNoClobber(shape, .{ - .ranks_start = @intCast(self.store.recursive_scc_ranks.items.len), - .digests_start = @intCast(self.store.recursive_scc_digests.items.len), - .block_count = block_count, - }); - self.store.recursive_scc_ranks.appendSliceAssumeCapacity(rank_of_member); - self.store.recursive_scc_digests.appendSliceAssumeCapacity(block_digest); - for (members) |node_index| { - const node = self.nodes.items[node_index]; - try self.store.setCachedDigest(node.ty, node.mode, node.digest); - } - } }; }; @@ -4257,11 +3770,10 @@ fn writeU32(hasher: *TypeDigestHasher, value: u32) void { hasher.update(std.mem.asBytes(&little)); } -// Digest sinks define `writeBytes`/`writeU32` methods, which shadow the -// file-scope hasher helpers inside their bodies; these aliases keep the -// helpers reachable there. +// Digest sinks define a `writeBytes` method, which shadows the file-scope +// hasher helper inside their bodies; this alias keeps the helper reachable +// there. const hashBytes = writeBytes; -const hashU32 = writeU32; fn sliceOverlaps(comptime T: type, a: []const T, b: []const T) bool { const a_start = @intFromPtr(a.ptr); @@ -4285,12 +3797,6 @@ fn decodeNode(node: StoredNode, named_nodes: []const NamedContent, erased_digest }; } -fn typeHashOf(bytes: []const u8) [32]u8 { - var hasher = TypeDigestHasher.init(); - hasher.update(bytes); - return hasher.finalResult(); -} - fn renderU32(gpa: std.mem.Allocator, out: *std.ArrayList(u8), value: u32) std.mem.Allocator.Error!void { const little = std.mem.nativeToLittle(u32, value); try out.appendSlice(gpa, std.mem.asBytes(&little)); @@ -6544,40 +6050,31 @@ test "monotype recursive group digest work is linear in the group's distinct mem } } -test "monotype digest keeps entangled equivalent knots conservatively distinct" { +test "monotype digest identifies a knot entangled with a separately tied copy" { // b1/b2 tie a knot that also points into c, a separately tied knot of the - // same infinite type. Per-SCC reduction cannot identify an in-SCC - // position with an equivalent position outside its own SCC, so the - // members digest apart from c even though all three are structurally - // equal. This pins a KNOWN conservative incompleteness: equal digests - // still imply structural equality (the direction interning relies on), - // and a future whole-graph reduction may intentionally flip the - // inequality assertions below. + // same infinite type. All three digest alike, whichever is digested first. var name_store = names.NameStore.init(std.testing.allocator); defer name_store.deinit(); - var store = Store.init(std.testing.allocator); - defer store.deinit(); + for ([_]bool{ false, true }) |copy_first| { + var store = Store.init(std.testing.allocator); + defer store.deinit(); + + const c = try store.reserveSlot(); + store.fillReservedSlot(c, .{ .tuple = try store.addSpan(&.{ c, c }) }); + const b1 = try store.reserveSlot(); + const b2 = try store.reserveSlot(); + store.fillReservedSlot(b1, .{ .tuple = try store.addSpan(&.{ b2, b1 }) }); + store.fillReservedSlot(b2, .{ .tuple = try store.addSpan(&.{ b1, c }) }); - const c = try store.reserveSlot(); - store.fillReservedSlot(c, .{ .tuple = try store.addSpan(&.{ c, c }) }); - const b1 = try store.reserveSlot(); - const b2 = try store.reserveSlot(); - store.fillReservedSlot(b1, .{ .tuple = try store.addSpan(&.{ b2, b1 }) }); - store.fillReservedSlot(b2, .{ .tuple = try store.addSpan(&.{ b1, c }) }); - - try std.testing.expect(try store.typeEql(&name_store, b1, c)); - try std.testing.expect(try store.typeEql(&name_store, b2, c)); - - const c_digest = store.typeDigest(&name_store, c); - const b1_digest = store.typeDigest(&name_store, b1); - const b2_digest = store.typeDigest(&name_store, b2); - try std.testing.expect(!std.mem.eql(u8, b1_digest.bytes[0..], c_digest.bytes[0..])); - try std.testing.expect(!std.mem.eql(u8, b2_digest.bytes[0..], c_digest.bytes[0..])); - - // Digests stay stable on repeat queries. - const b1_again = store.typeDigest(&name_store, b1); - try std.testing.expectEqualSlices(u8, b1_digest.bytes[0..], b1_again.bytes[0..]); + try std.testing.expect(try store.typeEql(&name_store, b1, c)); + try std.testing.expect(try store.typeEql(&name_store, b2, c)); + + const first_digest = store.typeDigest(&name_store, if (copy_first) b1 else c); + for ([_]TypeId{ c, b1, b2 }) |ty| { + try std.testing.expectEqualSlices(u8, first_digest.bytes[0..], store.typeDigest(&name_store, ty).bytes[0..]); + } + } } test "monotype digest separates tag unions by checked name" { @@ -7023,10 +6520,10 @@ test "monotype read sharing preparation retries recursive digest publication aft try std.testing.expectEqualDeep(expected.equality_digests.unsafeRawItemsForView(), store.equality_digests.unsafeRawItemsForView()); try std.testing.expectEqualDeep(expected.specialization_digests.unsafeRawItemsForView(), store.specialization_digests.unsafeRawItemsForView()); try std.testing.expectEqualDeep(expected.containment_cache.unsafeRawItemsForView(), store.containment_cache.unsafeRawItemsForView()); - try std.testing.expectEqual(expected.recursive_digest_unfoldings.count(), store.recursive_digest_unfoldings.count()); - var unfoldings = expected.recursive_digest_unfoldings.iterator(); + try std.testing.expectEqual(expected.recursive_digests.unfoldings.count(), store.recursive_digests.unfoldings.count()); + var unfoldings = expected.recursive_digests.unfoldings.iterator(); while (unfoldings.next()) |entry| { - try std.testing.expectEqual(entry.value_ptr.*, store.recursive_digest_unfoldings.get(entry.key_ptr.*).?); + try std.testing.expectEqual(entry.value_ptr.*, store.recursive_digests.unfoldings.get(entry.key_ptr.*).?); } // The prepared worker view must answer every query without allocation, // including each recursive root and its one-step unfolding. @@ -7069,9 +6566,9 @@ test "monotype digest byte fixtures preserve scalar and recursive encodings" { // Digests of the current encoding, which names a named type's checked // occurrence by its id and the module whose type store the id indexes. const expected = [_][]const u8{ - "0eeb10ebb7da5f2248b61d80cfb52c2d93417bc692c528deb12cfaf96c6896d1", - "285944b1cda1306ad6ab6a86397a97c6f0b24ece8365c3c2cf11fa961a9ff5d1", - "51999cf46a730a706829bbdcb8d7dfab9c4457ddf5d8ea11d297e896f4a3cafb", + "51f21375a3e6bfea7625324f0b95e42a3b47bf54b331bb7da4e3e7a3619a3cfa", + "3aa55077e7dcae806b1a7a706f7cb039b6220adab61952fa683bd2c8d10768b1", + "25d4e8a313f03981552dd326d3df189e1608c38707294112b80ea02234781776", }; for ([_]Store.NamedDigestMode{ .full, .identity_only, .equality }, expected) |mode, hex| { const digest = try store.computeDigest(&name_store, function, mode, null); diff --git a/src/postcheck/proc_identity.zig b/src/postcheck/proc_identity.zig index 2e51bff0fbb..f0089f1b160 100644 --- a/src/postcheck/proc_identity.zig +++ b/src/postcheck/proc_identity.zig @@ -20,21 +20,13 @@ //! closure returns a stream holding that same closure, and recursive nominal //! types are cycles in the graph by nature. //! -//! Two programs draw the same recursive type as different graphs: one may -//! hold a single node per knot while another rolls the knot out a level -//! before tying it back, depending on which parts of the type each program's -//! lambda solving touched. The identity must not see that difference, so a -//! type digests as its bisimulation class rather than as its graph. Every -//! request runs the same reduction Monotype's type digests use: the reachable -//! subgraph is decomposed into strongly connected components, which resolve -//! in reverse topological order; an acyclic node hashes its encoding with its -//! children's finished digests, and a cyclic component is refined to its -//! coarsest bisimulation partition, rendered once as a group in label order -//! with in-group references written as ranks, and every member digests as its -//! rank against the group. Each member's one-step unfolding (its encoding -//! with every child as a finished digest) is remembered, so an acyclic node -//! that is a rolled-out copy of a group position folds to that position's -//! digest instead of getting one of its own. +//! Two programs draw the same recursive type as different graphs, depending +//! on which parts of the type each program's lambda solving touched: one may +//! tie a single knot where another rolls it out a level, or ties one copy +//! whose references lead into a second, separately tied copy. The identity +//! must not see that difference, so a type digests as its infinite unfolding +//! through the reduction every type identity shares +//! (`base.recursive_digest`). //! //! Every digest is a function of the infinite type alone, so the `Memo` shared //! across procedures remembers every type it has rendered: a solved type graph @@ -53,21 +45,23 @@ const Lifted = @import("monotype_lifted/ast.zig"); const SolvedType = @import("lambda_solved/type.zig"); const names = check.CheckedNames; const TypeDigestHasher = base.TypeDigestHasher; +const recursive_digest = base.recursive_digest; const Allocator = std.mem.Allocator; /// SHA-256 content identity of one procedure. pub const Identity = [TypeDigestHasher.digest_length]u8; -const domain = "roc.proc.identity.v3"; +const domain = "roc.proc.identity.v4"; /// Everything one solved program has rendered so far, shared by every /// identity rendered over it. pub const Memo = struct { /// Digest of every type rendered so far, keyed by its root variable. digests: collections.DenseMap(SolvedType.TypeVarId, Identity), - /// The one-step unfolding of every cyclic group position rendered so far, - /// mapped to that position's digest. - unfoldings: std.AutoHashMap(Identity, Identity), + /// Every recursive group rendered so far. + settled: recursive_digest.Settled, + /// Reduction scratch kept across reductions. + graph: recursive_digest.Graph, /// Reduction node maps kept across reductions. A fresh map spanning old /// and new variables would allocate and clear a chunk table for that /// whole span on every reduction. @@ -76,14 +70,16 @@ pub const Memo = struct { pub fn init(allocator: Allocator) Memo { return .{ .digests = collections.DenseMap(SolvedType.TypeVarId, Identity).init(allocator), - .unfoldings = std.AutoHashMap(Identity, Identity).init(allocator), + .settled = recursive_digest.Settled.init(allocator, domain), + .graph = recursive_digest.Graph.init(allocator), .node_maps = collections.DenseMapPool(SolvedType.TypeVarId, u32).init(allocator), }; } pub fn deinit(self: *Memo) void { self.digests.deinit(); - self.unfoldings.deinit(); + self.settled.deinit(); + self.graph.deinit(); self.node_maps.deinit(); } }; @@ -162,122 +158,75 @@ pub const Renderer = struct { } }; -/// A child position in a node's encoding: another node of the current -/// reduction, or a type whose digest is already known. -const Child = union(enum) { - node: u32, - digest: Identity, -}; - -/// One piece of a node's encoding, in order: literal bytes or a child. -const Item = union(enum) { - bytes: struct { start: u32, len: u32 }, - child: Child, -}; - -const Node = struct { - ty: SolvedType.TypeVarId, - item_start: u32 = 0, - item_len: u32 = 0, - child_start: u32 = 0, - child_len: u32 = 0, - index: u32 = unvisited, - lowlink: u32 = unvisited, - on_stack: bool = false, - digest: ?Identity = null, -}; - -const unvisited = std.math.maxInt(u32); -const no_position = std.math.maxInt(u32); - /// One reduction: the subgraph reachable from a requested type through /// types the memo does not hold yet. const Engine = struct { renderer: *const Renderer, gpa: Allocator, - nodes: std.ArrayList(Node) = .empty, + graph: *recursive_digest.Graph, node_of_ty: collections.DenseMap(SolvedType.TypeVarId, u32), - items: std.ArrayList(Item) = .empty, - children: std.ArrayList(Child) = .empty, - bytes: std.ArrayList(u8) = .empty, + /// The type of every node of the reduction. + tys: std.ArrayList(SolvedType.TypeVarId) = .empty, /// Nodes discovered but not yet encoded. pending: std.ArrayList(u32) = .empty, - /// Scratch `resolveCyclicComponent` keeps across the components of one - /// run. `position_of_node` covers every node and holds `no_position` - /// except at the members of the component being resolved. - position_of_node: std.ArrayList(u32) = .empty, - label_scratch: std.ArrayList(Identity) = .empty, - distinct_labels: std.AutoHashMapUnmanaged(Identity, u32) = .empty, - sorted_labels: std.ArrayList(Identity) = .empty, - rank_of_member: std.ArrayList(u32) = .empty, - block_rep: std.ArrayList(u32) = .empty, + /// Content of the node being encoded. + hasher: TypeDigestHasher = undefined, fn init(renderer: *const Renderer) Engine { + renderer.memo.graph.reset(); return .{ .renderer = renderer, .gpa = renderer.allocator, + .graph = &renderer.memo.graph, .node_of_ty = renderer.memo.node_maps.acquire(), }; } fn deinit(self: *Engine) void { - self.nodes.deinit(self.gpa); + self.graph.reset(); self.renderer.memo.node_maps.release(&self.node_of_ty); - self.items.deinit(self.gpa); - self.children.deinit(self.gpa); - self.bytes.deinit(self.gpa); + self.tys.deinit(self.gpa); self.pending.deinit(self.gpa); - self.position_of_node.deinit(self.gpa); - self.label_scratch.deinit(self.gpa); - self.distinct_labels.deinit(self.gpa); - self.sorted_labels.deinit(self.gpa); - self.rank_of_member.deinit(self.gpa); - self.block_rep.deinit(self.gpa); } fn run(self: *Engine, root: SolvedType.TypeVarId) Allocator.Error!Identity { const root_node = try self.discover(root); while (self.pending.pop()) |node| try self.encode(node); - try self.resolveAll(); - return self.nodes.items[root_node].digest orelse - Common.invariant("procedure identity reduction left its root type without a digest"); + const memo = self.renderer.memo; + try self.graph.resolve(&memo.settled, null); + for (self.tys.items, 0..) |ty, node| try memo.digests.put(ty, self.graph.digest(@intCast(node))); + return self.graph.digest(root_node); } /// The node for `ty`, created and queued for encoding if new. fn discover(self: *Engine, ty: SolvedType.TypeVarId) Allocator.Error!u32 { const gop = try self.node_of_ty.getOrPut(ty); if (gop.found_existing) return gop.value_ptr.*; - const index: u32 = @intCast(self.nodes.items.len); - gop.value_ptr.* = index; - try self.nodes.append(self.gpa, .{ .ty = ty }); - try self.pending.append(self.gpa, index); - return index; - } - - /// The child position for `ty`: a finished digest when the memo holds - /// it, otherwise a node of this reduction. - fn childFor(self: *Engine, ty: SolvedType.TypeVarId) Allocator.Error!Child { - const root = self.renderer.renderedRoot(ty); - if (self.renderer.memo.digests.get(root)) |digest| return .{ .digest = digest }; - return .{ .node = try self.discover(root) }; + const node = try self.graph.addNode(); + gop.value_ptr.* = node; + try self.tys.append(self.gpa, ty); + try self.pending.append(self.gpa, node); + return node; } fn putBytes(self: *Engine, bytes: []const u8) Allocator.Error!void { - const start: u32 = @intCast(self.bytes.items.len); - try self.bytes.appendSlice(self.gpa, bytes); - try self.items.append(self.gpa, .{ .bytes = .{ .start = start, .len = @intCast(bytes.len) } }); + writeBytes(&self.hasher, bytes); } fn putU32(self: *Engine, value: u32) Allocator.Error!void { - var buffer: [4]u8 = undefined; - encodeU32(&buffer, value); - try self.putBytes(&buffer); + writeU32(&self.hasher, value); } + /// A child slot: a type the memo holds is a settled digest, any other is + /// a node of this reduction. fn putChild(self: *Engine, ty: SolvedType.TypeVarId) Allocator.Error!void { - const child = try self.childFor(ty); - try self.items.append(self.gpa, .{ .child = child }); - try self.children.append(self.gpa, child); + writeBytes(&self.hasher, "child"); + const root = self.renderer.renderedRoot(ty); + if (self.renderer.memo.digests.get(root)) |digest| { + try self.graph.putChild(.{ .settled = digest }); + } else { + try self.graph.putChild(.{ .node = try self.discover(root) }); + } } fn putSpan(self: *Engine, span: SolvedType.Span) Allocator.Error!void { @@ -306,9 +255,9 @@ const Engine = struct { /// and its ordered children. fn encode(self: *Engine, node: u32) Allocator.Error!void { const renderer = self.renderer; - const item_start: u32 = @intCast(self.items.items.len); - const child_start: u32 = @intCast(self.children.items.len); - const content = renderer.types.get(self.nodes.items[node].ty); + self.hasher = TypeDigestHasher.init(); + self.graph.beginNode(node); + const content = renderer.types.get(self.tys.items[node]); switch (content) { .mono, .link, .unbound, .forall => Common.invariant("unresolved Lambda Solved type reached procedure identity rendering"), .primitive => |primitive| { @@ -409,291 +358,7 @@ const Engine = struct { }, .lambda_set => |members| try self.putMembers(members), } - const entry = &self.nodes.items[node]; - entry.item_start = item_start; - entry.item_len = @intCast(self.items.items.len - item_start); - entry.child_start = child_start; - entry.child_len = @intCast(self.children.items.len - child_start); - } - - fn itemsOf(self: *const Engine, node: u32) []const Item { - const entry = self.nodes.items[node]; - return self.items.items[entry.item_start .. entry.item_start + entry.item_len]; - } - - fn childrenOf(self: *const Engine, node: u32) []const Child { - const entry = self.nodes.items[node]; - return self.children.items[entry.child_start .. entry.child_start + entry.child_len]; - } - - fn bytesOf(self: *const Engine, start: u32, len: u32) []const u8 { - return self.bytes.items[start .. start + len]; - } - - /// Iterative Tarjan discovery over every node. Components pop in - /// reverse topological order, so every component a node reaches is - /// resolved before the node's own. - fn resolveAll(self: *Engine) Allocator.Error!void { - const Frame = struct { node: u32, next_child: u32 }; - var frames: std.ArrayList(Frame) = .empty; - defer frames.deinit(self.gpa); - var scc_stack: std.ArrayList(u32) = .empty; - defer scc_stack.deinit(self.gpa); - var members: std.ArrayList(u32) = .empty; - defer members.deinit(self.gpa); - var next_index: u32 = 0; - - for (0..self.nodes.items.len) |start_index| { - const start: u32 = @intCast(start_index); - if (self.nodes.items[start].index != unvisited) continue; - try self.visit(start, &next_index, &scc_stack); - try frames.append(self.gpa, .{ .node = start, .next_child = 0 }); - while (frames.items.len != 0) { - const frame = &frames.items[frames.items.len - 1]; - const node = frame.node; - const children = self.childrenOf(node); - if (frame.next_child < children.len) { - const child = children[frame.next_child]; - frame.next_child += 1; - const target = switch (child) { - .digest => continue, - .node => |target| target, - }; - const target_entry = &self.nodes.items[target]; - if (target_entry.index == unvisited) { - try self.visit(target, &next_index, &scc_stack); - try frames.append(self.gpa, .{ .node = target, .next_child = 0 }); - } else if (target_entry.on_stack) { - const entry = &self.nodes.items[node]; - entry.lowlink = @min(entry.lowlink, target_entry.index); - } - continue; - } - _ = frames.pop(); - const entry = self.nodes.items[node]; - if (frames.items.len != 0) { - const parent = &self.nodes.items[frames.items[frames.items.len - 1].node]; - parent.lowlink = @min(parent.lowlink, entry.lowlink); - } - if (entry.lowlink == entry.index) { - members.clearRetainingCapacity(); - while (true) { - const member = scc_stack.pop() orelse - Common.invariant("procedure identity reduction popped past its component root"); - self.nodes.items[member].on_stack = false; - try members.append(self.gpa, member); - if (member == node) break; - } - try self.resolveComponent(members.items); - } - } - } - } - - fn visit(self: *Engine, node: u32, next_index: *u32, scc_stack: *std.ArrayList(u32)) Allocator.Error!void { - const entry = &self.nodes.items[node]; - entry.index = next_index.*; - entry.lowlink = next_index.*; - entry.on_stack = true; - next_index.* += 1; - try scc_stack.append(self.gpa, node); - } - - fn resolveComponent(self: *Engine, members: []const u32) Allocator.Error!void { - if (members.len == 1 and !self.hasSelfEdge(members[0])) { - const node = members[0]; - const key = self.unfoldingKey(node); - const digest = self.renderer.memo.unfoldings.get(key) orelse key; - try self.finalize(node, digest); - return; - } - try self.resolveCyclicComponent(members); - } - - fn hasSelfEdge(self: *const Engine, node: u32) bool { - for (self.childrenOf(node)) |child| switch (child) { - .node => |target| if (target == node) return true, - .digest => {}, - }; - return false; - } - - fn childDigest(self: *const Engine, child: Child) Identity { - return switch (child) { - .digest => |digest| digest, - .node => |target| self.nodes.items[target].digest orelse - Common.invariant("procedure identity reduction read a child digest before resolving it"), - }; - } - - /// The node's encoding with every child as its finished digest. - fn unfoldingKey(self: *const Engine, node: u32) Identity { - var hasher = TypeDigestHasher.init(); - writeBytes(&hasher, "node"); - for (self.itemsOf(node)) |item| switch (item) { - .bytes => |range| writeBytes(&hasher, self.bytesOf(range.start, range.len)), - .child => |child| { - writeBytes(&hasher, "child"); - hasher.update(&self.childDigest(child)); - }, - }; - return hasher.finalResult(); - } - - fn finalize(self: *Engine, node: u32, digest: Identity) Allocator.Error!void { - const entry = &self.nodes.items[node]; - if (entry.digest != null) Common.invariant("procedure identity reduction resolved a type twice"); - entry.digest = digest; - try self.renderer.memo.digests.put(entry.ty, digest); - } - - /// Reduce one cyclic component by bisimulation refinement over content - /// labels, order the reduced positions by their final labels, and digest - /// every member as its position in that one group rendering. - fn resolveCyclicComponent(self: *Engine, members: []const u32) Allocator.Error!void { - const member_count = members.len; - const known_nodes = self.position_of_node.items.len; - if (known_nodes < self.nodes.items.len) { - try self.position_of_node.appendNTimes(self.gpa, no_position, self.nodes.items.len - known_nodes); - } - const position_of_node = self.position_of_node.items; - for (members, 0..) |node, pos| position_of_node[node] = @intCast(pos); - defer for (members) |node| { - position_of_node[node] = no_position; - }; - - // Refine to the stable bisimulation partition. Every member starts - // from its content label: its encoding with every out-of-component - // child as a finished digest and every in-component child as a bare - // marker. Each round relabels a member by its own label followed by - // the labels of its ordered in-component children until a round - // stops separating members. A label is a pure function of the - // member's unfolding to the current depth, so two members carry - // equal labels exactly when they are bisimilar, and the labels are - // identical for bisimilar positions of any two knots regardless of - // how many nodes either knot uses. - try self.label_scratch.resize(self.gpa, 2 * member_count); - var labels = self.label_scratch.items[0..member_count]; - var next_labels = self.label_scratch.items[member_count..]; - const distinct_labels = &self.distinct_labels; - distinct_labels.clearRetainingCapacity(); - for (members, 0..) |node, pos| { - var hasher = TypeDigestHasher.init(); - writeBytes(&hasher, "label"); - for (self.itemsOf(node)) |item| switch (item) { - .bytes => |range| writeBytes(&hasher, self.bytesOf(range.start, range.len)), - .child => |child| { - const in_component = switch (child) { - .digest => false, - .node => |target| position_of_node[target] != no_position, - }; - if (in_component) { - writeBytes(&hasher, "in-component"); - } else { - writeBytes(&hasher, "child"); - hasher.update(&self.childDigest(child)); - } - }, - }; - labels[pos] = hasher.finalResult(); - try distinct_labels.put(self.gpa, labels[pos], no_position); - } - var label_count: u32 = distinct_labels.count(); - while (true) { - distinct_labels.clearRetainingCapacity(); - for (members, 0..) |node, pos| { - var hasher = TypeDigestHasher.init(); - hasher.update(&labels[pos]); - for (self.childrenOf(node)) |child| { - const target = switch (child) { - .digest => continue, - .node => |target| target, - }; - const target_pos = position_of_node[target]; - if (target_pos == no_position) continue; - hasher.update(&labels[target_pos]); - } - next_labels[pos] = hasher.finalResult(); - try distinct_labels.put(self.gpa, next_labels[pos], no_position); - } - const next_count: u32 = distinct_labels.count(); - const stable = next_count == label_count; - std.mem.swap([]Identity, &labels, &next_labels); - label_count = next_count; - if (stable) break; - } - - // Group order: the reduced positions sorted by label. Labels are - // intrinsic to the infinite type, so the order is too. - const block_count = label_count; - try self.sorted_labels.resize(self.gpa, block_count); - const sorted_labels = self.sorted_labels.items; - { - var it = distinct_labels.keyIterator(); - var next: usize = 0; - while (it.next()) |label| : (next += 1) sorted_labels[next] = label.*; - std.debug.assert(next == block_count); - } - std.mem.sort(Identity, sorted_labels, {}, struct { - fn lessThan(_: void, lhs: Identity, rhs: Identity) bool { - return std.mem.order(u8, &lhs, &rhs) == .lt; - } - }.lessThan); - for (sorted_labels, 0..) |label, rank| { - const slot = distinct_labels.getPtr(label) orelse unreachable; // sorted_labels holds exactly the keys of distinct_labels - slot.* = @intCast(rank); - } - try self.rank_of_member.resize(self.gpa, member_count); - const rank_of_member = self.rank_of_member.items; - try self.block_rep.resize(self.gpa, block_count); - const block_rep = self.block_rep.items; - @memset(block_rep, no_position); - for (members, 0..) |node, pos| { - const rank = distinct_labels.get(labels[pos]) orelse unreachable; // distinct_labels holds every label of the final stable round - rank_of_member[pos] = rank; - // The partition is stable, so any member represents its position. - if (block_rep[rank] == no_position) block_rep[rank] = node; - } - - // One group rendering: every reduced position in label order with - // in-component references written as ranks. Every member then - // digests as its rank against the rendering's digest. - var group_hasher = TypeDigestHasher.init(); - writeBytes(&group_hasher, "group"); - writeU32(&group_hasher, block_count); - for (block_rep) |rep| { - writeBytes(&group_hasher, "position"); - for (self.itemsOf(rep)) |item| switch (item) { - .bytes => |range| writeBytes(&group_hasher, self.bytesOf(range.start, range.len)), - .child => |child| { - const in_component: ?u32 = switch (child) { - .digest => null, - .node => |target| if (position_of_node[target] == no_position) null else rank_of_member[position_of_node[target]], - }; - if (in_component) |rank| { - writeBytes(&group_hasher, "rank"); - writeU32(&group_hasher, rank); - } else { - writeBytes(&group_hasher, "child"); - group_hasher.update(&self.childDigest(child)); - } - }, - }; - } - const group_digest = group_hasher.finalResult(); - for (members, 0..) |node, pos| { - var hasher = TypeDigestHasher.init(); - writeBytes(&hasher, "group-member"); - hasher.update(&group_digest); - writeU32(&hasher, rank_of_member[pos]); - try self.finalize(node, hasher.finalResult()); - } - // A rolled-out copy of any position unfolds to the same one-step - // encoding, so it folds to that position's digest. - for (members) |node| { - const digest = self.nodes.items[node].digest orelse unreachable; // the loop above finalized every member's digest - try self.renderer.memo.unfoldings.put(self.unfoldingKey(node), digest); - } + self.graph.endNode(node, self.hasher.finalResult()); } }; @@ -702,23 +367,8 @@ fn transparentAliasBacking(content: SolvedType.Content) ?SolvedType.TypeVarId { return (content.named.backing orelse Common.invariant("transparent alias reached procedure identity rendering without a backing type")).ty; } -fn writeBytes(hasher: *TypeDigestHasher, bytes: []const u8) void { - writeU32(hasher, @intCast(bytes.len)); - hasher.update(bytes); -} - -fn encodeU32(buffer: *[4]u8, value: u32) void { - buffer[0] = @truncate(value); - buffer[1] = @truncate(value >> 8); - buffer[2] = @truncate(value >> 16); - buffer[3] = @truncate(value >> 24); -} - -fn writeU32(hasher: *TypeDigestHasher, value: u32) void { - var buffer: [4]u8 = undefined; - encodeU32(&buffer, value); - hasher.update(&buffer); -} +const writeBytes = recursive_digest.writeBytes; +const writeU32 = recursive_digest.writeU32; test "procedure identity excludes outer callable sets but retains nested callable sets" { const allocator = std.testing.allocator; @@ -820,3 +470,62 @@ test "procedure identity excludes outer callable sets but retains nested callabl &try erased_renderer.specIdentity(first_fn, takes_boxed_other, &.{}, "finite", "none"), )); } + +test "procedure identity digests a knot entangled with a separately tied copy as that copy" { + // `copy` and `copy_list` tie a knot of List(Node) whose second reference + // leads into `node`, a separately tied knot of the same infinite type, as + // one program's lambda solving can draw it. Another program draws only + // `node`. Both must render one identity, whichever is rendered first. + const allocator = std.testing.allocator; + var types = SolvedType.Store.init(allocator); + defer types.deinit(); + var name_store = names.NameStore.init(allocator); + defer name_store.deinit(); + var fn_by_symbol = std.AutoHashMap(Common.Symbol, Lifted.FnId).init(allocator); + defer fn_by_symbol.deinit(); + + const node = try types.add(.zst); + const node_list = try types.add(.{ .list = node }); + try types.set(node, .{ .tuple = try types.addSpan(&.{ node_list, node_list }) }); + const copy = try types.add(.zst); + const copy_list = try types.add(.{ .list = copy }); + try types.set(copy, .{ .tuple = try types.addSpan(&.{ copy_list, node_list }) }); + + var digests: [2][2]Identity = undefined; + for (&digests, [_][2]SolvedType.TypeVarId{ .{ node, copy }, .{ copy, node } }) |*out, order| { + var memo = Memo.init(allocator); + defer memo.deinit(); + const renderer = Renderer{ + .allocator = allocator, + .types = types.view(), + .names = &name_store, + .fn_tys = &.{}, + .source_digests = &.{}, + .fn_by_symbol = &fn_by_symbol, + .memo = &memo, + }; + out[0] = try renderer.typeDigest(order[0]); + out[1] = try renderer.typeDigest(order[1]); + } + try std.testing.expectEqual(digests[0][0], digests[0][1]); + try std.testing.expectEqual(digests[1][0], digests[1][1]); + try std.testing.expectEqual(digests[0][0], digests[1][0]); + + var alone_types = SolvedType.Store.init(allocator); + defer alone_types.deinit(); + const alone = try alone_types.add(.zst); + const alone_list = try alone_types.add(.{ .list = alone }); + try alone_types.set(alone, .{ .tuple = try alone_types.addSpan(&.{ alone_list, alone_list }) }); + var memo = Memo.init(allocator); + defer memo.deinit(); + const renderer = Renderer{ + .allocator = allocator, + .types = alone_types.view(), + .names = &name_store, + .fn_tys = &.{}, + .source_digests = &.{}, + .fn_by_symbol = &fn_by_symbol, + .memo = &memo, + }; + try std.testing.expectEqual(digests[0][0], try renderer.typeDigest(alone)); +} diff --git a/src/postcheck/type_digest.zig b/src/postcheck/type_digest.zig deleted file mode 100644 index c0ae1477109..00000000000 --- a/src/postcheck/type_digest.zig +++ /dev/null @@ -1,340 +0,0 @@ -//! Reusable structural type-digest graph reduction. -//! Nodes contain scalar content digests and explicit ordered child edges. Discovery is -//! owned by the producer; this reducer resolves acyclic nodes once and reduces -//! cyclic components by bisimulation before hashing positions ordered by label. -const std = @import("std"); -const TypeDigestHasher = @import("base").TypeDigestHasher; -const Common = @import("common.zig"); -const Allocator = std.mem.Allocator; -/// SHA-256 structural digest. -pub const Identity = [32]u8; -const Node = struct { - scalar: Identity = undefined, - child_start: u32 = 0, - child_len: u32 = 0, - index: u32 = unvisited, - lowlink: u32 = unvisited, - on_stack: bool = false, - digest: ?Identity = null, -}; -const unvisited = std.math.maxInt(u32); -const no_position = std.math.maxInt(u32); -const Frame = struct { node: u32, next_child: u32 }; -/// Scratch graph. Reset discards every result but retains all buffer capacity. -pub const Graph = struct { - gpa: Allocator, - nodes: std.ArrayList(Node) = .empty, - children: std.ArrayList(u32) = .empty, - frames: std.ArrayList(Frame) = .empty, - scc_stack: std.ArrayList(u32) = .empty, - members: std.ArrayList(u32) = .empty, - positions: std.ArrayList(u32) = .empty, - labels: std.ArrayList(Identity) = .empty, - next_labels: std.ArrayList(Identity) = .empty, - sorted_labels: std.ArrayList(Identity) = .empty, - ranks: std.ArrayList(u32) = .empty, - representatives: std.ArrayList(u32) = .empty, - distinct_labels: std.AutoHashMap(Identity, u32), - unfoldings: std.AutoHashMap(Identity, Identity), - - /// Create empty scratch buffers. - pub fn init(gpa: Allocator) Graph { - return .{ .gpa = gpa, .distinct_labels = std.AutoHashMap(Identity, u32).init(gpa), .unfoldings = std.AutoHashMap(Identity, Identity).init(gpa) }; - } - /// Release retained buffers. - pub fn deinit(self: *Graph) void { - inline for (.{ "nodes", "children", "frames", "scc_stack", "members", "positions", "labels", "next_labels", "sorted_labels", "ranks", "representatives" }) |field| @field(self, field).deinit(self.gpa); - self.distinct_labels.deinit(); - self.unfoldings.deinit(); - } - /// Start a new request, including after allocation failure. - pub fn reset(self: *Graph) void { - inline for (.{ "nodes", "children", "frames", "scc_stack", "members", "positions", "labels", "next_labels", "sorted_labels", "ranks", "representatives" }) |field| @field(self, field).clearRetainingCapacity(); - self.distinct_labels.clearRetainingCapacity(); - self.unfoldings.clearRetainingCapacity(); - } - /// Reserve a node before discovering its children. - pub fn addNode(self: *Graph) Allocator.Error!u32 { - const node: u32 = @intCast(self.nodes.items.len); - try self.nodes.append(self.gpa, .{}); - return node; - } - /// Begin the ordered encoding of a reserved node. - pub fn beginNode(self: *Graph, node: u32) void { - self.nodes.items[node].child_start = @intCast(self.children.items.len); - } - /// Finish the ordered encoding of a reserved node. - pub fn endNode(self: *Graph, node: u32) void { - const entry = &self.nodes.items[node]; - entry.child_len = @intCast(self.children.items.len - entry.child_start); - } - /// Record the producer's complete scalar content digest for a node. - pub fn setScalar(self: *Graph, node: u32, scalar: Identity) void { - self.nodes.items[node].scalar = scalar; - } - /// Append one child edge. - pub fn putChild(self: *Graph, node: u32) Allocator.Error!void { - try self.children.append(self.gpa, node); - } - /// Resolve the graph and return the requested node's digest. - pub fn resolve(self: *Graph, root: u32) Allocator.Error!Identity { - try self.resolveAll(); - return self.nodes.items[root].digest orelse unreachable; // resolveAll finalizes the digest of every node in the graph - } - fn childrenOf(self: *const Graph, node: u32) []const u32 { - const entry = self.nodes.items[node]; - return self.children.items[entry.child_start .. entry.child_start + entry.child_len]; - } - - /// Iterative Tarjan discovery over every node. Components pop in - /// reverse topological order, so every component a node reaches is - /// resolved before the node's own. - fn resolveAll(self: *Graph) Allocator.Error!void { - const frames = &self.frames; - const scc_stack = &self.scc_stack; - const members = &self.members; - var next_index: u32 = 0; - try self.positions.resize(self.gpa, self.nodes.items.len); - @memset(self.positions.items, no_position); - - for (0..self.nodes.items.len) |start_index| { - const start: u32 = @intCast(start_index); - if (self.nodes.items[start].index != unvisited) continue; - try self.visit(start, &next_index, scc_stack); - try frames.append(self.gpa, .{ .node = start, .next_child = 0 }); - while (frames.items.len != 0) { - const frame = &frames.items[frames.items.len - 1]; - const node = frame.node; - const children = self.childrenOf(node); - if (frame.next_child < children.len) { - const target = children[frame.next_child]; - frame.next_child += 1; - const target_entry = &self.nodes.items[target]; - if (target_entry.index == unvisited) { - try self.visit(target, &next_index, scc_stack); - try frames.append(self.gpa, .{ .node = target, .next_child = 0 }); - } else if (target_entry.on_stack) { - const entry = &self.nodes.items[node]; - entry.lowlink = @min(entry.lowlink, target_entry.index); - } - continue; - } - _ = frames.pop(); - const entry = self.nodes.items[node]; - if (frames.items.len != 0) { - const parent = &self.nodes.items[frames.items[frames.items.len - 1].node]; - parent.lowlink = @min(parent.lowlink, entry.lowlink); - } - if (entry.lowlink == entry.index) { - members.clearRetainingCapacity(); - while (true) { - const member = scc_stack.pop() orelse - Common.invariant("type digest reduction popped past its component root"); - self.nodes.items[member].on_stack = false; - try members.append(self.gpa, member); - if (member == node) break; - } - try self.resolveComponent(members.items); - } - } - } - } - - fn visit(self: *Graph, node: u32, next_index: *u32, scc_stack: *std.ArrayList(u32)) Allocator.Error!void { - const entry = &self.nodes.items[node]; - entry.index = next_index.*; - entry.lowlink = next_index.*; - entry.on_stack = true; - next_index.* += 1; - try scc_stack.append(self.gpa, node); - } - - fn resolveComponent(self: *Graph, members: []const u32) Allocator.Error!void { - if (members.len == 1 and !self.hasSelfEdge(members[0])) { - const node = members[0]; - const key = self.unfoldingKey(node); - const digest = self.unfoldings.get(key) orelse key; - self.finalize(node, digest); - return; - } - try self.resolveCyclicComponent(members); - } - - fn hasSelfEdge(self: *const Graph, node: u32) bool { - for (self.childrenOf(node)) |child| if (child == node) return true; - return false; - } - - fn childDigest(self: *const Graph, child: u32) Identity { - return self.nodes.items[child].digest orelse - Common.invariant("type digest reduction read a child digest before resolving it"); - } - - /// The node's encoding with every child as its finished digest. - fn unfoldingKey(self: *const Graph, node: u32) Identity { - var hasher = TypeDigestHasher.init(); - writeBytes(&hasher, "roc.lambda-mono.type.v2"); - writeBytes(&hasher, "node"); - hasher.update(&self.nodes.items[node].scalar); - for (self.childrenOf(node)) |child| { - writeBytes(&hasher, "child"); - hasher.update(&self.childDigest(child)); - } - return hasher.finalResult(); - } - - fn finalize(self: *Graph, node: u32, digest: Identity) void { - const entry = &self.nodes.items[node]; - if (entry.digest != null) Common.invariant("type digest reduction resolved a type twice"); - entry.digest = digest; - } - - /// Reduce one cyclic component by bisimulation refinement over content - /// labels, order the reduced positions by their final labels, and digest - /// every member as its position in that one group rendering. - fn resolveCyclicComponent(self: *Graph, members: []const u32) Allocator.Error!void { - const member_count = members.len; - const position_of_node = self.positions.items; - defer for (members) |node| { - position_of_node[node] = no_position; - }; - for (members, 0..) |node, pos| position_of_node[node] = @intCast(pos); - - // Refine to the stable bisimulation partition. Every member starts - // from its content label: its encoding with every out-of-component - // child as a finished digest and every in-component child as a bare - // marker. Each round relabels a member by its own label followed by - // the labels of its ordered in-component children until a round - // stops separating members. A label is a pure function of the - // member's unfolding to the current depth, so two members carry - // equal labels exactly when they are bisimilar, and the labels are - // identical for bisimilar positions of any two knots regardless of - // how many nodes either knot uses. - try self.labels.resize(self.gpa, member_count); - try self.next_labels.resize(self.gpa, member_count); - var labels = self.labels.items; - var next_labels = self.next_labels.items; - const distinct_labels = &self.distinct_labels; - distinct_labels.clearRetainingCapacity(); - for (members, 0..) |node, pos| { - var hasher = TypeDigestHasher.init(); - writeBytes(&hasher, "label"); - hasher.update(&self.nodes.items[node].scalar); - for (self.childrenOf(node)) |child| { - if (position_of_node[child] != no_position) { - writeBytes(&hasher, "in-component"); - } else { - writeBytes(&hasher, "child"); - hasher.update(&self.childDigest(child)); - } - } - labels[pos] = hasher.finalResult(); - try distinct_labels.put(labels[pos], no_position); - } - var label_count: u32 = distinct_labels.count(); - while (true) { - distinct_labels.clearRetainingCapacity(); - for (members, 0..) |node, pos| { - var hasher = TypeDigestHasher.init(); - hasher.update(&labels[pos]); - for (self.childrenOf(node)) |target| { - const target_pos = position_of_node[target]; - if (target_pos == no_position) continue; - hasher.update(&labels[target_pos]); - } - next_labels[pos] = hasher.finalResult(); - try distinct_labels.put(next_labels[pos], no_position); - } - const next_count: u32 = distinct_labels.count(); - const stable = next_count == label_count; - std.mem.swap([]Identity, &labels, &next_labels); - label_count = next_count; - if (stable) break; - } - - // Group order: the reduced positions sorted by label. Labels are - // intrinsic to the infinite type, so the order is too. - const block_count = label_count; - try self.sorted_labels.resize(self.gpa, block_count); - const sorted_labels = self.sorted_labels.items; - { - var it = distinct_labels.keyIterator(); - var next: usize = 0; - while (it.next()) |label| : (next += 1) sorted_labels[next] = label.*; - std.debug.assert(next == block_count); - } - std.mem.sort(Identity, sorted_labels, {}, struct { - fn lessThan(_: void, lhs: Identity, rhs: Identity) bool { - return std.mem.order(u8, &lhs, &rhs) == .lt; - } - }.lessThan); - for (sorted_labels, 0..) |label, rank| { - const slot = distinct_labels.getPtr(label) orelse unreachable; // sorted_labels holds exactly the keys of distinct_labels - slot.* = @intCast(rank); - } - try self.ranks.resize(self.gpa, member_count); - const rank_of_member = self.ranks.items; - try self.representatives.resize(self.gpa, block_count); - const block_rep = self.representatives.items; - @memset(block_rep, no_position); - for (members, 0..) |node, pos| { - const rank = distinct_labels.get(labels[pos]) orelse unreachable; // distinct_labels holds every label of the final stable round - rank_of_member[pos] = rank; - // The partition is stable, so any member represents its position. - if (block_rep[rank] == no_position) block_rep[rank] = node; - } - - // One group rendering: every reduced position in label order with - // in-component references written as ranks. Every member then - // digests as its rank against the rendering's digest. - var group_hasher = TypeDigestHasher.init(); - writeBytes(&group_hasher, "roc.lambda-mono.type.v2"); - writeBytes(&group_hasher, "group"); - writeU32(&group_hasher, block_count); - for (block_rep) |rep| { - writeBytes(&group_hasher, "position"); - group_hasher.update(&self.nodes.items[rep].scalar); - for (self.childrenOf(rep)) |child| { - if (position_of_node[child] != no_position) { - writeBytes(&group_hasher, "rank"); - writeU32(&group_hasher, rank_of_member[position_of_node[child]]); - } else { - writeBytes(&group_hasher, "child"); - group_hasher.update(&self.childDigest(child)); - } - } - } - const group_digest = group_hasher.finalResult(); - for (members, 0..) |node, pos| { - var hasher = TypeDigestHasher.init(); - writeBytes(&hasher, "group-member"); - hasher.update(&group_digest); - writeU32(&hasher, rank_of_member[pos]); - self.finalize(node, hasher.finalResult()); - } - // A rolled-out copy of any position unfolds to the same one-step - // encoding, so it folds to that position's digest. - for (members) |node| { - const digest = self.nodes.items[node].digest orelse unreachable; // the loop above finalized every member's digest - try self.unfoldings.put(self.unfoldingKey(node), digest); - } - } -}; - -fn writeBytes(hasher: *TypeDigestHasher, bytes: []const u8) void { - writeU32(hasher, @intCast(bytes.len)); - hasher.update(bytes); -} - -fn encodeU32(buffer: *[4]u8, value: u32) void { - buffer[0] = @truncate(value); - buffer[1] = @truncate(value >> 8); - buffer[2] = @truncate(value >> 16); - buffer[3] = @truncate(value >> 24); -} - -fn writeU32(hasher: *TypeDigestHasher, value: u32) void { - var buffer: [4]u8 = undefined; - encodeU32(&buffer, value); - hasher.update(&buffer); -} From 52db9350d9a3c224c789bf1507f8a7cd10480810 Mon Sep 17 00:00:00 2001 From: Richard Feldman Date: Tue, 27 Oct 2026 10:07:16 -0400 Subject: [PATCH 2/5] Use findScalar and update dev object hashes for the new procedure identities --- src/base/recursive_digest.zig | 4 +- test/snapshots/dev_object_float_math.md | 62 ++++++++++++------------- 2 files changed, 33 insertions(+), 33 deletions(-) diff --git a/src/base/recursive_digest.zig b/src/base/recursive_digest.zig index b00ab0a2365..18707d0f1cd 100644 --- a/src/base/recursive_digest.zig +++ b/src/base/recursive_digest.zig @@ -610,7 +610,7 @@ pub const Graph = struct { for (self.childrenOf(node)) |child| { if (self.memberChild(child) != null) continue; const member_ref = settled.member_of.get(self.childDigest(child)) orelse continue; - if (std.mem.indexOfScalar(u32, self.candidate_groups.items, member_ref.group) == null) { + if (std.mem.findScalar(u32, self.candidate_groups.items, member_ref.group) == null) { try self.candidate_groups.append(self.gpa, member_ref.group); } } @@ -660,7 +660,7 @@ pub const Graph = struct { } } } - return std.mem.indexOfScalar(u32, paired, no_position) == null; + return std.mem.findScalar(u32, paired, no_position) == null; } }; diff --git a/test/snapshots/dev_object_float_math.md b/test/snapshots/dev_object_float_math.md index 0377c438b9e..6557e542e6a 100644 --- a/test/snapshots/dev_object_float_math.md +++ b/test/snapshots/dev_object_float_math.md @@ -51,37 +51,37 @@ main = |x, y, n| { ~~~ # DEV OUTPUT ~~~ini -x64mac=6fd5136f3792850017b66446ecf13dc17e6e079b8f64c80892afd438973dac06 -x64win=efe16a28d32e1f2b4cef3e899999ff76194a48ed8c0ec8f228b766dbe5475098 -x64mingw=efe16a28d32e1f2b4cef3e899999ff76194a48ed8c0ec8f228b766dbe5475098 -x64freebsd=115b22e345ac30f0236504b531fcec95eac7be53ceb4f3b472e6c4289ad7b6f9 -x64openbsd=815ff1c8c357a7619d79d34e497e516c74c2bb7218c77e720a86b43062279440 -x64netbsd=d25a35d7e048dd89e17d5ce6790b564fbe4b3e99133e08fca0a4f27f4bca07f1 -x64musl=d25a35d7e048dd89e17d5ce6790b564fbe4b3e99133e08fca0a4f27f4bca07f1 -x64glibc=d25a35d7e048dd89e17d5ce6790b564fbe4b3e99133e08fca0a4f27f4bca07f1 -x64linux=d25a35d7e048dd89e17d5ce6790b564fbe4b3e99133e08fca0a4f27f4bca07f1 -x64elf=d25a35d7e048dd89e17d5ce6790b564fbe4b3e99133e08fca0a4f27f4bca07f1 -x64v1mac=6fd5136f3792850017b66446ecf13dc17e6e079b8f64c80892afd438973dac06 -x64v1win=efe16a28d32e1f2b4cef3e899999ff76194a48ed8c0ec8f228b766dbe5475098 -x64v1mingw=efe16a28d32e1f2b4cef3e899999ff76194a48ed8c0ec8f228b766dbe5475098 -x64v1freebsd=115b22e345ac30f0236504b531fcec95eac7be53ceb4f3b472e6c4289ad7b6f9 -x64v1openbsd=815ff1c8c357a7619d79d34e497e516c74c2bb7218c77e720a86b43062279440 -x64v1netbsd=d25a35d7e048dd89e17d5ce6790b564fbe4b3e99133e08fca0a4f27f4bca07f1 -x64v1musl=d25a35d7e048dd89e17d5ce6790b564fbe4b3e99133e08fca0a4f27f4bca07f1 -x64v1glibc=d25a35d7e048dd89e17d5ce6790b564fbe4b3e99133e08fca0a4f27f4bca07f1 -x64v1linux=d25a35d7e048dd89e17d5ce6790b564fbe4b3e99133e08fca0a4f27f4bca07f1 -x64v1elf=d25a35d7e048dd89e17d5ce6790b564fbe4b3e99133e08fca0a4f27f4bca07f1 -arm64mac=7390125e5b782fb22c05cbe2e2de0a8b8df85b8c10f6dfcb94968ab6ec94ba74 -arm64win=1c4d0360a4a6122cebf7d34c3fabc34de2fd555393a90e0188ec33d4bfbf68c2 -arm64mingw=1c4d0360a4a6122cebf7d34c3fabc34de2fd555393a90e0188ec33d4bfbf68c2 -arm64linux=f0a0638e0cc6499f44ec49d0506e5d451a3578dcd4d5131e88aa5b2824a1144c -arm64musl=f0a0638e0cc6499f44ec49d0506e5d451a3578dcd4d5131e88aa5b2824a1144c -arm64glibc=f0a0638e0cc6499f44ec49d0506e5d451a3578dcd4d5131e88aa5b2824a1144c -arm64v1win=1c4d0360a4a6122cebf7d34c3fabc34de2fd555393a90e0188ec33d4bfbf68c2 -arm64v1mingw=1c4d0360a4a6122cebf7d34c3fabc34de2fd555393a90e0188ec33d4bfbf68c2 -arm64v1linux=f0a0638e0cc6499f44ec49d0506e5d451a3578dcd4d5131e88aa5b2824a1144c -arm64v1musl=f0a0638e0cc6499f44ec49d0506e5d451a3578dcd4d5131e88aa5b2824a1144c -arm64v1glibc=f0a0638e0cc6499f44ec49d0506e5d451a3578dcd4d5131e88aa5b2824a1144c +x64mac=82d64fb118f3a3ca13b7e8ab232fcb3d6eaf0de1a725f3d11445e20030b59ee3 +x64win=07ada194ced28237c6495ab60e995924295789c486b151d23e47d4ba517c3035 +x64mingw=07ada194ced28237c6495ab60e995924295789c486b151d23e47d4ba517c3035 +x64freebsd=75ed4c7b6323afa32fe104e167e8f0443ee03a1071f64ed1633625bbcb4a0310 +x64openbsd=3a5f1fea8fde17ac3d28b0dbcfd6ce145019e6f166faad83393f196766d075a7 +x64netbsd=b4e2dece3ca792bd1c7ea95b62a0761fd261d2342bd450d6aaac6d9b7f1c44ca +x64musl=b4e2dece3ca792bd1c7ea95b62a0761fd261d2342bd450d6aaac6d9b7f1c44ca +x64glibc=b4e2dece3ca792bd1c7ea95b62a0761fd261d2342bd450d6aaac6d9b7f1c44ca +x64linux=b4e2dece3ca792bd1c7ea95b62a0761fd261d2342bd450d6aaac6d9b7f1c44ca +x64elf=b4e2dece3ca792bd1c7ea95b62a0761fd261d2342bd450d6aaac6d9b7f1c44ca +x64v1mac=82d64fb118f3a3ca13b7e8ab232fcb3d6eaf0de1a725f3d11445e20030b59ee3 +x64v1win=07ada194ced28237c6495ab60e995924295789c486b151d23e47d4ba517c3035 +x64v1mingw=07ada194ced28237c6495ab60e995924295789c486b151d23e47d4ba517c3035 +x64v1freebsd=75ed4c7b6323afa32fe104e167e8f0443ee03a1071f64ed1633625bbcb4a0310 +x64v1openbsd=3a5f1fea8fde17ac3d28b0dbcfd6ce145019e6f166faad83393f196766d075a7 +x64v1netbsd=b4e2dece3ca792bd1c7ea95b62a0761fd261d2342bd450d6aaac6d9b7f1c44ca +x64v1musl=b4e2dece3ca792bd1c7ea95b62a0761fd261d2342bd450d6aaac6d9b7f1c44ca +x64v1glibc=b4e2dece3ca792bd1c7ea95b62a0761fd261d2342bd450d6aaac6d9b7f1c44ca +x64v1linux=b4e2dece3ca792bd1c7ea95b62a0761fd261d2342bd450d6aaac6d9b7f1c44ca +x64v1elf=b4e2dece3ca792bd1c7ea95b62a0761fd261d2342bd450d6aaac6d9b7f1c44ca +arm64mac=85b0eb4b323af39fd3c48cf6854d3b1b8ea65ea14bfc21583e21b817981e8a91 +arm64win=231d21e5b31e5fbe484d07ae85983ea8d56889c2e8f5036ee7ac7def0a5453e7 +arm64mingw=231d21e5b31e5fbe484d07ae85983ea8d56889c2e8f5036ee7ac7def0a5453e7 +arm64linux=ce8a063be314cca6f3810ca9ac283b8f53762172c5f8f18ea7d2012faf0ca68e +arm64musl=ce8a063be314cca6f3810ca9ac283b8f53762172c5f8f18ea7d2012faf0ca68e +arm64glibc=ce8a063be314cca6f3810ca9ac283b8f53762172c5f8f18ea7d2012faf0ca68e +arm64v1win=231d21e5b31e5fbe484d07ae85983ea8d56889c2e8f5036ee7ac7def0a5453e7 +arm64v1mingw=231d21e5b31e5fbe484d07ae85983ea8d56889c2e8f5036ee7ac7def0a5453e7 +arm64v1linux=ce8a063be314cca6f3810ca9ac283b8f53762172c5f8f18ea7d2012faf0ca68e +arm64v1musl=ce8a063be314cca6f3810ca9ac283b8f53762172c5f8f18ea7d2012faf0ca68e +arm64v1glibc=ce8a063be314cca6f3810ca9ac283b8f53762172c5f8f18ea7d2012faf0ca68e arm32linux=NOT_IMPLEMENTED arm32musl=NOT_IMPLEMENTED wasm32=NOT_IMPLEMENTED From 4ff6c1dea30a48a94a7a0ea5dc948bf42dfe0f51 Mon Sep 17 00:00:00 2001 From: Richard Feldman Date: Wed, 28 Oct 2026 08:02:10 -0400 Subject: [PATCH 3/5] Run postcheck allocation-failure checks over an allocator that refuses in-place growth --- src/postcheck/monotype/lower.zig | 5 ++++- src/postcheck/monotype_lifted/spec_constr.zig | 5 ++++- 2 files changed, 8 insertions(+), 2 deletions(-) diff --git a/src/postcheck/monotype/lower.zig b/src/postcheck/monotype/lower.zig index 9162c9d321c..b60fe18288a 100644 --- a/src/postcheck/monotype/lower.zig +++ b/src/postcheck/monotype/lower.zig @@ -71503,7 +71503,10 @@ test "function context identity excludes draft local allocation ids" { test "issue 11362: checked instantiation allocates placeholders only for recursive lookups" { try testLazyCheckedInstantiationAliases(std.testing.allocator); - try std.testing.checkAllAllocationFailures(std.testing.allocator, testLazyCheckedInstantiationAliases, .{}); + // In-place growth under the testing allocator depends on the heap's + // layout, so the failure check runs over an allocator that refuses it. + var deterministic = base.DeterministicAllocator.init(std.testing.allocator); + try std.testing.checkAllAllocationFailures(deterministic.allocator(), testLazyCheckedInstantiationAliases, .{}); } fn testLazyCheckedInstantiationAliases(gpa: Allocator) (Allocator.Error || error{ TestUnexpectedResult, TestExpectedEqual })!void { diff --git a/src/postcheck/monotype_lifted/spec_constr.zig b/src/postcheck/monotype_lifted/spec_constr.zig index 1095e7986a9..f93bc6824b8 100644 --- a/src/postcheck/monotype_lifted/spec_constr.zig +++ b/src/postcheck/monotype_lifted/spec_constr.zig @@ -17761,7 +17761,10 @@ fn checkSpecConstrRequestAllocationFailure(allocator: Allocator) (Allocator.Erro } test "staged SpecConstr request copying and admission allocation failures release owners" { - try std.testing.checkAllAllocationFailures(std.testing.allocator, checkSpecConstrRequestAllocationFailure, .{}); + // In-place growth under the testing allocator depends on the heap's + // layout, so the failure check runs over an allocator that refuses it. + var deterministic = @import("base").DeterministicAllocator.init(std.testing.allocator); + try std.testing.checkAllAllocationFailures(deterministic.allocator(), checkSpecConstrRequestAllocationFailure, .{}); } test "issue 10313 value-aware call-pattern collection does not append lifted IR" { From fadac0cbb1ab254602941ca78f0ce145ba236090 Mon Sep 17 00:00:00 2001 From: Richard Feldman Date: Wed, 28 Oct 2026 17:15:34 -0400 Subject: [PATCH 4/5] Run the persistent interface allocation-failure check over an allocator that refuses in-place growth --- src/postcheck/monotype/interface_summary.zig | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/src/postcheck/monotype/interface_summary.zig b/src/postcheck/monotype/interface_summary.zig index 160decc660a..d8d687ef05a 100644 --- a/src/postcheck/monotype/interface_summary.zig +++ b/src/postcheck/monotype/interface_summary.zig @@ -1355,7 +1355,10 @@ fn allocationFailureRoundTrip(gpa: Allocator) error{ InvalidSummary, NotRepresen } test "persistent interface allocation failures roll back construction" { - try std.testing.checkAllAllocationFailures(std.testing.allocator, allocationFailureRoundTrip, .{}); + // In-place growth under the testing allocator depends on the heap's + // layout, so the failure check runs over an allocator that refuses it. + var deterministic = base.DeterministicAllocator.init(std.testing.allocator); + try std.testing.checkAllAllocationFailures(deterministic.allocator(), allocationFailureRoundTrip, .{}); } test "persistent interface rejects corrupt graph data before changing the receiving store" { From 08049f06d11b2be28c309451904e0163239c0317 Mon Sep 17 00:00:00 2001 From: Richard Feldman Date: Wed, 28 Oct 2026 18:50:54 -0400 Subject: [PATCH 5/5] Record the dev object hashes the shared recursive digest produces for float math --- test/snapshots/dev_object_float_math.md | 62 ++++++++++++------------- 1 file changed, 31 insertions(+), 31 deletions(-) diff --git a/test/snapshots/dev_object_float_math.md b/test/snapshots/dev_object_float_math.md index 7769ffc6ca0..6557e542e6a 100644 --- a/test/snapshots/dev_object_float_math.md +++ b/test/snapshots/dev_object_float_math.md @@ -51,37 +51,37 @@ main = |x, y, n| { ~~~ # DEV OUTPUT ~~~ini -x64mac=2974cd8b80f24d070b1ede8204dd7a9b4fa96ae638b1f38d342bfc4216e2b53f -x64win=38a39dbe92a9807073bcb2656d525973013c6efa3f3313b71dd39af257276693 -x64mingw=38a39dbe92a9807073bcb2656d525973013c6efa3f3313b71dd39af257276693 -x64freebsd=b163cfa13afe2e00b77ac9a89ddf9033ca6754addef8128d80cbef4e89637205 -x64openbsd=f657d0ffa61cf2b97bd8b872bc481dfa9f9a5786afb0bf692903c1a3c53f1a20 -x64netbsd=55752b4e3c2ce17f895709ee41c9a28c77ced478ef00de3cba1f2d4476b08f43 -x64musl=55752b4e3c2ce17f895709ee41c9a28c77ced478ef00de3cba1f2d4476b08f43 -x64glibc=55752b4e3c2ce17f895709ee41c9a28c77ced478ef00de3cba1f2d4476b08f43 -x64linux=55752b4e3c2ce17f895709ee41c9a28c77ced478ef00de3cba1f2d4476b08f43 -x64elf=55752b4e3c2ce17f895709ee41c9a28c77ced478ef00de3cba1f2d4476b08f43 -x64v1mac=2974cd8b80f24d070b1ede8204dd7a9b4fa96ae638b1f38d342bfc4216e2b53f -x64v1win=38a39dbe92a9807073bcb2656d525973013c6efa3f3313b71dd39af257276693 -x64v1mingw=38a39dbe92a9807073bcb2656d525973013c6efa3f3313b71dd39af257276693 -x64v1freebsd=b163cfa13afe2e00b77ac9a89ddf9033ca6754addef8128d80cbef4e89637205 -x64v1openbsd=f657d0ffa61cf2b97bd8b872bc481dfa9f9a5786afb0bf692903c1a3c53f1a20 -x64v1netbsd=55752b4e3c2ce17f895709ee41c9a28c77ced478ef00de3cba1f2d4476b08f43 -x64v1musl=55752b4e3c2ce17f895709ee41c9a28c77ced478ef00de3cba1f2d4476b08f43 -x64v1glibc=55752b4e3c2ce17f895709ee41c9a28c77ced478ef00de3cba1f2d4476b08f43 -x64v1linux=55752b4e3c2ce17f895709ee41c9a28c77ced478ef00de3cba1f2d4476b08f43 -x64v1elf=55752b4e3c2ce17f895709ee41c9a28c77ced478ef00de3cba1f2d4476b08f43 -arm64mac=2ba0801c28aae081411e76379f7e23a9c37dbfbf3a290aa0b71df10dd0b4762d -arm64win=3f167f7e302622215cdd09891a57dfe498aac576dc9ed8d0b7d5ed10b2c4005f -arm64mingw=3f167f7e302622215cdd09891a57dfe498aac576dc9ed8d0b7d5ed10b2c4005f -arm64linux=2c26e5e37d4250b70dc11c5878e3be2baa68a527be354f136b9062297c88162c -arm64musl=2c26e5e37d4250b70dc11c5878e3be2baa68a527be354f136b9062297c88162c -arm64glibc=2c26e5e37d4250b70dc11c5878e3be2baa68a527be354f136b9062297c88162c -arm64v1win=3f167f7e302622215cdd09891a57dfe498aac576dc9ed8d0b7d5ed10b2c4005f -arm64v1mingw=3f167f7e302622215cdd09891a57dfe498aac576dc9ed8d0b7d5ed10b2c4005f -arm64v1linux=2c26e5e37d4250b70dc11c5878e3be2baa68a527be354f136b9062297c88162c -arm64v1musl=2c26e5e37d4250b70dc11c5878e3be2baa68a527be354f136b9062297c88162c -arm64v1glibc=2c26e5e37d4250b70dc11c5878e3be2baa68a527be354f136b9062297c88162c +x64mac=82d64fb118f3a3ca13b7e8ab232fcb3d6eaf0de1a725f3d11445e20030b59ee3 +x64win=07ada194ced28237c6495ab60e995924295789c486b151d23e47d4ba517c3035 +x64mingw=07ada194ced28237c6495ab60e995924295789c486b151d23e47d4ba517c3035 +x64freebsd=75ed4c7b6323afa32fe104e167e8f0443ee03a1071f64ed1633625bbcb4a0310 +x64openbsd=3a5f1fea8fde17ac3d28b0dbcfd6ce145019e6f166faad83393f196766d075a7 +x64netbsd=b4e2dece3ca792bd1c7ea95b62a0761fd261d2342bd450d6aaac6d9b7f1c44ca +x64musl=b4e2dece3ca792bd1c7ea95b62a0761fd261d2342bd450d6aaac6d9b7f1c44ca +x64glibc=b4e2dece3ca792bd1c7ea95b62a0761fd261d2342bd450d6aaac6d9b7f1c44ca +x64linux=b4e2dece3ca792bd1c7ea95b62a0761fd261d2342bd450d6aaac6d9b7f1c44ca +x64elf=b4e2dece3ca792bd1c7ea95b62a0761fd261d2342bd450d6aaac6d9b7f1c44ca +x64v1mac=82d64fb118f3a3ca13b7e8ab232fcb3d6eaf0de1a725f3d11445e20030b59ee3 +x64v1win=07ada194ced28237c6495ab60e995924295789c486b151d23e47d4ba517c3035 +x64v1mingw=07ada194ced28237c6495ab60e995924295789c486b151d23e47d4ba517c3035 +x64v1freebsd=75ed4c7b6323afa32fe104e167e8f0443ee03a1071f64ed1633625bbcb4a0310 +x64v1openbsd=3a5f1fea8fde17ac3d28b0dbcfd6ce145019e6f166faad83393f196766d075a7 +x64v1netbsd=b4e2dece3ca792bd1c7ea95b62a0761fd261d2342bd450d6aaac6d9b7f1c44ca +x64v1musl=b4e2dece3ca792bd1c7ea95b62a0761fd261d2342bd450d6aaac6d9b7f1c44ca +x64v1glibc=b4e2dece3ca792bd1c7ea95b62a0761fd261d2342bd450d6aaac6d9b7f1c44ca +x64v1linux=b4e2dece3ca792bd1c7ea95b62a0761fd261d2342bd450d6aaac6d9b7f1c44ca +x64v1elf=b4e2dece3ca792bd1c7ea95b62a0761fd261d2342bd450d6aaac6d9b7f1c44ca +arm64mac=85b0eb4b323af39fd3c48cf6854d3b1b8ea65ea14bfc21583e21b817981e8a91 +arm64win=231d21e5b31e5fbe484d07ae85983ea8d56889c2e8f5036ee7ac7def0a5453e7 +arm64mingw=231d21e5b31e5fbe484d07ae85983ea8d56889c2e8f5036ee7ac7def0a5453e7 +arm64linux=ce8a063be314cca6f3810ca9ac283b8f53762172c5f8f18ea7d2012faf0ca68e +arm64musl=ce8a063be314cca6f3810ca9ac283b8f53762172c5f8f18ea7d2012faf0ca68e +arm64glibc=ce8a063be314cca6f3810ca9ac283b8f53762172c5f8f18ea7d2012faf0ca68e +arm64v1win=231d21e5b31e5fbe484d07ae85983ea8d56889c2e8f5036ee7ac7def0a5453e7 +arm64v1mingw=231d21e5b31e5fbe484d07ae85983ea8d56889c2e8f5036ee7ac7def0a5453e7 +arm64v1linux=ce8a063be314cca6f3810ca9ac283b8f53762172c5f8f18ea7d2012faf0ca68e +arm64v1musl=ce8a063be314cca6f3810ca9ac283b8f53762172c5f8f18ea7d2012faf0ca68e +arm64v1glibc=ce8a063be314cca6f3810ca9ac283b8f53762172c5f8f18ea7d2012faf0ca68e arm32linux=NOT_IMPLEMENTED arm32musl=NOT_IMPLEMENTED wasm32=NOT_IMPLEMENTED