Skip to content

2026.9.29.1: a workspace is one graph per configuration; the planning regression of 2026.9.28.3 fixed at its cause - #738

Merged
Sunrisepeak merged 11 commits into
mainfrom
feat/workspace-build-graph
Sep 28, 2026
Merged

Sunrisepeak merged 11 commits into
mainfrom
feat/workspace-build-graph

Conversation

@speak-agent

Copy link
Copy Markdown
Member

Summary

A workspace is built as one graph per configuration (design:
.agents/docs/2026-09-29-workspace-build-graph-design.md, revision 4, with the
implementation record in §17). Released as 2026.9.29.1.

  • The planning regression of 2026.9.28.3 is fixed at its cause. E1 planned a
    shared member as the root of a nested build, which met E1's condition again,
    so a chain of n members cost 2^n plans (a nine-member workspace with nothing
    to build: 79 s against 6.3 s). E1 is removed with everything that served it
    (nested plans, member stage edges, the member lock, packageKeys, e2e 830).
  • One graph per configuration. A command on a workspace selects members
    (--workspace, -p X, the member directory, the rooted package) and plans
    them under a virtual root with member edges. Members that share their
    root-position values are one plan with one build.ninja; a member used by
    several members is compiled once. Members that differ are separate plans,
    built concurrently under a static share of the jobs.
  • Product directories. A member's products are in
    target/<triple>/<configuration>/bin/<package name>/ (qualified on a name
    collision; the rooted package keeps bin/), with the runtime files and the
    graph-built shared libraries its programs load. Each member links its own
    closure: a link group holds the closure's flags and runtime contract, and the
    link edge carries the group's line.
  • The directory name is the configuration. A package's own flags, sources
    and include directories reach its commands, not the directory name; editing
    them rebuilds that package in place. build.ninja records a request tag
    (members and features) that every fast path compares.
  • Fast path per selection and configuration. The chain of §1.1: 36.8 s
    (2026.9.28.3) against 0.51 s planned and 3 ms replayed.
  • Members keep a root's duties: targets (gated by required_features),
    dev-dependencies under mcpp test, hooks, when = "dev" xlings entries,
    --features, and a build program run after every dependency's program with
    its own graph document.
  • One lock and one compile database per workspace, merged for partial
    selections; clean --stale removes the members' old build directories.
  • Linked placement of shared libraries (copies for every other file).
  • mcpp.graph (modules/graph): stable topological order with the cycle
    path, levels, closure; five hand-written algorithms migrated.
  • Version 2026.9.29.1; the xlings pin moves to 2026.9.29.1.

Refs #734. The module-name limit across members is documented and tracked by #732.

Test plan

  • mcpp build (self-host) and mcpp test: 135 unit tests; the new ones are
    WorkspacePlan.*, BuildStage linked placement, BuildProfile
    configuration key, GraphShape.TheHeaderNamesTheRequest, and
    mcpp test -p graph (22 tests).
  • New e2e 833 (one graph per configuration: compiled once, -p after
    --workspace compiles nothing, a flag edit rebuilds one member, two
    configurations, product directories, qualified names, clean --stale),
    834 (planning once, fast path in under a second), 835 (linked placement).
  • Every e2e that declares a workspace, with the tests whose expectations
    moved with the layout updated (35, 120, 211, 218, 321, 770, 806).
  • Full e2e suite locally: every failure also fails with the released
    2026.9.28.3 on the same machine (its default toolchain is llvm; the tests
    assume gcc), except the tests updated above.
  • Release canaries locally with the candidate: xlings build and test;
    mcppls build, build -p devtools and test.
  • Five package-index members through tests/run_members.sh.
  • CI on every platform.

… the configuration, linked placement, the virtual root and member edges
modules/graph provides topological_order, levels and closure over plain
index-addressed adjacency lists (std::vector<std::vector<std::size_t>>,
edge u -> v meaning "u depends on v"). Kahn's algorithm keeps its ready
set ordered, so ties break by ascending node index; a cycle is reported
as the ordered ring that walks it rather than the set of nodes left
over.

Five hand-written variants are migrated onto it, each previously a
private DFS or Kahn's-algorithm implementation with its own tie-break
and cycle-reporting shape:

- src/modgraph/graph.cppm: topo_sort's Kahn's algorithm. The cycle
  diagnostic in validate.cppm now names the ring ("a -> b -> c -> a")
  instead of an unordered list of suspects.
- src/build/prepare/features.cpp: the within-package unit order by
  imports (falls back to declaration order on a cycle, preserving the
  existing "leave it to the compiler" behaviour), and the cross-package
  host-module provider order (restricted to each consumer's own
  reachable closure before calling into the module, so a cycle outside
  that closure cannot fail an unrelated consumer's build).
- src/build/prepare/graph.cpp: step4b_cycle_check's package-cycle DFS.
- src/build/dep_graph.cppm: transitive_dependencies, on mcpp::graph::closure.
- src/build/prepare/plan.cpp: the build-cache key fold walks
  topological_order instead of recursing, turning the memoized
  recursion into a loop; the per-node fold and taint stay local to the
  caller.

modules/graph/tests covers ties, diamonds, disconnected components,
self-loops, a longer cycle (path checked exactly), levels and closure,
including the include/exclude-roots and cycle-tolerant cases.
…rectories, the workspace fast path, the scheduler, the lock, clean --stale, tests, documentation and the 2026.9.29.1 version
…e package reads its own workspace dependencies, pooled flags kept for units no member owns, the group link line checked for hermeticity, runtime files placed in the product directory
… shared helpers (file length), the design record carries its front matter, and PackageRoot holds no optional string (clang with the MSVC STL cannot copy one)
…ink order, and so Mach-O initializer order) and the host-module depth-first orders; groups sharing one build directory build one after the other; status lines written whole
…ter-free, ninja-build/ninja#2662); pack reads runtime files against the product directory; a program shipped through artifacts is linked with its package's intermediates and placed in each member's product directory; a file never takes a product directory's path
@Sunrisepeak
Sunrisepeak merged commit e7b3218 into main Sep 28, 2026
1 of 33 checks passed
Sunrisepeak added a commit to openxlings/xim-pkgindex that referenced this pull request Sep 29, 2026
mcpp 2026.9.29.1: a workspace is one graph per configuration (mcpp-community/mcpp#738).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants