Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions interscript-py/src/interscript/grammar/actions.py
Original file line number Diff line number Diff line change
Expand Up @@ -393,8 +393,8 @@ def item_list(self, v: Tree | None) -> list | None:

def rule_list(self, trees: list) -> list:
"""rule_items for parallel/sequence blocks: skip comment/noop rows;
`use` directives splice into the pass (GRAMMAR_EXTENSIONS
rule_items — the expansion resolves them in place)."""
`use` directives splice into the pass (canonical grammar
rule_line — the expansion resolves them in place, spec §3.2)."""
out: list[Any] = []
for t in trees:
h = _hash(t)
Expand Down
6 changes: 3 additions & 3 deletions interscript-py/src/interscript/grammar/grammar_source.py

Large diffs are not rendered by default.

36 changes: 15 additions & 21 deletions interscript-py/src/interscript/grammar/isc_parser.py
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@
Single-sourcing contract: the grammar TEXT embedded in grammar_source.py
is the semantic source of truth (synced from interscript-ruby's
lib/interscript/isc/grammar/isc.parg via scripts/sync_isc_grammar.py).
The four documented grammar-table extensions below (GRAMMAR_EXTENSIONS)
The three documented grammar-table extensions below (GRAMMAR_EXTENSIONS)
restore hand-parser behaviors the canonical .parg rejects — each is
spelled in the grammar language, named, and corpus-proven dormant.
"""
Expand Down Expand Up @@ -69,7 +69,7 @@ def compile_grammar() -> CompiledGrammar:
# Shape-guarded merge: if a future .parg edit renames any of the three
# rules, the merge raises instead of silently misparsing.
ext = parse_parg(GRAMMAR_EXTENSIONS)
for name in ("any_arg", "block_rule", "run_dep", "rule_items"):
for name in ("any_arg", "block_rule", "run_dep"):
if name not in rules or name not in ext.rules:
raise RuntimeError(
f"grammar extension target {name} missing from isc.parg — "
Expand Down Expand Up @@ -141,16 +141,15 @@ def _hand_style_failure(source, c, interp, peg_error) -> tuple[str, int]:
any_arg = range_arg / set_arg / alias_arg / ((item) as any_item)
block_rule = ("sub" [ws] "{" [ws] "from" whitespace ((item) as from) [ws] "to" whitespace ((item) as to) ((*(whitespace constraint)) as constraints) [ws] "}" [(when_guard)]) / ("sub" [ws] "{" [ws] "from" whitespace ((item) as from) [ws] "to" whitespace ((item) as to) ((*(whitespace constraint)) as constraints) [(when_guard)] [ws] "}")
run_dep = "map." (((identifier *("-" identifier)) / identifier) as dep) ".stage." (identifier as stage)
rule_items = *(([ws] ((use_directive / rule / comment_item)) [ws]))
}
"""

# GRAMMAR_EXTENSIONS — the places where the former hand parser accepted
# syntax the canonical .parg rejects. Each is spelled in the grammar
# language itself and merged into the rule table, so the .parg remains
# the only semantic source and every extension is one visible, named
# block (the first three are the same extensions the ts generated
# parser ships):
# block (all three are the same extensions the ts generated parser
# ships):
#
# 1. `any_arg` — restore the ruby DSL's final `item as :any_item`
# fallback (concerns/items.rb any_constructor), which the canonical
Expand All @@ -167,20 +166,15 @@ def _hand_style_failure(source, c, interp, peg_error) -> tuple[str, int]:
# parser's identifier admitted `-` in structural positions. Only the
# dependency slot is widened; the corpus is hyphen-free in every
# identifier position (proven by the corpus parity driver).
# 4. `rule_items` — accept `use` directives inside parallel/sequence
# blocks (ISC-WF-1 §3: a ruleset can be spliced into one pass). The
# canonical .parg only feeds rule_line (sub-rows/comments) to pass
# blocks; the hand parser accepted `use` there and the py suite pins
# it (test_isc_parser.py::test_use_inside_parallel_participates_in_pass).
# `use` shadows nothing in rule position (rule always starts with
# "sub", comment with "#"). The override block's rule_line is left
# canonical — the hand parser silently DROPPED non-sub rows there,
# so a `use` inside `override {}` is a generated-parser rejection of
# a construct the corpus never uses.
# The former fourth extension (`rule_items` accepting `use` inside
# parallel/sequence blocks) was RETIRED by WO18 G5: the canonical .parg
# now carries use_directive in rule_line (spec §3.2, mirroring the rs
# grammar), so the extension is redundant and its merge-guard slot is
# gone. The override block's rule_line accepts `use` rows for free —
# the expansion flattens them (isc._Expander._expand_override_rows).
#
# Ruby rejects (1)'s absence aside, the outside-when, hyphen-dep, and
# use-in-pass forms are cross-runtime divergences (ruby parses none);
# aligning ruby/ts/py on one placement is an owner decision. None of the
# four extensions fires on any corpus map (proven: the corpus py-vs-py
# document-diff driver passes byte-identical with them in place, and a
# corpus-wide scan finds no `use` inside a pass block).
# Ruby rejects (1)'s absence aside, the outside-when and hyphen-dep
# forms are cross-runtime divergences (ruby parses none); aligning
# ruby/ts/py on one placement is an owner decision. None of the three
# extensions fires on any corpus map (proven: the corpus py-vs-py
# document-diff driver passes byte-identical with them in place).
78 changes: 65 additions & 13 deletions interscript-py/src/interscript/isc.py
Original file line number Diff line number Diff line change
Expand Up @@ -45,9 +45,10 @@
corpus-dormant; see grammar/actions.py for the full deviation list.
- `languages` is declared once per map; a second declaration is a parse
error. `use` may appear in stage bodies, ruleset bodies and inside
`parallel`/`sequence` blocks; a ruleset whose expansion contains a
pass block may not itself be used inside a pass block (a nested pass
has no meaning in the one-simultaneous-pass model).
`parallel`/`sequence` blocks (WO18 G5, spec §3.2): inside a pass block
the expansion flattens the referenced ruleset's rules into the
enclosing collection, and a referenced ruleset carrying non-rule
items (funcalls/stage runs) is a named load error.

Performance note (measured, same machine): the generated parser parses
the full 434-file corpus in ~86s where the hand parser took ~7s — the
Expand Down Expand Up @@ -219,11 +220,6 @@ def expand_items(
if isinstance(it, RawUse):
out.extend(self.expand_use(it, chain, in_block))
continue
if in_block and isinstance(it, (ParallelBlock, SequenceBlock)):
self.error(
"a ruleset containing a parallel/sequence block cannot be "
"used inside a parallel/sequence block"
)
if isinstance(it, (ParallelBlock, SequenceBlock)):
it = type(it)(rules=tuple(self.expand_items(list(it.rules), chain, True)))
out.append(it)
Expand All @@ -237,16 +233,72 @@ def expand_use(
else:
expanded = self.expand_ruleset(use.name, chain)

if in_block and any(isinstance(x, (ParallelBlock, SequenceBlock)) for x in expanded):
self.error(
"a ruleset containing a parallel/sequence block cannot be used "
"inside a parallel/sequence block"
)
if in_block:
# WO18 G5 — spec §3.2: items inside a parallel/sequence block
# participate in that block's single pass; the expansion
# inherits the enclosing pass structure — the target's rule
# content flattens into the enclosing collection (interscript-rs
# resolve_use_rules). Overrides merge per-rule on the flat list.
rules = self._flatten_pass_items(expanded, chain)
if use.override is not None:
rules = self._merge_override_rules(
rules, self._expand_override_rows(use.override, chain)
)
return rules

if use.override is not None:
expanded = self._apply_override(expanded, list(use.override))
return list(expanded)

def _flatten_pass_items(
self, items: list[Any], chain: tuple[tuple[str, str], ...]
) -> list[Any]:
"""WO18 G5: flatten a structural expansion into the ordered rule
list of an enclosing pass (§3.2). Pass blocks contribute their
(already-expanded) rules in position; raw uses inside them were
flattened during expand_items. Non-rule items (funcalls, stage
runs) cannot join a pass — named load error."""
out: list[Any] = []
for it in items:
if isinstance(it, (ParallelBlock, SequenceBlock)):
out.extend(it.rules)
elif isinstance(it, ParsedRule):
out.append(it)
elif isinstance(it, RawUse):
out.extend(self.expand_use(it, chain, True))
else:
self.error(
"use inside a parallel/sequence block: the referenced ruleset "
"carries non-rule items (funcalls/stage runs) that cannot join "
"the pass"
)
return out

def _expand_override_rows(
self, rows: tuple[ParsedRule, ...], chain: tuple[tuple[str, str], ...]
) -> list[ParsedRule]:
"""WO18 G5: an override block is a rule_line list, so its rows may
themselves be `use` directives (rs extract_rule_list recursion);
their content flattens into the override row list."""
out: list[ParsedRule] = []
for row in rows:
if isinstance(row, RawUse):
out.extend(self.expand_use(row, chain, True))
else:
out.append(row)
return out

def _merge_override_rules(
self, rules: list[Any], overrides: list[ParsedRule]
) -> list[Any]:
"""§4.2 child-wins merge over a FLAT rule list (the block-position
use): matching rules drop wherever they sit; the override rows land
at the expansion tail. No trailing-parallel join here — the caller
re-wraps the list into the enclosing block's structure."""
o_keys = {self._from_key(o) for o in overrides}
kept = [r for r in rules if self._from_key(r) not in o_keys]
return kept + list(overrides)

def expand_via_dependency(
self, use: RawUse, chain: tuple[tuple[str, str], ...]
) -> list[Any]:
Expand Down
58 changes: 55 additions & 3 deletions interscript-py/tests/test_isc_parser.py
Original file line number Diff line number Diff line change
Expand Up @@ -310,13 +310,65 @@ def test_unknown_ruleset_error(self):
with pytest.raises(IscParseError, match="unknown ruleset"):
parse('system "s" { stage main { use nope } }')

def test_block_in_block_error(self):
with pytest.raises(IscParseError, match="parallel/sequence block"):
# WO18 G5 — §3.2: items inside a pass block participate in that
# block's single pass; a ruleset containing a parallel block
# contributes that block as one pass at the expansion position.
# The expansion flattens the referenced ruleset's rules into the
# enclosing collection; non-rule targets are a named load error.

def test_use_in_sequence_flattens_in_position(self):
# mirrors maps probe tst-use-inseq (rs expects "12")
doc = parse(
'system "T:tst-use-inseq:1" { ruleset r1 { parallel { sub "a" "1" } } '
'stage main { sequence { use r1 sub "b" "2" } } }'
)
cm = isc_to_compiled_map(doc)
assert execute_stage(cm, "main", "ab") == "12"

def test_use_in_parallel_joins_the_pass(self):
doc = parse(
'system "T:tst-use-inpar:1" { ruleset r1 { parallel { sub "a" "1" } } '
'stage main { parallel { use r1 sub "b" "2" } } }'
)
cm = isc_to_compiled_map(doc)
# one simultaneous pass: a→1 and b→2 together
assert execute_stage(cm, "main", "ab") == "12"

def test_ruleset_with_parallel_block_is_one_pass_in_sequence(self):
doc = parse(
'system "s" { ruleset tbl { parallel { sub "a" "b" } sub "c" "d" } '
'stage main { sequence { use tbl sub "e" "E" } } }'
)
# the used ruleset's rules splice flat, in position (spec §3.2)
seq = doc.stages[0].items[0]
assert [row(r)[:2] for r in seq.rules] == [("a", "b"), ("c", "d"), ("e", "E")]

def test_override_in_block_merges_per_rule(self):
doc = parse(
'system "s" { ruleset tbl { parallel { sub "a" "A1" sub "b" "B" } } '
'stage main { parallel { use tbl override { sub "a" "A2" } } } }'
)
cm = isc_to_compiled_map(doc)
# child-wins replacement on the flat row list, all inside one pass
assert execute_stage(cm, "main", "xa xb") == "xA2 xB"

def test_non_rule_target_in_block_is_named_error(self):
with pytest.raises(IscParseError, match="cannot join the pass"):
parse(
'system "s" { ruleset tbl { parallel { sub "a" "b" } } '
'system "s" { ruleset tbl { sub "a" "A" upcase } '
'stage main { parallel { use tbl } } }'
)

def test_non_rule_target_through_nested_use_is_named_error(self):
# the funcall sits in the indirectly referenced ruleset; the
# flattening reaches it through one use-of-a-use hop
with pytest.raises(IscParseError, match="cannot join the pass"):
parse(
'system "s" { ruleset tbl { sub "a" "A" upcase } '
'ruleset wrap { use tbl } '
'stage main { sequence { use wrap } } }'
)

def test_cross_dependency_use(self):
base = parse('system "base" { ruleset tbl { sub "a" "A" sub "b" "B" } }')
doc = parse(
Expand Down
19 changes: 15 additions & 4 deletions lib/interscript/isc/document_builder.rb
Original file line number Diff line number Diff line change
Expand Up @@ -279,9 +279,9 @@ def extract_ruleset(item)
def extract_stage_items(n)
return [] unless n.is_a?(Hash)
if n[:sequence]
[{kind: :sequence, rules: filter_noop(Array(n[:sequence]).map { |r| extract_rule(r) })}]
[{kind: :sequence, rules: filter_noop(Array(n[:sequence]).map { |r| extract_stage_row(r) })}]
elsif n[:parallel]
[{kind: :parallel, rules: filter_noop(Array(n[:parallel]).map { |r| extract_rule(r) })}]
[{kind: :parallel, rules: filter_noop(Array(n[:parallel]).map { |r| extract_stage_row(r) })}]
elsif n[:separate]
[{kind: :separate, separator: n[:separator] ? materialize(n[:separator]) : nil}]
elsif n[:compose]
Expand Down Expand Up @@ -314,15 +314,26 @@ def extract_stage_items(n)
end
end

# WO18 G5: one rule-line entry of a parallel/sequence block — a
# rule def, or (since §3.2) a `use` directive splicing into the
# block's rule list.
def extract_stage_row(r)
return extract_use(r[:use]) if r.is_a?(Hash) && r.key?(:use)
extract_rule(r)
end

# `use <name>` (same-map ruleset) or `use <dep>.<name>` (dependency-
# declared ruleset), plus the optional override rows (W2).
def extract_use(u)
dep = u[:qualified_name] ? ident(u[:use_alias]) : nil
name = ident(u[:qualified_name] || u[:use_alias])
item = {kind: :use, ruleset: name, dependency: dep}
# WO18 G5: override rows are rule_line items, so they may
# themselves be `use` directives — carried as nested use defs and
# flattened at expansion time (rs extract_rule_list recursion).
overrides = Array(u[:use_overrides])
.select { |r| r.is_a?(Hash) && r.key?(:from) }
.map { |r| extract_rule(r) }
.select { |r| r.is_a?(Hash) && (r.key?(:from) || r.key?(:use)) }
.map { |r| r.key?(:use) ? extract_use(r[:use]) : extract_rule(r) }
item[:overrides] = overrides if overrides.any?
item
end
Expand Down
5 changes: 4 additions & 1 deletion lib/interscript/isc/grammar/concerns/stages.rb
Original file line number Diff line number Diff line change
Expand Up @@ -121,7 +121,10 @@ module Stages
end

rule(:rule_line) do
whitespace? >> (rule | comment_item) >> whitespace?
# ISC-WF-1 #3.2: a `use` directive is a rule-line item - items
# inside a parallel/sequence block participate in that block's
# single pass (mirrors rs grammar/isc.parg rule_line).
whitespace? >> (rule | use_directive | comment_item) >> whitespace?
end

# A rule is either compact form (single line) or block form
Expand Down
6 changes: 5 additions & 1 deletion lib/interscript/isc/grammar/isc.parg
Original file line number Diff line number Diff line change
Expand Up @@ -153,7 +153,11 @@ grammar InterscriptIsc version "0.1.0" {
rule = block_rule / compact_rule

comment_item = (("#" *(!%x0A chr)) as comment) / ((!(keyword) identifier) as noop)
rule_line = [ws] (rule / comment_item) [ws]
# ISC-WF-1 §3.2: a `use` directive is a rule-line item — items inside a
# parallel/sequence block participate in that block's single pass, and
# the expansion flattens the referenced ruleset's rules at this
# position (mirrors interscript-rs grammar/isc.parg rule_line).
rule_line = [ws] (rule / use_directive / comment_item) [ws]
rule_items = *(rule_line)
sequence_block = "sequence" [ws] "{" [ws] (rule_items as sequence) [ws] "}"
parallel_block = "parallel" [ws] "{" [ws] (rule_items as parallel) [ws] "}"
Expand Down
Loading
Loading