Skip to content

Add emitter-diff: language-agnostic cross-version generated-code diffing - #4784

Closed
Libba Lawrence (l0lawrence) wants to merge 18 commits into
mainfrom
l0lawrence-emitter-diff-test-tool
Closed

Add emitter-diff: language-agnostic cross-version generated-code diffing#4784
Libba Lawrence (l0lawrence) wants to merge 18 commits into
mainfrom
l0lawrence-emitter-diff-test-tool

Conversation

@l0lawrence

@l0lawrence Libba Lawrence (l0lawrence) commented Jun 30, 2026 •
edited
Loading

Copy link
Copy Markdown
Member

Summary

Adds eng/emitter-diff, a language-agnostic CLI for diffing the generated-code output of two TypeSpec emitter versions (baseline vs. head), plus CI workflows that surface the diff on PRs. The goal is a shared way for every language emitter that lives (or will live) in this repo to regenerate their code-gen tests and review cross-version diffs — locally and in CI.

Each emitter version can come from a published npm version, a local folder, or a GitHub url/sha. The tool drives each emitter's own generate command (it never reimplements codegen), diffs the two outputs, and renders the result as HTML, a terminal patch, or a .patch file. For typespec-python this can replace npm run regenerate for diffing purposes (both are kept for now).

What's included

  • Core tool (eng/emitter-diff/) — zero runtime deps (Node built-ins run the .ts directly; the HTML renderer is dependency-free). A pluggable EmitterAdapter contract keeps all language specifics behind per-language adapters; the CLI itself contains zero language logic. Baseline/head/specs refs are resolved uniformly (npm: / local: / github: / gh:).

  • Adapters (eng/emitter-diff/src/adapters/):

    • python (@azure-tools/typespec-python) — wraps the package's own regenerate.ts, driven with --pluginDir (emitter build) + --generatedFolder (isolated output root).
    • typescript (@azure-tools/typespec-ts) — wraps the package's test/commands/gen-spector.js; adds --emitter-dir / --output-dir override hooks to run.ts + gen-spector.js so any emitter build can target an isolated output dir. Runs only the client phase (the generated sources the diff compares) and stages specs once via copy:typespec.
    • rust (@azure-tools/typespec-rust) — inert-until-present: typespec-rust hasn't moved into this repo yet, so the adapter fails fast with guidance. It's ready to activate once the package lands and its .scripts/tspcompile.js gains the same --emitter-dir / --output-dir hooks (documented in the adapter).

  • Output modes — default writes a clickable HTML report + file:// link; --terminal prints the colored patch; --patch <file> / --html <file> write to disk; --fail-on-diff gates in CI (exit 2 = diff present, 1 = hard error). --name scopes which specs generate; parallel by default, --sequential to serialize.

  • CI — one informational workflow per language:

    • .github/workflows/ci-emitter-diff-python.yml
    • .github/workflows/ci-emitter-diff-typescript.yml

    On a matching PR each builds the emitter, diffs the current checkout against the merge-base baseline (the base-branch commit the PR is based on — survives squash/rebase, unlike a pinned SHA), uploads the rendered-diff artifact, and posts a sticky PR comment reusing the tool's own Diff summary: line. These checks are informational: they never fail on a diff, only on a real tool/build error.

Verification

  • python no-diff sanity check (current checkout vs itself) → ✓ No differences.
  • typescript cross-version diff (npm:0.54.2 vs npm:0.55.0-dev.8, --name routes) → 5 file(s), +2 / -28 — correctly captured the emitter dropping the empty models module and bumping the node engine >=20 → >=22. HTML + patch rendered.
  • rust guard verified — fails fast with a clear message because packages/typespec-rust is absent.

Notes

  • The two repo copies of the tool target different emitters: this PR targets the Azure emitters (@azure-tools/typespec-python / -ts / -rust); the microsoft/typespec copy targets @typespec/http-client-python. The typescript + rust adapters are Azure-only.
  • The python adapter can run codegen under Pyodide (WASM) via --opt pyodide=true where a native toolchain is unavailable.

Opening as draft for feedback on the approach and adapter contract.

@l0lawrence
Add emitter-diff tool for cross-version generated-code diffing
7d0c9f8 
Adds eng/emitter-diff, a language-agnostic CLI that generates code with two
emitter versions (npm/local/github refs) and diffs the output (terminal, VS
Code, or HTML). Includes a typespec-python adapter wrapping regenerate.ts and
run-tests.ts, a python PR CI workflow, and --emitter-dir/--output-dir/
--http-specs-dir/--azure-specs-dir/--use-pyodide overrides on regenerate.ts.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@l0lawrence
emitter-diff: produce a real VS Code source diff via git overlay
5200616 
VS Code has no folder-diff CLI, so --open now stages the comparison as a
throwaway git working tree (<work-dir>/vscode-diff): baseline is committed and
head is overlaid + staged, so the Source Control panel shows every changed
generated file with side-by-side diffs. Documents the flag and the equivalent
manual command in the README.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@l0lawrence
emitter-diff: add --patch to write the diff to a file; silence DEP0190
8e77116 
--patch <file> writes the raw unified diff to a file (printing only the summary
to the terminal) instead of dumping the full patch. Also build a single quoted
command line for shell-routed spawns so Node's DEP0190 (args array + shell:true)
warning no longer prints when launching code/npm shims on Windows.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@l0lawrence
emitter-diff: default to clickable HTML report; --vscode opt-in; pyod…
3901bc5 
…ide default for python on Windows

Output presentation:
- Default output is now a clickable HTML report written to <work-dir>/emitter-diff.html,
  with a file:// link printed to the terminal.
- --vscode opens the live VS Code source diff (--open kept as an alias).
- --terminal restores the full colored patch dump; --patch/--html still target files.

Python adapter:
- Default --use-pyodide on Windows (native black fails under deep pnpm venv paths)
  and off elsewhere (Linux CI native is faster). Override with --opt pyodide=true|false.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@l0lawrence
emitter-diff: fix --vscode missing same-size file modifications
1f8e56d 
openInVsCode committed the baseline tree then overlaid head and staged with a
plain 'git add -A'. Git decides a file is unchanged from (size, mtime) and skips
re-hashing; since the baseline commit and head overlay are written in the same
second, a same-size content edit was treated as clean and never staged, so it
silently did not appear in VS Code (adds/deletes/size-changing edits still did).
Clear the index (git rm -r --cached) before re-adding so git re-hashes every
file's content and surfaces every real modification.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@l0lawrence
emitter-diff(ci): post a sticky PR comment with the diff summary
49bbf91 
The python emitter-diff workflow already uploaded the rendered HTML as a per-PR
artifact and wrote a job summary; add a sticky PR comment (updated in place via a
hidden marker) that lists the changed files and +/- counts and links to the
artifact download. Also emit --patch so the comment parses exact file/line counts
straight from the unified diff, grant pull-requests: write, and mark the HTML
upload if-no-files-found: ignore for the no-diff case. Fork PRs get a read-only
token so the comment is best-effort (continue-on-error).

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@microsoft-github-policy-service microsoft-github-policy-service Bot added eng emitter:python Issues for @azure-tools/typespec-python emitter labels Jun 30, 2026
@l0lawrence
Merge remote-tracking branch 'upstream/main' into l0lawrence-emitter-…
089912d 
…diff-test-tool

# Conflicts:
#	packages/typespec-python/eng/scripts/ci/regenerate-common.ts
#	pnpm-lock.yaml
@l0lawrence
Re-integrate python adapter with post-merge ci/regenerate
8cc03e1 
Upstream PR #4399 moved/rewrote the python regenerate driver to
eng/scripts/ci/regenerate.ts (in-process compilation). Re-apply the
emitter-diff overrides on the new wrapper and point the adapter at it:

- regenerate.ts: add --httpSpecsDir/--azureSpecsDir/--use-pyodide/--no-baseline
  options; thread use-pyodide into emitter task options; gate the
  azure-sdk-for-python baseline clone behind --no-baseline. All overrides live
  in the non-synced wrapper so regenerate-common.ts stays identical upstream.
- python adapter: drive eng/scripts/ci/regenerate.ts with --pluginDir
  (emitter build) + --generatedFolder (output root, anchored so files land in
  <outputDir>/tests/generated); pin specs to the current checkout so only the
  emitter varies; pass --no-baseline. Update runTests sync source for the new
  nested layout.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@azure-sdk-automation

azure-sdk-automation Bot commented Jun 30, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

❌ There is undocummented changes. Run chronus add to add a changeset or click here.

The following packages have changes but are not documented.

  • ❌@azure-tools/typespec-ts

The following packages have already been documented:

  • ✅ @azure-tools/typespec-python
Show changes

@azure-tools/typespec-python - internal ✏️

Add optional --emitter-dir, --output-dir, --http-specs-dir, --azure-specs-dir and --use-pyodide overrides to the regenerate script (defaults preserve existing behavior). These let the new repo-level emitter-diff tool drive generation with an arbitrary emitter build and output location so it can diff generated code across emitter versions; --use-pyodide forces WASM generation for environments where the native Python toolchain (e.g. the black formatter) cannot run.

@github-actions

github-actions Bot commented Jun 30, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Python emitter diff

Baseline 5c8a2d875f98051d61bb786ff3b30d212294a0bf vs this PR.

No changes to generated output.

Full rendered diff: emitter-diff-html artifact on the run.

Informational check (eng/emitter-diff); does not block the PR.

@l0lawrence
emitter-diff: use .js import specifiers, run via tsx
5946e8b 
The eng/emitter-diff sources imported sibling modules with explicit .ts
extensions, which only typechecks under allowImportingTsExtensions. The
repo-wide check:eng (tsc -p tsconfig.eng.json) compiles ./eng without
that flag, so the build failed with TS5097.

Switch to the repo's standard convention instead of enabling the flag:
- import sibling modules with .js specifiers (NodeNext resolves them to
  the .ts sources during typecheck; tsx/esbuild resolves them at runtime).
- drop allowImportingTsExtensions from the package tsconfig.
- run the CLI via 'npx tsx' (workflow, README, shebang) since node's
  type-stripping does not remap .js to .ts.

Runtime script-path arguments (eng/scripts/ci/regenerate.ts etc.) are
left as .ts since they are passed to tsx, not imported.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@l0lawrence
emitter-diff: satisfy spell-check and catalog consistency
2e9faf9 
- cspell: allow 'gpgsign' (a git config key used in diff.ts overlay).
- versions consistency: move diff2html to the pnpm catalog instead of a
  pinned version; add diff2html (^3.4.48) to the workspace catalog and
  reference it via 'catalog:' from eng/emitter-diff, updating the lockfile.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@pkg-pr-new

pkg-pr-new Bot commented Jun 30, 2026 •
edited
Loading

Copy link
Copy Markdown

Open in StackBlitz

npm i https://pkg.pr.new/@azure-tools/typespec-python@4784

npm i https://pkg.pr.new/@azure-tools/typespec-ts@4784

commit: 44b2205

@azure-sdk-automation

azure-sdk-automation Bot commented Jun 30, 2026

Copy link
Copy Markdown
Contributor

You can try these changes here

🛝 Playground 🌐 Website

@github-actions

github-actions Bot commented Jun 30, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

⚡ Benchmark Results

⚠️ 33 metric(s) regressed above the +5% threshold:

Metric Baseline Current Change
total 🔴 729.1ms 🔴 1.06s +45.2% 🔴
loader 🟡 209.3ms 🟡 326.7ms +56.1% 🔴
resolver 🟢 23.1ms 🟢 41.0ms +77.8% 🔴
checker 🟡 235.2ms 🟡 379.4ms +61.3% 🔴
validation 🟢 53.9ms 🟢 87.4ms +62.1% 🔴
 ↳ validation/@azure-tools/typespec-azure-core 🟢 7.9ms 🟡 13.1ms +66.9% 🔴
 ↳ validation/@typespec/http 🟢 7.9ms 🟡 16.2ms +105.7% 🔴
 ↳ validation/@typespec/versioning 🔴 34.7ms 🔴 52.3ms +51.0% 🔴
 ↳ validation/compiler 🟢 2.2ms 🟢 3.9ms +81.1% 🔴
linter 🟢 161.0ms 🟡 223.6ms +38.9% 🔴
 ↳ linter/@azure-tools/typespec-azure-core/byos 🟢 6.6ms 🟢 8.5ms +29.7% 🔴
 ↳ linter/@azure-tools/typespec-azure-core/no-header-explode 🔴 21.3ms 🔴 28.2ms +32.2% 🔴
 ↳ linter/@azure-tools/typespec-azure-core/no-query-explode 🔴 22.6ms 🔴 29.2ms +29.2% 🔴
 ↳ linter/@azure-tools/typespec-azure-core/no-response-body 🔴 26.5ms 🔴 34.9ms +31.7% 🔴
 ↳ linter/@azure-tools/typespec-azure-core/no-route-parameter-name-mismatch 🟢 5.8ms 🟢 8.1ms +40.8% 🔴
 ↳ linter/@azure-tools/typespec-azure-core/response-schema-problem 🔴 24.9ms 🔴 34.5ms +38.7% 🔴
 ↳ linter/@azure-tools/typespec-azure-core/use-standard-names 🟢 6.1ms 🟢 7.9ms +29.6% 🔴
 ↳ linter/@azure-tools/typespec-azure-resource-manager/arm-agent-base-type-child-resources 🟢 7.5ms 🔴 21.1ms +180.3% 🔴
 ↳ linter/@azure-tools/typespec-azure-resource-manager/arm-common-types-version 🟢 5.9ms 🟡 12.2ms +107.4% 🔴
 ↳ linter/@azure-tools/typespec-azure-resource-manager/arm-delete-operation-response-codes 🟢 2.3ms 🟢 4.5ms +99.9% 🔴
 ↳ linter/@azure-tools/typespec-azure-resource-manager/arm-no-path-casing-conflicts 🟡 12.6ms 🟡 16.8ms +34.2% 🔴
 ↳ linter/@azure-tools/typespec-azure-resource-manager/arm-resource-operation-response 🟢 5.9ms 🟢 9.5ms +61.4% 🔴
 ↳ linter/@azure-tools/typespec-azure-resource-manager/lro-location-header 🟡 15.4ms 🔴 21.1ms +37.3% 🔴
 ↳ linter/@azure-tools/typespec-azure-resource-manager/no-response-body 🔴 22.6ms 🔴 32.4ms +43.2% 🔴
 ↳ linter/@azure-tools/typespec-client-generator-core/property-name-conflict 🟢 1.4ms 🟢 2.4ms +74.3% 🔴
 ↳ linter/@azure-tools/typespec-client-generator-core/require-client-suffix 🟢 0.5ms 🟢 1.7ms +228.4% 🔴
emit 🔴 5.63s 🔴 6.20s +10.1% 🔴
 ↳ emit/@azure-tools/typespec-autorest 🟢 91.0ms 🟢 127.8ms +40.5% 🔴
 ↳ emit/@azure-tools/typespec-python 🔴 2.15s 🔴 2.31s +7.4% 🔴
 ↳ emit/@typespec/http-client-js 🔴 487.7ms 🔴 589.5ms +20.9% 🔴
 ↳ emit/@typespec/openapi3 🟢 81.8ms 🟢 111.0ms +35.7% 🔴
 ↳ emit/@typespec/openapi3/compute 🟢 71.3ms 🟢 96.3ms +35.1% 🔴
 ↳ emit/@typespec/openapi3/write 🟢 10.3ms 🟢 14.4ms +39.6% 🔴
Full details – comparing 3d90464 vs baseline rolling-baseline-03c0619-5c8a2d8 (rolling baseline (20 main runs))
Metric Baseline Current Change
total 🔴 729.1ms 🔴 1.06s +45.2% 🔴
loader 🟡 209.3ms 🟡 326.7ms +56.1% 🔴
resolver 🟢 23.1ms 🟢 41.0ms +77.8% 🔴
checker 🟡 235.2ms 🟡 379.4ms +61.3% 🔴
validation 🟢 53.9ms 🟢 87.4ms +62.1% 🔴
 ↳ validation/@azure-tools/typespec-azure-core 🟢 7.9ms 🟡 13.1ms +66.9% 🔴
 ↳ validation/@typespec/http 🟢 7.9ms 🟡 16.2ms +105.7% 🔴
 ↳ validation/@typespec/rest 🟢 0.8ms 🟢 1.7ms +100.6%
 ↳ validation/@typespec/versioning 🔴 34.7ms 🔴 52.3ms +51.0% 🔴
 ↳ validation/compiler 🟢 2.2ms 🟢 3.9ms +81.1% 🔴
linter 🟢 161.0ms 🟡 223.6ms +38.9% 🔴
 ↳ linter/@azure-tools/typespec-azure-core/auth-required 🟢 0.0ms 🟢 0.1ms +200.7%
 ↳ linter/@azure-tools/typespec-azure-core/bad-record-type 🟢 0.3ms 🟢 0.6ms +94.7%
 ↳ linter/@azure-tools/typespec-azure-core/byos 🟢 6.6ms 🟢 8.5ms +29.7% 🔴
 ↳ linter/@azure-tools/typespec-azure-core/casing-style 🟢 0.8ms 🟢 1.5ms +84.6%
 ↳ linter/@azure-tools/typespec-azure-core/composition-over-inheritance 🟢 0.1ms 🟢 0.2ms +97.4%
 ↳ linter/@azure-tools/typespec-azure-core/documentation-required 🟢 1.1ms 🟢 2.0ms +82.6%
 ↳ linter/@azure-tools/typespec-azure-core/friendly-name 🟢 0.8ms 🟢 1.4ms +81.5%
 ↳ linter/@azure-tools/typespec-azure-core/key-visibility-required 🟢 0.2ms 🟢 0.4ms +82.3%
 ↳ linter/@azure-tools/typespec-azure-core/known-encoding 🟢 0.3ms 🟢 0.5ms +54.3%
 ↳ linter/@azure-tools/typespec-azure-core/long-running-polling-operation-required 🟢 0.4ms 🟢 0.8ms +103.7%
 ↳ linter/@azure-tools/typespec-azure-core/no-case-mismatch 🟢 0.3ms 🟢 0.7ms +108.0%
 ↳ linter/@azure-tools/typespec-azure-core/no-closed-literal-union 🟢 0.4ms 🟢 1.0ms +131.8%
 ↳ linter/@azure-tools/typespec-azure-core/no-enum 🟢 0.1ms 🟢 0.3ms +191.9%
 ↳ linter/@azure-tools/typespec-azure-core/no-error-status-codes 🟢 0.1ms 🟢 0.3ms +106.8%
 ↳ linter/@azure-tools/typespec-azure-core/no-explicit-routes-resource-ops 🟢 0.1ms 🟢 0.2ms +55.3%
 ↳ linter/@azure-tools/typespec-azure-core/no-format 🟢 0.6ms 🟢 0.8ms +33.1%
 ↳ linter/@azure-tools/typespec-azure-core/no-generic-numeric 🟢 0.5ms 🟢 0.8ms +59.5%
 ↳ linter/@azure-tools/typespec-azure-core/no-header-explode 🔴 21.3ms 🔴 28.2ms +32.2% 🔴
 ↳ linter/@azure-tools/typespec-azure-core/no-legacy-usage 🟢 1.3ms 🟢 2.0ms +53.1%
 ↳ linter/@azure-tools/typespec-azure-core/no-multiple-discriminator 🟢 0.1ms 🟢 0.3ms +120.0%
 ↳ linter/@azure-tools/typespec-azure-core/no-nullable 🟢 0.3ms 🟢 0.4ms +47.2%
 ↳ linter/@azure-tools/typespec-azure-core/no-offsetdatetime 🟢 1.4ms 🟢 2.1ms +49.6%
 ↳ linter/@azure-tools/typespec-azure-core/no-openapi 🟢 2.1ms 🟢 2.3ms +11.7%
 ↳ linter/@azure-tools/typespec-azure-core/no-private-usage 🟢 2.2ms 🟢 3.1ms +40.4%
 ↳ linter/@azure-tools/typespec-azure-core/no-query-explode 🔴 22.6ms 🔴 29.2ms +29.2% 🔴
 ↳ linter/@azure-tools/typespec-azure-core/no-response-body 🔴 26.5ms 🔴 34.9ms +31.7% 🔴
 ↳ linter/@azure-tools/typespec-azure-core/no-rest-library-interfaces 🟢 0.0ms 🟢 0.1ms +178.7%
 ↳ linter/@azure-tools/typespec-azure-core/no-route-parameter-name-mismatch 🟢 5.8ms 🟢 8.1ms +40.8% 🔴
 ↳ linter/@azure-tools/typespec-azure-core/no-rpc-path-params 🟢 0.2ms 🟢 0.4ms +73.5%
 ↳ linter/@azure-tools/typespec-azure-core/no-string-discriminator 🟢 0.1ms 🟢 0.2ms +129.0%
 ↳ linter/@azure-tools/typespec-azure-core/no-unknown 🟢 0.2ms 🟢 0.4ms +57.0%
 ↳ linter/@azure-tools/typespec-azure-core/no-unnamed-union 🟢 0.5ms 🟢 0.8ms +76.0%
 ↳ linter/@azure-tools/typespec-azure-core/operation-missing-api-version 🟢 0.2ms 🟢 0.4ms +67.5%
 ↳ linter/@azure-tools/typespec-azure-core/request-body-problem 🟢 0.3ms 🟢 0.5ms +36.5%
 ↳ linter/@azure-tools/typespec-azure-core/require-versioned 🟢 0.0ms 🟢 0.1ms +228.4%
 ↳ linter/@azure-tools/typespec-azure-core/response-schema-problem 🔴 24.9ms 🔴 34.5ms +38.7% 🔴
 ↳ linter/@azure-tools/typespec-azure-core/rpc-operation-request-body 🟢 0.4ms 🟢 0.7ms +86.0%
 ↳ linter/@azure-tools/typespec-azure-core/spread-discriminated-model 🟢 0.3ms 🟢 0.5ms +51.7%
 ↳ linter/@azure-tools/typespec-azure-core/use-standard-names 🟢 6.1ms 🟢 7.9ms +29.6% 🔴
 ↳ linter/@azure-tools/typespec-azure-core/use-standard-operations 🟢 0.2ms 🟢 0.3ms +66.5%
 ↳ linter/@azure-tools/typespec-azure-resource-manager/arm-agent-base-type-child-resources 🟢 7.5ms 🔴 21.1ms +180.3% 🔴
 ↳ linter/@azure-tools/typespec-azure-resource-manager/arm-agent-base-type-lifecycle-operations 🟢 0.1ms 🟢 0.2ms +330.6%
 ↳ linter/@azure-tools/typespec-azure-resource-manager/arm-common-types-version 🟢 5.9ms 🟡 12.2ms +107.4% 🔴
 ↳ linter/@azure-tools/typespec-azure-resource-manager/arm-custom-resource-no-key 🟢 0.1ms 🟢 0.2ms +104.5%
 ↳ linter/@azure-tools/typespec-azure-resource-manager/arm-custom-resource-usage-discourage 🟢 0.1ms 🟢 0.1ms +76.4%
 ↳ linter/@azure-tools/typespec-azure-resource-manager/arm-delete-operation-response-codes 🟢 2.3ms 🟢 4.5ms +99.9% 🔴
 ↳ linter/@azure-tools/typespec-azure-resource-manager/arm-no-path-casing-conflicts 🟡 12.6ms 🟡 16.8ms +34.2% 🔴
 ↳ linter/@azure-tools/typespec-azure-resource-manager/arm-no-record 🟢 0.4ms 🟢 0.7ms +62.0%
 ↳ linter/@azure-tools/typespec-azure-resource-manager/arm-post-operation-response-codes 🟢 0.7ms 🟢 1.4ms +101.6%
 ↳ linter/@azure-tools/typespec-azure-resource-manager/arm-put-operation-response-codes 🟢 0.1ms 🟢 0.2ms +240.6%
 ↳ linter/@azure-tools/typespec-azure-resource-manager/arm-resource-action-no-segment 🟢 0.3ms 🟢 0.5ms +67.0%
 ↳ linter/@azure-tools/typespec-azure-resource-manager/arm-resource-duplicate-property 🟢 0.2ms 🟢 0.4ms +132.5%
 ↳ linter/@azure-tools/typespec-azure-resource-manager/arm-resource-interface-requires-decorator 🟢 0.0ms 🟢 0.1ms +200.3%
 ↳ linter/@azure-tools/typespec-azure-resource-manager/arm-resource-invalid-action-verb 🟢 0.1ms 🟢 0.2ms +70.0%
 ↳ linter/@azure-tools/typespec-azure-resource-manager/arm-resource-invalid-envelope-property 🟢 0.1ms 🟢 0.3ms +116.7%
 ↳ linter/@azure-tools/typespec-azure-resource-manager/arm-resource-invalid-version-format 🟢 0.1ms 🟢 0.2ms +164.6%
 ↳ linter/@azure-tools/typespec-azure-resource-manager/arm-resource-key-invalid-chars 🟢 0.3ms 🟢 0.5ms +78.0%
 ↳ linter/@azure-tools/typespec-azure-resource-manager/arm-resource-name-pattern 🟢 0.0ms 🟢 0.1ms +276.3%
 ↳ linter/@azure-tools/typespec-azure-resource-manager/arm-resource-operation 🟢 0.2ms 🟢 0.6ms +138.7%
 ↳ linter/@azure-tools/typespec-azure-resource-manager/arm-resource-operation-response 🟢 5.9ms 🟢 9.5ms +61.4% 🔴
 ↳ linter/@azure-tools/typespec-azure-resource-manager/arm-resource-patch 🟢 0.4ms 🟢 0.9ms +102.6%
 ↳ linter/@azure-tools/typespec-azure-resource-manager/arm-resource-path-segment-invalid-chars 🟢 0.2ms 🟢 0.4ms +66.7%
 ↳ linter/@azure-tools/typespec-azure-resource-manager/arm-resource-provisioning-state 🟢 0.2ms 🟢 0.4ms +147.8%
 ↳ linter/@azure-tools/typespec-azure-resource-manager/beyond-nesting-levels 🟢 0.1ms 🟢 0.2ms +112.5%
 ↳ linter/@azure-tools/typespec-azure-resource-manager/empty-updateable-properties 🟢 0.2ms 🟢 0.4ms +109.6%
 ↳ linter/@azure-tools/typespec-azure-resource-manager/improper-subscription-list-operation 🟢 0.0ms 🟢 0.1ms +321.1%
 ↳ linter/@azure-tools/typespec-azure-resource-manager/lro-location-header 🟡 15.4ms 🔴 21.1ms +37.3% 🔴
 ↳ linter/@azure-tools/typespec-azure-resource-manager/missing-operations-endpoint 🟢 0.0ms 🟢 0.2ms +204.3%
 ↳ linter/@azure-tools/typespec-azure-resource-manager/missing-x-ms-identifiers 🟢 0.4ms 🟢 1.0ms +128.8%
 ↳ linter/@azure-tools/typespec-azure-resource-manager/no-empty-model 🟢 0.2ms 🟢 0.3ms +77.4%
 ↳ linter/@azure-tools/typespec-azure-resource-manager/no-override-props 🟢 0.1ms 🟢 0.3ms +165.8%
 ↳ linter/@azure-tools/typespec-azure-resource-manager/no-resource-delete-operation 🟢 0.2ms 🟢 0.5ms +101.3%
 ↳ linter/@azure-tools/typespec-azure-resource-manager/no-response-body 🔴 22.6ms 🔴 32.4ms +43.2% 🔴
 ↳ linter/@azure-tools/typespec-azure-resource-manager/patch-envelope 🟢 0.2ms 🟢 0.4ms +110.3%
 ↳ linter/@azure-tools/typespec-azure-resource-manager/resource-name 🟢 0.2ms 🟢 0.5ms +118.8%
 ↳ linter/@azure-tools/typespec-azure-resource-manager/secret-prop 🟢 4.5ms 🟢 5.3ms +18.6%
 ↳ linter/@azure-tools/typespec-azure-resource-manager/unsupported-type 🟢 0.4ms 🟢 0.7ms +60.1%
 ↳ linter/@azure-tools/typespec-azure-resource-manager/version-progression 🟢 0.1ms 🟢 0.3ms +231.9%
 ↳ linter/@azure-tools/typespec-client-generator-core/property-name-conflict 🟢 1.4ms 🟢 2.4ms +74.3% 🔴
 ↳ linter/@azure-tools/typespec-client-generator-core/require-client-suffix 🟢 0.5ms 🟢 1.7ms +228.4% 🔴
emit 🔴 5.63s 🔴 6.20s +10.1% 🔴
 ↳ emit/Azure-Tools 🟢 0.0ms 🟢 0.0ms +0.0%
 ↳ emit/@azure-tools/typespec-autorest 🟢 91.0ms 🟢 127.8ms +40.5% 🔴
 ↳ emit/@azure-tools/typespec-python 🔴 2.15s 🔴 2.31s +7.4% 🔴
 ↳ emit/typespec 🟢 0.0ms 🟢 0.0ms +0.0%
 ↳ emit/@typespec/http-client-js 🔴 487.7ms 🔴 589.5ms +20.9% 🔴
 ↳ emit/@typespec/openapi3 🟢 81.8ms 🟢 111.0ms +35.7% 🔴
 ↳ emit/@typespec/openapi3/compute 🟢 71.3ms 🟢 96.3ms +35.1% 🔴
 ↳ emit/@typespec/openapi3/write 🟢 10.3ms 🟢 14.4ms +39.6% 🔴

Averaged across 3 specs (azure-arm-resource-manager, azure-core-dataplane, azure-full).
Threshold: changes > ±5% are highlighted.
🟢 Fast · 🟡 Moderate (stages >200ms, rules >10ms) · 🔴 Slow (stages >400ms, rules >20ms)

Libba Lawrence (l0lawrence) and others added 7 commits July 1, 2026 09:27
@l0lawrence
Port emitter-diff fixes to typespec-azure and remove --vscode
2079887 
- Absolute work-dir fix, --specs fast-fail, parallelization + logPrefix (from core)
- Remove --vscode/openInVsCode (HTML report is sufficient)
- Workflow: use PIPESTATUS for tool exit code, drop per-file list (link HTML),
  add hard-error PR comment branch

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@l0lawrence
Emitter diff (Azure): merge-base baseline, hardening, drop emojis
2844eed 
Mirrors the core emitter-diff improvements to the Azure workflow:
- Baseline is now the PR's git merge-base with its base branch (survives
  squash-merge / rebase / force-push) instead of the latest npm version.
- The check is informational: it never fails the PR, only reports the
  generated-output diff via job summary, PR comment, and HTML artifact.
- Security hardening in resolver.ts (validate git refs/repos, --end-of-options,
  npm install --ignore-scripts) and de-duplicated HTML report in diff.ts.
- Removed emojis from tool output, job summary, and PR comment.
- Moved untrusted workflow inputs into env vars (Actions expression injection).

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@l0lawrence
Emitter diff (Azure): fail only on real tool error
bfafb25 
Adds a "Fail on tool error" step so the informational check fails only when the
tool hits a real build/venv/generate error (non-zero exit), not on a diff.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@l0lawrence
Simplify emitter-diff: zero-dep, plain node scripts
a513493 
Trim the tool down to plain TypeScript scripts with no runtime
dependencies and no package wrapper.

- Replace diff2html with a self-contained, GitHub-style HTML renderer
  (inline CSS, light/dark, no external requests); remove it from the
  package deps, pnpm catalog, and lockfile.
- Drop the --run-tests/--test-env/--test-target surface and the adapter
  runTests contract; parallel generation and --sequential are unchanged.
- Delete package.json and drop the tool from pnpm-workspace.yaml so it is
  no longer a named package (removes npm dependency-confusion surface).
- Run with plain
ode (Node 24 strips TypeScript natively) instead of
  tsx: rewrite relative imports to .ts and set allowImportingTsExtensions.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@l0lawrence
Allow .ts extension imports in eng typecheck
7faecd5 
Running emitter-diff with plain node requires .ts import specifiers, but
the repo-wide check:eng gate (tsconfig.eng.json, --noEmit) rejected them.
Enable allowImportingTsExtensions there so the tool stays covered by the
existing typecheck.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@l0lawrence
Log baseline/head output dirs after generation
15a51e0 
A local run now prints the exact generated trees the diff compares, so
developers can open the head (current checkout) output directly instead of
inferring it from the work dir. Language-agnostic: lives in cli.ts, labeled
per side by the resolved emitter.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@l0lawrence
Add typescript and rust adapters to emitter-diff
5de3db4 
Add a typescript adapter (@azure-tools/typespec-ts) and a rust adapter
(@azure-tools/typespec-rust) to the language-agnostic emitter-diff tool,
plus a typescript CI workflow mirroring the python one.

- typescript adapter wraps the package's gen-spector.js driver; adds
  --emitter-dir/--output-dir override hooks to run.ts + gen-spector.js so
  any emitter build can target an isolated output dir.
- rust adapter wraps .scripts/tspcompile.js; inert-until-present (fails
  fast with guidance) since typespec-rust has not moved into this repo yet.
- ci-emitter-diff-typescript.yml: merge-base baseline, informational-only
  PR comment + HTML artifact.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@microsoft-github-policy-service microsoft-github-policy-service Bot added the emitter:typescript Issues for @azure-tools/typespec-ts emitter label Jul 1, 2026
@github-actions

github-actions Bot commented Jul 1, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

TypeScript emitter diff

Baseline 5c8a2d875f98051d61bb786ff3b30d212294a0bf vs this PR.

No changes to generated output.

Full rendered diff: emitter-diff-html-typescript artifact on the run.

Informational check (eng/emitter-diff); does not block the PR.

@l0lawrence
Slim down emitter-diff CI workflows
44b2205 
Cut the python + typescript workflows to ~half their size:
- trim explanatory comment headers
- drop the hand-rolled github-script patch parser; reuse the tool's own
  'Diff summary:' line (ANSI-stripped) as a step output for the PR comment
- drop the redundant job-summary step and the now-unused --patch output
- collapse 'Fail on tool error' to a one-liner

Behavior is unchanged: sticky PR comment, HTML artifact, informational-only
(fails only on a tool/build error, never on a diff).

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@iscai-msft iscai-msft Awaiting requested review from iscai-msft iscai-msft will be requested when the pull request is marked ready for review iscai-msft is a code owner
@msyyc Yuchao Yan (msyyc) Awaiting requested review from msyyc msyyc will be requested when the pull request is marked ready for review msyyc is a code owner
@tadelesh Chenjie Shi (tadelesh) Awaiting requested review from tadelesh tadelesh will be requested when the pull request is marked ready for review tadelesh is a code owner
@ChenxiJiang333 Chenxi Jiang (ChenxiJiang333) Awaiting requested review from ChenxiJiang333 ChenxiJiang333 will be requested when the pull request is marked ready for review ChenxiJiang333 is a code owner
@kashifkhan Kashif Khan (kashifkhan) Awaiting requested review from kashifkhan kashifkhan will be requested when the pull request is marked ready for review kashifkhan is a code owner
@xirzec Jeff Fisher (xirzec) Awaiting requested review from xirzec xirzec will be requested when the pull request is marked ready for review xirzec is a code owner
@maorleger Maor Leger (maorleger) Awaiting requested review from maorleger maorleger will be requested when the pull request is marked ready for review maorleger is a code owner
@qiaozha Qiaoqiao Zhang (qiaozha) Awaiting requested review from qiaozha qiaozha will be requested when the pull request is marked ready for review qiaozha is a code owner
@MaryGao Mary Gao (MaryGao) Awaiting requested review from MaryGao MaryGao will be requested when the pull request is marked ready for review MaryGao is a code owner
@JialinHuang803 Jialin Huang (JialinHuang803) Awaiting requested review from JialinHuang803 JialinHuang803 will be requested when the pull request is marked ready for review JialinHuang803 is a code owner
@markcowl Mark Cowlishaw (markcowl) Awaiting requested review from markcowl markcowl will be requested when the pull request is marked ready for review markcowl is a code owner
@timotheeguerin Timothee Guerin (timotheeguerin) Awaiting requested review from timotheeguerin timotheeguerin will be requested when the pull request is marked ready for review timotheeguerin is a code owner

Assignees

No one assigned

Labels

emitter:python Issues for @azure-tools/typespec-python emitter emitter:typescript Issues for @azure-tools/typespec-ts emitter eng

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@l0lawrence