Repository navigation
Fix typespec-java emitter build hanging in parallel repo build - #4972
Merged
Merged
Conversation
Copy-Sources.ps1 and SyncTests.ps1 no longer check out the pinned core/ commit in place. That mutated the shared submodule working tree and raced with the concurrent monorepo build / regen-all-packages-docs, hanging the full pnpm build at the java emitter (while an isolated package build was fine). CoreCommit.ps1 now reads the pinned sources via git archive into a temp dir, leaving the core/ checkout untouched.
microsoft-github-policy-service
Bot
added
the
emitter:java
Contributor
|
All changed packages have been documented.
Show changes
|
commit: |
Contributor
|
You can try these changes here
|
The emitter.jar reactor only builds http-client-generator, -mgmt and -core. http-client-generator-test and http-client-generator-clientcore-test live in a never-activated Maven test profile and are not needed by the build (nor by publish), so skip copying them. Cuts the generator copy from ~17s to ~2s.
Weidong Xu (weidongxu-microsoft)
approved these changes
Jul 21, 2026
Weidong Xu (weidongxu-microsoft)
left a comment
Weidong Xu (weidongxu-microsoft)
left a comment
Member
There was a problem hiding this comment.
Xiaofei Cao (@XiaofeiCao) I approved and will merge soon to unblock.
We can refine later if necessary.
Weidong Xu (weidongxu-microsoft)
marked this pull request as ready for review
July 21, 2026 01:49
Dapeng Zhang (ArcturusZhang)
pushed a commit
to ArcturusZhang/typespec-azure
that referenced
this pull request
Jul 21, 2026
…f tar (Azure#4983) ## Problem Following Azure#4972, `packages/typespec-java/CoreCommit.ps1` reads the pinned `core/` commit's sources by `git archive` into a temp dir and then extracts them with an external `tar`. On Windows this fails: the `tar` first on PATH is Git for Windows' GNU `tar`, which interprets the archive path `-f C:\...\core.tar` as a **remote host** (`host:path` rsh syntax) and aborts: ``` /usr/bin/tar: Cannot connect to C: resolve failed ``` Linux CI doesn't hit this (`/tmp/...` has no drive-letter colon), so it only bites Windows dev builds. ## Fix `Get-CoreSourceRoot` now archives with `git archive --format=zip` and extracts with PowerShell's built-in `Expand-Archive` (System.IO.Compression). This keeps the whole flow inside PowerShell/.NET with **no external archiver dependency** and no drive-letter parsing quirk. `core/` is still never checked out or modified, and the temp dir is still cleaned up by `Remove-CoreSourceRoot`. Also refreshes `CONTRIBUTING.md`, which still described the pre-Azure#4972 in-place `core/` checkout / `git checkout .` patch flow, to match the current `git archive` read and copy-and-patch-the-generator flow. ## Verification Pinned `core-commit.json` to a commit newer than the current `core` checkout to force the archive path, then ran `Copy-Sources.ps1`: - Took the archive branch (*"Reading core sources from pinned commit ... without checking it out"*). - `git archive --format=zip` + `Expand-Archive` succeeded; generator copied and `core.patch` applied; `exit=0`. - `git -C core status` clean (submodule untouched), pin restored, no temp dir leaked. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Yuchao Yan (msyyc)
pushed a commit
to msyyc/typespec-azure
that referenced
this pull request
Jul 22, 2026
…5004) ## Problem A full-repo `pnpm build` intermittently **hangs at the Java emitter build**. The output stops right after: ``` @azure-tools/typespec-java:build: core-commit.json SHA ... using current checkout. ``` and never reaches `Copy generator sources from core`. Building `packages/typespec-java` on its own always works. Prior attempts (Azure#4972, Azure#4983, and non-interactive/fetch hardening) did not resolve it. ## Root cause turbo runs each task's process in its own **background process group** under the build's **controlling terminal** (the TUI). When a PowerShell cmdlet renders **`Write-Progress`** — which `Copy-Item -Recurse` in `Copy-Sources.ps1` does — pwsh touches that terminal from a background process group and the OS stops the process with **SIGTTIN/SIGTTOU** (`T` state). It never resumes, so the build hangs forever. This explains all the symptoms: - **Java-only:** it is the only package that shells out to `pwsh`. - **Intermittent:** `Copy-Item` only renders progress when the copy is slow enough (under parallel-build load). - **Invisible when piped / in isolation:** with piped output (no TUI) or a standalone build, pwsh is not a background process group of a controlling terminal, so nothing stops it. - **Unaffected by `-NonInteractive`:** progress renders regardless. Reproduced directly: a pwsh running `Write-Progress` in a background process group under a controlling tty immediately enters `T` (stopped); with `$ProgressPreference = 'SilentlyContinue'` it never stops. ## Fix - Set `$ProgressPreference = 'SilentlyContinue'` in `Copy-Sources.ps1`, `Build-Generator.ps1`, and `CoreCommit.ps1` so no cmdlet touches the controlling terminal. **This is the actual fix.** Complementary hardening (removes secondary hang/network vectors): - `CoreCommit.ps1`: only `git fetch` the pinned core commit when it is genuinely missing locally (`git cat-file -e <sha>^{commit}` first) — previously it hit the network on *every* emitter build / doc regen. - `CoreCommit.ps1`: `GIT_TERMINAL_PROMPT=0` and `pwsh -NonInteractive` (package.json) so a git credential/host prompt can never block the headless build. - `CoreCommit.ps1`: extract pinned sources with `[System.IO.Compression.ZipFile]::ExtractToDirectory` instead of the pathologically slow `Expand-Archive`. ## Verification - Reproduced the `T`-state stop with `Write-Progress` under a controlling tty; confirmed `$ProgressPreference='SilentlyContinue'` prevents it. - 8/8 runs of the real `Copy-Sources.ps1` in a background process group under a controlling tty never stop. - Full `pnpm build` completes 63/63; `git -C core status` stays clean.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Running a full-repo
pnpm buildintermittently hangs at the Java emitter build, even though building directly inpackages/typespec-javaworks fine.Root cause
packages/typespec-java/Copy-Sources.ps1(andemitter-tests/SyncTests.ps1) read the emitter/generator sources from thecore/submodule at the commit pinned incore-commit.json. The oldCoreCommit.ps1did this by checking that commit out in place (Enter-CoreCommit→git -C core checkout <pinnedSha>,Restore-CoreCommit→ checkout back), mutating the sharedcore/working tree.During a full
pnpm build, turbo builds manycore/packages/*concurrently, and the website'sregen-all-packages-docsrunstypespec-java'sregen-docs(which re-runsbuild:emitter→Copy-Sources.ps1) in parallel with other packages'regen-docsthat readcore/. When the pin is newer than the localcore/checkout, the in-place checkout swapscore/files out from under those concurrent readers → intermittent hang. An isolated package build never hits this because nothing else runs concurrently.Fix
CoreCommit.ps1now reads the pinned sources withgit archiveinto a temp directory (Get-CoreSourceRoot/Remove-CoreSourceRoot) and never checks out or otherwise touches thecore/working tree. When the pin is not newer than the current checkout, it reads the live working tree as before.git statusoncore/now always stays clean.Verification
core/checkout: sources extracted viagit archive,core/left untouched, no temp dirs leaked.core/untouched.emitter.jarproduced.