Repository navigation
Clone mirrors aside and clean up interrupted clones - #3
Merged
Merged
Conversation
- Clone into a .clone-<name>-* directory and rename it into place after the first fetch, so a clone killed by the timeout, a restart or the OOM killer no longer leaves a directory that looks like a mirror or that git refuses to clone into - Remove junk without HEAD at the mirror path before cloning, so mirrors broken by earlier versions heal on their own - Repair clone config on every Ensure in one place: scrub a token from the remote URL and restore a missing fetch refspec; write it through a rename - Kill the whole process group on cancel (new pkg/client/proc), for git and the AST engine: helpers kept writing and held the pipes, so Run outlived its deadline - Give the first clone its own 30-minute timeout and limit clones to two at a time: a caller that gives up does not stop its clone, so the cron limit bounded waiting, not cloning - Remove abandoned clone directories and tmp_* packs in Prune, regardless of the disk budget, and report removals through Options.OnRemove for the log - Return git.ErrCloning when the caller stops waiting for a clone in progress: log it as "mirror still cloning" and answer tools with Timeout instead of Upstream
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
A mirror was cloned in place, and "HEAD exists" meant "the mirror exists". On one instance with little memory, clones were OOM-killed and cut off by the 5-minute timeout again and again. Nothing cleaned up after them: the mirror of a 0.85 GB repository grew to 65 GB of packs with no HEAD. After that every cron tick failed with
destination path already exists and is not an empty directory, and the mirror stayed broken until someone removed it by hand.What changed
.clone-<name>-*, then rename into place after the first fetch. A clone that fails or is killed leaves no directory that looks like a mirror.repairConfigruns on everyEnsure. It scrubs a token from the remote URL and restores a missing fetch refspec, and writes the config through a rename.pkg/client/proc), for git and for the AST engine, whose npm install is a node shim around the binary. Before, helpers kept writing and held the pipes, soRunoutlived its deadline by minutes.CloneTimeout, and at most 2 clones run at a time.once()lets work outlive a caller that gave up, so the cron limit bounded waiting, not cloning.Pruneremoves.clone-*directories andobjects/pack/tmp_*files that have been untouched longer than a clone plus its fetch, regardless of the disk budget. Removals are reported throughOptions.OnRemoveand logged asstorage removed.git.ErrCloningis returned when the caller stops waiting for a clone that is still running. Cron logs it asmirror still cloning, and tools answerTimeoutwith a hint to retry instead ofUpstream.Tests
Runhangs for 60 s after cancel.go test ./...and-raceon the touched packages pass,golangci-lint: 0 issues.Rollout notes
.clone-*directories are not cleaned by the old code; remove them by hand.