Repository navigation
Conversation
v2 deploys only projects with a pyproject.toml, on Astro and on Astro Private Cloud. Astro CLI 1.x keeps deploying the projects it made, so v2 drops the 1.x deploy rather than carrying it. Astro: astro deploy in a 1.x-layout project (a Dockerfile beside .astro, or a .astro/config.yaml) fails with no_project, saying to convert it with astro init or deploy it with Astro CLI 1.x. Outside any project it gives the no-project advice, now also no_project. --non-dags runs first, from anywhere: it deploys a separate directory, and from a pyproject.toml project it no longer runs a whole project deploy instead. APC: v2 builds no project. A deploy without --image-name is refused as a usage error saying to use Astro CLI 1.x (and, from a pyproject.toml project, that support is coming). --image-name (and --remote) deploy from anywhere but a 1.x project; --dags uploads a pyproject.toml project's dags/. Any deploy from a 1.x project is refused. The 1.x deploy is deleted with its tests: deploy.Deploy and its build, pytest and parse steps, the named DAG bundle create, APC's Dockerfile build, airflow's DAGChecker and ImageHandler.Pytest, and what deadcode found behind them. The flags only it read are removed and tombstoned in the removed-flags registry: --save, --pytest, --env, --test, --parse, --deployment-name, --dags-path and --dag-bundle-name on Astro, --save and --no-cache on APC. From #2290: config.IsHomeDir in IsProjectDir, the home-directory advice, and the bound-value --image-name check. Co-Authored-By: Claude <noreply@anthropic.com>
Coverage Report for CI Build 38055987394Coverage decreased (-0.2%) to 57.503%Details
Uncovered Changes
Coverage Regressions17 previously-covered lines in 4 files lost coverage.
Coverage Stats
💛 - Coveralls |
From review of the first commit: - astro deploy --image-name runs from any directory again, a 1.x project's included: it reads nothing from the project. Outside a pyproject.toml project it ships the image alone (no dags/), to an id named by the argument or --deployment, or the workspace's pick. - --non-dags validates and honours --output (one json object, pinned as deploy-non-dags), names its target as a deploy does (link or id, the argument and --deployment must agree) and reads --workspace. - APC deploy no longer falls back to the project.deployment a 1.x .astro/config.yaml saved; a build outside any project is no_project, as on Astro; the 1.x refusal no longer talks about building. - What a directory is comes from project.Discover, so a deploy below a 1.x project names it, and one below a pyproject.toml project says to run from its root. utils.Is1xLayout and EnsureProjectDir are gone; one helper gives every refusal, in plain text. - astro remote deploy follows the same rule, with --image-name anywhere. - astro deploy --prompt is removed and tombstoned. --force stays, hidden and read by nothing, because astronomer/deploy-action passes it. - The unreachable errNoImageName check and ansi.Red go. Co-Authored-By: Claude <noreply@anthropic.com>
…latforms From the second review: - --image-name follows one rule on Astro and APC: refused in or below a 1.x project and below a pyproject.toml project's root, as every deploy there is; the project's deploy at its root; and outside any project the image alone. There, Astro warns when the Deployment takes DAG deploys (stderr, and a warnings field in the deploy result), and APC uploads no dags/ and says the DAGs were not updated. - utils.Locate is the one walk astro deploy, remote deploy and APC deploy decide by: a pyproject.toml with [tool.astro] stops it, a tooling-only one does not, a Dockerfile beside .astro is a 1.x project except in the home directory, and an unreadable directory holds neither. No project.Discover or hostname derivation. - The flag combinations 1.x refused come back where they apply: --wait-time without --wait, --no-dags-base-dir with a deploy that ships no DAGs, and --dags with --image/--image-name naming only those given. - --non-dags takes a Deployment id without reading the manifest, and a link under the login for the project's host. - An image-only deploy with no target asks for an id, not a project link. - astro config get and set refuse project.deployment, which nothing reads. - The APC --no-cache tombstone points at Astro CLI 1.x only. Co-Authored-By: Claude <noreply@anthropic.com>
- --prompt and -p stay on astro deploy, hidden and read by nothing, as a pyproject.toml project's deploy always took them; the tombstone goes. - utils.IsManifestRoot is the one test for a project's root, and the deploy routes on Locate: a pyproject.toml that fails to parse is a root only when its text declares tool.astro, one that cannot be read is a root (as project.HasManifest has it), and a directory that cannot be looked in is passed over. Each ancestor's pyproject.toml is read once, and NoDeployableProject takes the walk's answer instead of walking again. - astro config list leaves out the removed keys get and set refuse. - --non-dags from a project deploys a bare Deployment id under the project's host, as a link already was; its bundle-path check is the walk. - An --image-name deploy outside a project to a Deployment without DAG deploys warns that it runs the DAGs inside the image. - APC: an image deployed alone from outside a project skips the uncommitted-changes check, and --dags with --image-name is refused in astro deploy's words. Co-Authored-By: Claude <noreply@anthropic.com>
- --non-dags under --output json sends what the platform prints (the git note) to stderr, so stdout is the one result object. - --non-dags naming no Deployment is refused with input_required when the run cannot be asked, as astro deploy is, instead of opening the picker. - Its login follows astro deploy from the same directory: the project's, for ids, links and the picker, in a pyproject.toml project that loads; the current context's anywhere else. The picker is offered only on the current context's host. - astro dbt deploy's containment check is the walk too, so it and --non-dags agree; config.IsWithinProjectDir and isWithinManifestProject go. - The walk carries the manifest it loaded (or why it did not), which the deploy and --non-dags use instead of loading it again, and looks the home directory up once. - nonDagsDeployJSON is built from the result directly; cmp.Or replaces a local helper. Co-Authored-By: Claude <noreply@anthropic.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The decision
v2 builds and deploys only
pyproject.toml(manifest) projects, on Astro and on Astro Private Cloud. Nobody is forced onto v2 and Astro CLI 1.x keeps working, so v2 does not carry a deploy for the 1.x project layout (Dockerfile+.astro/config.yaml). A 1.x-layout project is converted withastro init, or deployed with Astro CLI 1.x.What a directory is, and where the deploy routes, comes from one walk,
utils.Locate. It is used byastro deploy,astro remote deploy, APC's deploy, and the bundle-path checks of--non-dagsandastro dbt deploy; it carries the manifest it loaded, so routing reads each file once. From the working directory up, at each directory:utils.IsManifestRoot) stops the walk;.astro/is a 1.x project and stops it, except in the home directory, whose.astroholds the global config;A root is a
pyproject.tomlwhose[tool.astro]loads or fails to validate, one that fails to parse only if its text declares atool.astrotable, or one that exists but cannot be read. The last matchesproject.HasManifest, so the deploy reports the read error. A tooling-onlypyproject.tomldoes not stop the walk, and a directory that cannot be looked in holds neither. Each ancestor'spyproject.tomlis read once.Behaviour
Astro (
astro deploy)--non-dags--image-name <ref>dags/includedthis project uses the Astro CLI 1.x layout (a Dockerfile and .astro/config.yaml), and Astro CLI v2 deploys only pyproject.toml projects. Convert it with astro init, or deploy it with Astro CLI 1.x; from below:this directory is inside a project at <dir> that uses …. Convert it with astro init in <dir>, …--image-name X --force) would silently leave its DAGs stalethis directory is inside the project at <dir>. Run the deploy from the project directory, <dir>this is not an Astro project directory. Change to an Astro project directory, or run astro init …(in~, no init advice)dags/, no git commit; target by id (argument /--deployment) or the workspace pick; non-interactive with no target:pass the Deployment id as the argument or with --deployment. Always warns, on stderr and in the result's newwarningsfield: with DAG deploys, that the Deployment keeps its DAGs; without, that it runs the DAGs inside the image (APC's wording)--non-dagsruns from anywhere, before the walk.--outputis validated (a bad value is a usage error, exit 2).-o jsonpublishes one object (new goldendeploy-non-dags.json), and what the platform prints on the way (the git note) goes to stderr.astro deploy <id>from the same directory chooses it. In a pyproject project here whose manifest loads, the project's login (its host) is used for everything: ids, links and the picker. Outside a project, or when the manifest does not load, the current context is used for everything, so a brokenpyproject.tomldoes not stop an id-targeted deploy.--deployment, which must agree. With no name, a run that cannot be asked (-o json, no terminal) gets the sameinput_requiredrefusal asastro deploy. One that can is offered the picker (--workspace/--workspace-id/ the context) only on the current context's host; on another host it is told to pass--deployment <id>or switch context.astro dbt deploy.Flag combinations, refused as usage errors before anything is read:
--wait-timewithout--wait(all modes, as 1.x did);--no-dags-base-dirwith a deploy that ships no DAGs (--image, or--image-nameoutside a project; at a project root--image-nameships the project's DAGs, so the flag applies there);--dagswith--image/--image-namenames only the flags given.astro remote deployfollows the same walk: build at a pyproject root;--image-nameat a root or outside any project; refused in/below a 1.x project and below a root.Astro Private Cloud (
astro deployunder an APC context)--image-name, or--image-name=)--image-name/--remote--dagsAstro CLI v2 cannot build and deploy projects to Astro Private Cloud yet: use Astro CLI 1.x for now. Support for pyproject.toml projects on Astro Private Cloud is coming. …dags/to a Deployment that takes uploadsdags/… uses the Astro CLI 1.x layout (…), which Astro CLI v2 does not deploy to Astro Private Cloud. Deploy it with Astro CLI 1.x… use Astro CLI 1.x for now. To deploy an image you built yourself, pass --image-namedags/upload and no uncommitted-changes check (nothing is read from the directory); when the Deployment takes DAG uploads (or that could not be read) it says the DAGs were not updated, whatevershow_warningsis--dagswith--image-nameis refused as a usage error, in Astro's words:--dags deploys only your DAGs; drop --image-name.The target is the argument or the picker. The
project.deploymenta 1.x.astro/config.yamlsaved is no longer read (a test pins a stale one being ignored), andastro config set/get project.deploymentsay it was removed (the existing removed-config-key convention; the key stays registered, asdev.modedoes).astro config listleaves out every removed key, so list and get agree. Every refusal comes before Houston is asked anything.Removed code
deploy.Deployand everything only it reached (image build,.dockerignorerewriting, pytest/parse steps, the v1alpha1 named-bundle create,ValidRuntimeVersion,WarnIfNonLatestVersion,finalizeDeploy,deployDags); APC's Dockerfile build and FROM validation;airflow.DAGChecker,DAGCheck+ mock,ProjectNameUnique,ImageHandler.Pytest;docker.GetImageTagFromParsedFile;fileutil.GetFilesWithSpecificExtension;util.Exists;ansi.Red;utils.EnsureProjectDir/EnsureDockerfileProjectDir(replaced byutils.Locate+utils.NoDeployableProject, plain text);config.IsWithinProjectDirandisWithinManifestProject(the bundle deploys' containment checks are the walk now); unused testfiles. Net: 70 files, +2297 / −4534.Removed flags (tombstoned in
cmd/removed_flags.go,under: deploy)--save,-s--deployment;astro link add+default = trueto preselect--save,-s--pytest,--test(-t),--env(-e)uv run pytest && astro deploy--parseastro local check && astro deploy--dags-pathastro deploy --dagsfrom the project--dag-bundle-name--deployment-name,-n--deployment--no-cache--force(-f) and--prompt(-p) stay on Astro deploy, hidden and read by nothing. astronomer/deploy-action v0.16.0 passes--forceon every deploy (TestDeployManifestDeployActionInvocations), and a pyproject project's deploy always accepted both.Carried over from #2290 (superseded)
#2290 is superseded by this PR. Carried over:
config.IsHomeDirinIsProjectDir(with its test); the home-directory advice; reading--image-nameby value (--image-name=is a build). Dropped as moot:EnsureDockerfileProjectDir, the APC--saveguard, the APC--dagsacceptance logic.Merge-tree against the open PRs (at 09cbaad)
init-keep-apc-dockerfilecmd,cmd/astro,cmd/apc,cmd/local,cmd/utilstests passremoved-commands-and-bundle-outputcmd(both removed-flag guards),cmd/astro,cmd/apc,cmd/utilstests passproject-dir-advice-corecmd/apc/deploy.go,cmd/astro/deploy.go,cmd/astro/deploy_test.go,cmd/astro/remote_test.go,cmd/utils/utils.go,cmd/utils/utils_test.go,config/config.go,config/config_test.go(superseded)render-backticksairflow/docker.go,cmd/apc/deploy.go,cmd/astro/deploy.go,cmd/removed_flags.go,internal/platform/astro/deploy/deploy.go; take this side (already plain)deploy-ships-project-filescmd/apc/deploy.go,cmd/astro/deploy.go,cmd/astro/testdata/schema/deploy.json,internal/deploy/deploy.go,internal/platform/astro/deploy/manifest_image.go(both add to the deploy result and its routing; the golden needs both fields)Docs
docs/deploy.md(the walk, a per-location table,--image-nameoutside a project,--non-dags, flag combinations, remote deploy, an APC section),docs/upgrading-from-v1.md("v2 requires converting your project" near the top, removed flag rows,project.deploymentgone),docs/install.md,README.md,docs/architecture.md.Checks
go build ./...,GOOS=windows go build ./...,GOOS=windows go vet(touched root packages;e2ewith-tags e2e),make test,make test-e2e(tier 0),make lint(no autofix changes),make lint-submodules,make lint-e2e,make lint-goos,make deadcode(clean),make update-schemas(deploy-non-dags.jsonnew;deploy.jsongainswarnings).Tests cover:
-o jsonwith their kinds: at a root, below it, in/below a 1.x project (incl.--image-name), and in~;~with pyproject + Dockerfile +.astro→ not 1.x; non-ASCII names; an unreadable ancestor;--image-nameoutside a project: no DAGs, the DAG-deploy warning, the no-target message;--non-dagsoutput / agreement / links / id with a broken manifest, the stale saved APC deployment,config get/set project.deployment, remote deploy's rule, and a case per removed-flag entry.e2e tier 0:
TestDeployRefusesThe1xOnlyFlags,TestDeployRefusesA1xProject,TestDeployOutsideAProject,TestDeployImageNameOutsideAProject(outside: past the project check to the container engine, offline; 1.x: refused, no_project),TestAPCDeployRefusesToBuild(against a refusing Houston).Open questions
--forceon Astro deploy: drop it once deploy-action stops passing it?cosmos_boost.pre_deploynow affects only bundle and dbt deploys.