Skip to content

πŸ”§ Check conda-forge availability when a dependency is added, not at release timeΒ #604

Description

@sodre

Description

Add a rule to .claude/rules/ requiring a conda-forge availability check at the moment a runtime dependency is added or its floor is raised β€” not at release time, when the release has already shipped to GitHub and PyPI and the only remaining lever is a third-party feedstock.

Proposed location: a new file, .claude/rules/dependencies.md. No existing rule covers dependency management. release-planning.md is the nearest neighbour but is about milestones and scopes, and filing this rule there would contradict its own point β€” the check belongs at dependency-add time, not in the release checklist. A new file keeps the trigger ("I am editing pyproject.toml's dependencies") next to the rule.

Details

The rule should say: when adding or raising a runtime dependency, check conda-forge at that moment.

  1. Query the API, not the web page.

    curl -s https://api.anaconda.org/package/conda-forge/<name> \
      | python3 -c "import sys,json; d=json.load(sys.stdin); print(d['latest_version'], d['versions'])"

    https://anaconda.org/conda-forge/<name> returns HTTP 200 even for packages that do not exist, so a status-code check against the web page proves nothing. Verified:

    URL Nonexistent package
    https://anaconda.org/conda-forge/<name> 200 ❌
    https://api.anaconda.org/package/conda-forge/<name> 404 βœ…

    That exact mistake is what let this slip through.

  2. Confirm the constraint is satisfiable, not merely that the package exists. Read versions / latest_version and check the version range in pyproject.toml against them. A package present at an older version is the same blocker as one that is absent.

  3. If it is missing or too old, open the fix then. A staged-recipes PR for a genuinely new package, or a version-bump PR on the existing feedstock. A feedstock already existing means staged-recipes is the wrong route and the PR will be rejected β€” check for conda-forge/<name>-feedstock first.

  4. Also check that the upstream version publishes an sdist. conda-forge recipes conventionally source the PyPI sdist (/packages/source/...tar.gz); a wheel-only release breaks both the recipe URL and the autotick bot, silently.

The check should also run in CI, not only live in a rule

A rule relies on someone remembering at the moment they edit a dependency β€” which is exactly what failed for cronsim. A path-filtered job removes that dependence.

Shape: a workflow triggered on changes to pyproject.toml (the paths: filter idiom ci-tests.yml already uses at .github/workflows/ci-tests.yml:18) that parses [project] dependencies, queries https://api.anaconda.org/package/conda-forge/<name> for each, and checks whether the declared constraint is satisfiable against the versions conda-forge actually has. It fails, or comments on the PR, when one is missing or too old.

Two design points that change the cost:

  1. Deterministic script for the check, agent for the remediation. Parse, query, compare, report is cheap, runs on every PR, and has no judgement to get wrong β€” a script is the right tool. Choosing between a staged-recipes PR and a version bump on an existing feedstock, spotting that a release is wheel-only, noticing a requires_python floor moved β€” that is judgement, and the right tool is an agent. The repo already has agent-invoking workflows (claude.yml, claude-code-review.yml), so a failing check can hand off rather than trying to do everything itself.

  2. Name mapping is the part that will bite. PyPI and conda-forge names diverge, and this project's own recipe already carries the translation. Verified against the API right now:

    Dependency conda-forge query Result
    aws-lambda-builders .../conda-forge/aws-lambda-builders 404 β€” false "missing"
    aws-lambda-builders .../conda-forge/aws_lambda_builders 200 βœ…

    A naive check produces a false "missing package" failure on every such dependency. The script needs the mapping, or must treat a miss as "verify by hand" rather than a hard failure β€” otherwise the first implementation is noisy enough to get disabled.

The same job can also check for a missing sdist on the pinned version β€” the condition that silently defeats the conda-forge autotick bot, and what left cronsim-feedstock un-bumped since February 2025.

Why β€” the concrete cost

During #222, cronsim>=2.7 and tzdata were added as core runtime dependencies. Nobody checked conda-forge. This surfaced only after v0.14.0 was tagged, released on GitHub, and published to PyPI, at which point:

  • conda-forge had cronsim 2.6, and 2.6 is not API-compatible β€” CronSim does not expose the parsed field sets we read, so relaxing the constraint was not an option.
  • conda-forge/cronsim-feedstock had never been bumped since it was created in Feb 2025 β€” four commits, all infrastructure, zero PRs.
  • cronsim 2.7 publishes no sdist (wheel only), so the recipe's /packages/source/...tar.gz URL 404s β€” almost certainly why no automatic bump ever landed.
  • Fixing it needed a fresh feedstock PR moving the source to the GitHub tag archive and a local python_min: "3.10" pin, because cronsim also raised its floor from >=3.8 to >=3.10 in that release.

Net effect: the conda-forge package for v0.14.0 was blocked after the release had already shipped everywhere else, and unblocking it required work on a third-party feedstock. Checking at dependency-add time would have cost minutes and surfaced all four problems while there was still slack.

Evidence:

Refs #222

Acceptance Criteria

  • A rule file exists under .claude/rules/ covering the conda-forge check at dependency-add time
  • The rule gives the api.anaconda.org command and states explicitly that anaconda.org/conda-forge/<name> returns 200 for nonexistent packages
  • The rule requires checking the declared version constraint against versions, not just package existence
  • The rule requires checking for an upstream sdist
  • The rule distinguishes staged-recipes (new package) from a feedstock version-bump PR (existing package)
  • The rule is reachable from CLAUDE.md or the rules index the same way the other .claude/rules/ files are
  • A CI job runs on pull requests that modify pyproject.toml and checks every entry in [project] dependencies against api.anaconda.org
  • The CI check resolves PyPI β†’ conda-forge name differences, or reports an unresolved name as "verify by hand" rather than as a hard failure (verified against aws-lambda-builders β†’ aws_lambda_builders)
  • The CI check reports a dependency whose declared constraint is unsatisfiable on conda-forge, not merely one that is absent
  • The CI check reports a pinned version that publishes no sdist on PyPI

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area/ciCI/CD workflowsdocumentationImprovements or additions to documentation

    Type

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions