fix(sources): allow sdist hooks with source config

docs/concepts/hooks-and-overrides.rst

@@ -66,6 +66,16 @@ arguments.
    artifact kind. Package overrides remain active. See :ref:`prepare_source_hook`
    for their arguments and the public Git checkout helper.
 
+   With ``source:`` configured, set ``build_sdist: hook`` to use the package's
+   custom sdist builder. See :ref:`build_sdist_hook` for details.
+
+.. versionchanged:: 0.104.0
+
+   ``source.build_sdist: hook`` requires a callable package override. Fromager
+   rejects a missing or non-callable hook before resolving or downloading the
+   package. Hooks that delegate to ``default_build_sdist()`` exclude VCS
+   metadata by default when ``source:`` is configured.
+
 Global Hooks
 ------------
 

docs/concepts/package-settings.rst

@@ -35,6 +35,11 @@ Settings Loading Flow
 ``PackageBuildInfo`` objects are created lazily the first time a
 package is queried and cached for the rest of the run.
 
+.. versionchanged:: 0.104.0
+   With ``source.build_sdist: hook``, Fromager checks that the selected
+   variant's hook exists and can be called before resolving or downloading
+   the package. Hooks used only by other variants are not required.
+
 ``PackageSettings`` validates source settings when it loads each package.
 A package-level ``source`` cannot be combined with an explicitly set
 variant ``pre_built`` or ``wheel_server_url``. A variant-level ``source``

docs/concepts/resolver-architecture.rst

@@ -212,17 +212,32 @@ Building the Sdist
    Git checkouts can be prepared directly. Configured source resolvers
    also select the sdist build method.
 
-For source builds, the configured resolver selects one of two methods:
+For source builds, ``source.build_sdist`` selects the build method:
 
 - ``pep517`` calls the package's build backend. Git checkout resolvers
-  default to it.
+  default to it. Archive-based PyPI resolvers can select it explicitly.
 - ``tarball`` repacks the prepared source without VCS metadata such as
-  ``.git``. PyPI sdist resolvers default to it.
+  ``.git``. Archive-based PyPI resolvers default to it.
+- ``hook`` calls the package's custom ``build_sdist`` hook. It can generate
+  files or choose which directory to pack. A missing or non-callable hook is
+  rejected before resolution or download.
 
-``hook-sdist`` requires an explicit ``build_sdist: pep517`` or
-``build_sdist: tarball`` setting. A configured method takes precedence
-over a ``build_sdist`` override and logs a warning when the override is
-ignored; packages without ``source:`` keep using the override.
+``hook-sdist`` requires an explicit build method. Other profiles that build
+from source can also select ``hook``. The ``pep517`` and ``tarball`` methods
+skip installed build hooks with a warning. Without ``source:``, Fromager
+uses the hook when present, or the default otherwise.
+
+Hooks that delegate to ``default_build_sdist()`` inherit its default VCS
+exclusion for configured source profiles. An explicit ``exclude_vcs`` value
+overrides this policy. Hooks that create archives directly choose their contents.
+
+See :ref:`build_sdist_hook` for the arguments, archive rules, and an example.
+
+.. versionchanged:: 0.104.0
+
+   ``source.build_sdist: hook`` enables package overrides for configured
+   source profiles. ``default_build_sdist()`` excludes VCS metadata by default
+   for those profiles.
 
 Replaying a Build Order
 -----------------------

docs/proposals/new-resolver-config.md

@@ -174,6 +174,11 @@ hook and `source` settings, then at least one `source` setting (top-level or
 variant) must use `provider: hook-sdist` or `provider: hook-prebuilt`. The
 rule ensures that the hooks are used.
 
+Set `build_sdist: hook` under `source` to use the package's custom sdist
+builder with any profile that builds from source. A missing hook is an error.
+The `pep517` and `tarball` methods skip installed build hooks with a warning.
+Packages without `source` keep their existing build-hook behavior.
+
 ### git clone
 
 Git cloning uses the more efficient `git_clone_fast` function. Like pip's

docs/reference/config-reference.rst

@@ -76,6 +76,19 @@ Source Resolver
 
    .. autoattribute:: pep517
    .. autoattribute:: tarball
+   .. autoattribute:: hook
+
+.. versionadded:: 0.104.0
+
+   Set ``build_sdist: hook`` to use the package's custom sdist builder.
+   The hook must be installed and callable. This option is only available for
+   source builds. See :ref:`build_sdist_hook` for details and an example.
+
+.. versionchanged:: 0.104.0
+
+   ``pypi-sdist`` and ``pypi-download`` accept an explicit
+   ``build_sdist: pep517``. Their default remains ``tarball`` because some
+   build backends require VCS metadata that source archives do not contain.
 
 .. autoclass:: DownloadedSource
    :members: path, kind

docs/reference/hooks.rst

@@ -361,17 +361,55 @@ Source hooks
     .. versionchanged:: 0.103.0
        Passing ``DownloadKind.git_checkout`` prepares a clean Git checkout.
 
+.. _build_sdist_hook:
+
 .. autofromagerhook:: default_build_sdist
 
-    The ``build_sdist()`` function is responsible for creating a new source
-    distribution from the prepared source tree and placing it in
-    ``ctx.sdists_build``.  The dist name and version of the sdist file must
-    match the ``Requirement`` and ``Version``.
+    The ``build_sdist()`` hook packs prepared sources into an sdist and returns
+    its ``Path``. The file must exist in ``ctx.sdists_builds`` and match the
+    requested package name and version.
+
+    The hook receives ``ctx``, ``extra_environ``, ``req``, ``version``,
+    ``sdist_root_dir``, ``build_env``, and ``build_dir``. Build dependencies are
+    installed when the build pipeline prepares them. The standalone
+    ``step build-sdist`` command does not install dependencies; run
+    ``step prepare-build`` first. Use the hook to generate files or pack a
+    directory other than the wheel build directory.
+
+    .. versionchanged:: 0.104.0
+
+       With ``source:``, set ``build_sdist: hook`` to call this hook.
+       A missing or non-callable hook is rejected before resolution or
+       download. The ``pep517`` and ``tarball`` methods
+       skip installed hooks with a warning. Without ``source:``, Fromager
+       uses the hook when present, or the default otherwise.
+
+    For a package downloaded as a tarball:
+
+    .. code-block:: yaml
+
+       build_dir: python
+       source:
+         provider: pypi-download
+         download_url: https://sources.example/project-{version}.tar.gz
+         download_kind: tarball
+         build_sdist: hook
+
+    Any profile that builds from source can use this option. The
+    ``hook-sdist`` profile still needs a resolver or download hook.
+
+    .. versionchanged:: 0.104.0
+
+       ``default_build_sdist()`` now excludes VCS metadata such as ``.git``
+       by default when the effective package or variant has ``source:``
+       configured, including when called by a hook. Pass ``exclude_vcs=False``
+       to retain the previous behavior.
 
-    The arguments are the ``WorkContext``, the ``Requirement`` being evaluated, and the
-    `Path` to the root of the source tree.
+    Without ``source:``, the helper retains VCS metadata by default; pass
+    ``exclude_vcs=True`` to omit it.
 
-    The return value is the ``Path`` to the newly created source distribution.
+    Hooks that create archives directly choose their contents and must omit
+    VCS metadata themselves.
 
 Source helper functions
 ~~~~~~~~~~~~~~~~~~~~~~~

src/fromager/packagesettings/_pbi.py

@@ -21,6 +21,7 @@
     PurlConfig,
     VariantInfo,
 )
+from ._resolver import BuildSDist
 from ._templates import _resolve_template, substitute_template
 from ._typedefs import Annotations, PackageVersion, PatchMap, Template, Variant
 
@@ -52,6 +53,22 @@ def __init__(self, settings: Settings, ps: PackageSettings) -> None:
         self._plugin_module: types.ModuleType | typing.Literal[False] | None = False
         self._patches: PatchMap | None = None
         self._annotations: Annotations | None = None
+        self._validate_build_sdist_hook()
+
+    def _validate_build_sdist_hook(self) -> None:
+        """Check the selected variant's hook before resolution or download."""
+        source_resolver = self.source_resolver
+        if (
+            source_resolver is None
+            or source_resolver.build_sdist is not BuildSDist.hook
+        ):
+            return
+        hook = overrides.find_override_method(self.package, "build_sdist")
+        if not callable(hook):
+            raise ValueError(
+                f"{self.package}: source.build_sdist='hook' requires a callable "
+                "build_sdist override hook"
+            )
 
     @property
     def package(self) -> NormalizedName:

src/fromager/packagesettings/_resolver.py

@@ -28,8 +28,15 @@
 
 
 class BuildSDist(enum.StrEnum):
+    """Source distribution build method."""
+
     pep517 = "pep517"
     tarball = "tarball"
+    hook = "hook"
+    """Call the package's ``build_sdist`` override hook.
+
+    .. versionadded:: 0.104.0
+    """
 
 
 class DownloadKind(enum.StrEnum):
@@ -71,10 +78,14 @@ class AbstractResolver(pydantic.BaseModel):
     provider: str
 
     build_sdist: BuildSDist | None
-    """Sdist build strategy; None for resolvers that do not build from source."""
+    """Sdist build strategy; None for resolvers that do not build from source.
+
+    ``hook`` requires a package ``build_sdist`` override independently of
+    the profile's resolver and download hooks.
+    """
 
     supports_override_hooks: typing.ClassVar[bool] = False
-    """Does resolver support override hooks?"""
+    """Does resolver support resolution and download override hooks?"""
 
     download_kinds: typing.ClassVar[DownloadKindSet] = frozenset()
     """Set of download kinds this resolver can return."""
@@ -248,9 +259,11 @@ class PyPISDistResolver(AbstractPyPIResolver):
 
     provider: typing.Literal["pypi-sdist"]
 
-    # It is not safe to use PEP 517 to re-generate a source distribution.
-    # Some PEP 517 backends require VCS to generate correct sdist.
-    build_sdist: typing.Literal[BuildSDist.tarball] = BuildSDist.tarball
+    # Tarball repacking is the safe default because some PEP 517 backends
+    # require VCS metadata to generate a correct sdist.
+    build_sdist: typing.Literal[
+        BuildSDist.pep517, BuildSDist.tarball, BuildSDist.hook
+    ] = BuildSDist.tarball
     download_kinds: typing.ClassVar[DownloadKindSet] = frozenset({DownloadKind.sdist})
 
     def resolver_provider(
@@ -362,7 +375,9 @@ class PyPIDownloadResolver(AbstractPyPIResolver):
     URL must contain '{version}' template string.
     """
 
-    build_sdist: typing.Literal[BuildSDist.tarball] = BuildSDist.tarball
+    build_sdist: typing.Literal[
+        BuildSDist.pep517, BuildSDist.tarball, BuildSDist.hook
+    ] = BuildSDist.tarball
     download_kinds: typing.ClassVar[DownloadKindSet] = frozenset(
         {DownloadKind.sdist, DownloadKind.tarball}
     )
@@ -808,7 +823,7 @@ class AbstractHookResolver(AbstractResolver, CooldownMixin):
     """Abstract base class for hook-based resolvers"""
 
     supports_override_hooks: typing.ClassVar[bool] = True
-    """Hook resolvers support override hooks."""
+    """Hook resolvers support resolution and download override hooks."""
 
     hookless_provider: typing.ClassVar[str]
     """Profile to suggest for packages without hooks."""
@@ -953,7 +968,8 @@ class HookSDistResolver(AbstractHookResolver):
     ``Path`` treated as an sdist.
     Without a download hook, the resolved URL is downloaded as an sdist.
     Specify ``build_sdist`` explicitly: ``tarball`` to repack an existing
-    sdist, or ``pep517`` to build from a git checkout.
+    sdist, ``pep517`` to build from a git checkout, or ``hook`` to call
+    the package's ``build_sdist`` override.
 
     Example::
 

src/fromager/sources.py

@@ -624,58 +624,65 @@ def build_sdist(
     sdist_root_dir: pathlib.Path,
     build_env: build_environment.BuildEnvironment,
 ) -> pathlib.Path:
-    """Build a source distribution using the configured strategy or legacy hook."""
+    """Build a source distribution using the configured strategy or legacy hook.
+
+    ``source.build_sdist: hook`` requires a package override. Configured
+    ``pep517`` and ``tarball`` strategies take precedence over overrides.
+
+    .. versionchanged:: 0.104.0
+       Configured source profiles can select the ``build_sdist`` override.
+    """
     pbi = ctx.package_build_info(req)
     source_resolver = pbi.source_resolver
+    strategy = source_resolver.build_sdist if source_resolver is not None else None
     build_dir = pbi.build_dir(sdist_root_dir)
 
     logger.info(f"building {ctx.variant} source distribution for {req} in {build_dir}")
+    if source_resolver is not None:
+        build_sdist_override = overrides.find_override_method(req.name, "build_sdist")
+        if strategy is packagesettings.BuildSDist.hook:
+            if not callable(build_sdist_override):
+                raise ValueError(
+                    f"{req.name}: source.build_sdist='hook' requires a callable "
+                    "build_sdist override hook"
+                )
+        elif build_sdist_override is not None:
+            logger.warning(
+                "%s: ignoring build_sdist override because source.build_sdist is %s",
+                req.name,
+                strategy,
+            )
     extra_environ = packagesettings.get_extra_environ(
         ctx=ctx,
         req=req,
         version=version,
         sdist_root_dir=sdist_root_dir,
         build_env=build_env,
     )
-    if source_resolver is not None:
-        if overrides.find_override_method(req.name, "build_sdist") is not None:
-            logger.warning(
-                "%s: ignoring build_sdist override because source.build_sdist is %s",
-                req.name,
-                source_resolver.build_sdist,
-            )
-    if (
-        source_resolver is not None
-        and source_resolver.build_sdist is packagesettings.BuildSDist.pep517
-    ):
-        sdist_file = pep517_build_sdist(
+    if source_resolver is None or strategy is packagesettings.BuildSDist.hook:
+        sdist_file: pathlib.Path = overrides.find_and_invoke(
+            req.name,
+            "build_sdist",
+            default_build_sdist,
             ctx=ctx,
             extra_environ=extra_environ,
             req=req,
             version=version,
             sdist_root_dir=sdist_root_dir,
+            build_dir=build_dir,
             build_env=build_env,
         )
-    elif (
-        source_resolver is not None
-        and source_resolver.build_sdist is packagesettings.BuildSDist.tarball
-    ):
-        sdist_file = default_build_sdist(
+    elif strategy is packagesettings.BuildSDist.pep517:
+        sdist_file = pep517_build_sdist(
             ctx=ctx,
             extra_environ=extra_environ,
             req=req,
             version=version,
             sdist_root_dir=sdist_root_dir,
-            build_dir=build_dir,
             build_env=build_env,
-            # Git checkouts keep their `.git`; keep it out of the sdist.
-            exclude_vcs=True,
         )
-    elif source_resolver is None:
-        sdist_file = overrides.find_and_invoke(
-            req.name,
-            "build_sdist",
-            default_build_sdist,
+    elif strategy is packagesettings.BuildSDist.tarball:
+        sdist_file = default_build_sdist(
             ctx=ctx,
             extra_environ=extra_environ,
             req=req,
@@ -686,8 +693,7 @@ def build_sdist(
         )
     else:
         raise ValueError(
-            f"{req.name}: unsupported source.build_sdist strategy "
-            f"{source_resolver.build_sdist!r}"
+            f"{req.name}: unsupported source.build_sdist strategy {strategy!r}"
         )
     logger.info(f"built source distribution {sdist_file}")
 
@@ -713,12 +719,20 @@ def default_build_sdist(
     build_env: build_environment.BuildEnvironment,
     build_dir: pathlib.Path,
     *,
-    exclude_vcs: bool = False,
+    exclude_vcs: bool | None = None,
 ) -> pathlib.Path:
     """Repack *build_dir* as an sdist.
 
-    With *exclude_vcs*, VCS metadata such as ``.git`` is left out.
+    With *exclude_vcs*, VCS metadata such as ``.git`` is left out. By default,
+    exclude it when ``source:`` is configured, and retain it otherwise.
+    Explicit booleans override this policy.
+
+    .. versionchanged:: 0.104.0
+       With ``exclude_vcs=None``, exclude VCS metadata when the effective
+       package or variant has ``source:`` configured; retain it otherwise.
     """
+    if exclude_vcs is None:
+        exclude_vcs = ctx.package_build_info(req).source_resolver is not None
     # It seems like the "correct" way to do this would be to run the
     # PEP 517 API in the source tree we have modified. However, quite
     # a few packages assume their source distribution is being built