From 4aeae7369fea9f437b9f00da39e7501e6a9ae109 Mon Sep 17 00:00:00 2001 From: rochala Date: Mon, 19 Oct 2026 13:41:10 +0200 Subject: [PATCH 1/3] Document private repository credentials cellar reads only coursier's credential sources; document where they live, the formats, and how to carry over an sbt .credentials file, instead of reading other tools' credentials. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01ETVFFSQUFsAxXzjVsFwthR --- README.md | 53 ++++++++++++++++++++++++++++++++++++++++++++++++----- 1 file changed, 48 insertions(+), 5 deletions(-) diff --git a/README.md b/README.md index 1797f20..fa30b02 100644 --- a/README.md +++ b/README.md @@ -186,10 +186,8 @@ cellar get-external org.typelevel:cats-core_3:latest cats.Monad | `-v`, `--verbose` | all except `telemetry` | Log progress and warnings to stderr | | `--debug` | all except `telemetry` | Log detailed diagnostics and stack traces to stderr | -Private repositories: credentials from `~/.config/coursier/credentials.properties` (or -`COURSIER_CONFIG_DIR`) and the `COURSIER_CREDENTIALS` environment variable are read the same way -the coursier CLI, sbt and Mill read them, and applied to every repository whose host matches. -See the [coursier credentials docs](https://get-coursier.io/docs/other-credentials). +Private repositories need credentials in coursier's format — see +[Private repositories](#private-repositories). Diagnostics are written to stderr, never stdout, so `--verbose` and `--debug` are safe to use when piping output into a prompt. Set `CELLAR_LOG=verbose` or `CELLAR_LOG=debug` to get the same @@ -320,10 +318,55 @@ maven { Maven Central and Ivy local stay available, and CLI `-r` values are appended to the configured list. Lists replace rather than merge, so a project-level `maven.repositories` overrides the user-level list and `repositories = []` clears it for that project. Configuration holds URLs only — -credentials come from coursier, as described above. +credentials come from coursier, see [Private repositories](#private-repositories). Or via environment: `CELLAR_SBT_BINARY=sbtn cellar get --module core cats.Monad` +### Private repositories + +Project-aware commands get the classpath from your build tool, so they use whatever credentials +the build already has. + +External commands resolve with coursier and read only coursier's credentials — the same ones the +coursier CLI, scala-cli and Metals use. cellar never reads another tool's credentials +(`~/.sbt/.credentials`, `~/.m2/settings.xml`, `~/.gradle/gradle.properties`). Coursier looks in: + +1. `COURSIER_CREDENTIALS` (or `-Dcoursier.credentials`): inline entries, or a path to a + properties file. When set, the files in 2. are ignored. +2. `credentials.properties` and `credentials/*.properties` in the coursier config directory: + `~/.config/coursier` on Linux, `~/Library/Application Support/Coursier` on macOS, or + `COURSIER_CONFIG_DIR`. +3. `repositories.credentials` in the scala-cli config (`scala-cli config repositories.credentials …`). + +`credentials.properties`: + +```properties +company.host=artifactory.company.com +company.username=alice +company.password=s3cret +# Optional: +company.realm=Artifactory Realm +company.https-only=true +``` + +Inline, one entry per line: `artifactory.company.com alice:s3cret`, or +`artifactory.company.com(Artifactory Realm) alice:s3cret` with a realm. + +Credentials apply to every request to a matching host, but are sent only after the server +answers `401`; a repository that rejects anonymous requests some other way never gets them. + +Coming from sbt, an `~/.sbt/.credentials` file maps one-to-one: + +| sbt `.credentials` | coursier `credentials.properties` | +|--------------------|-----------------------------------| +| `realm=…` | `.realm=…` | +| `host=…` | `.host=…` | +| `user=…` | `.username=…` | +| `password=…` | `.password=…` | + +A failed resolution lists every location coursier tried; a missing or wrong credential shows there +as `unauthorized: ()`. + Trust a TLS-intercepting proxy's CA. The native binary ignores `JAVA_TOOL_OPTIONS`, so set the trust store in `~/.cellar/cellar.conf` (same format as scala-cli's `java.properties`): From 915589d4548fafc8345a5c01aa3ffe95a00a64cb Mon Sep 17 00:00:00 2001 From: rochala Date: Tue, 20 Oct 2026 22:35:03 +0200 Subject: [PATCH 2/3] Reject 203 sign-in pages and honor credentials in repository URLs Coursier holds file/env credentials back until a server rejects an anonymous request with a 4xx. Azure Artifacts answers with 203 and a sign-in page instead, which coursier cached as the POM: credentials were never sent and the cache stayed broken with a checksum error. - Route every coursier call through one FileCache with rejectNonAuthoritativeResponses, so a 203 fails cleanly. - Parse -r / maven.repositories URLs with coursier's RepositoryParser: user:password in the URL becomes repository authentication, sent up front (HTTPS-only unless the URL is http). Error messages mask it. - README: correct the retry rule (any 4xx, not just 401), document URL credentials with ${ENV} substitution, and a troubleshooting table. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01ETVFFSQUFsAxXzjVsFwthR --- README.md | 49 +++++++++++++++---- cli/src/cellar/cli/RepositoryUrl.scala | 21 ++++++-- .../src/cellar/cli/RepositoryUrlTest.scala | 17 +++++++ lib/src/cellar/CoordinateCompleter.scala | 4 +- lib/src/cellar/CoursierCache.scala | 14 ++++++ lib/src/cellar/CoursierFetchClient.scala | 4 +- lib/src/cellar/CoursierResolveClient.scala | 2 +- lib/src/cellar/DocstringExtractor.scala | 2 +- lib/src/cellar/MavenCoordinate.scala | 2 +- lib/src/cellar/SourcesResolver.scala | 5 +- 10 files changed, 96 insertions(+), 24 deletions(-) create mode 100644 lib/src/cellar/CoursierCache.scala diff --git a/README.md b/README.md index fa30b02..e3c948c 100644 --- a/README.md +++ b/README.md @@ -317,8 +317,8 @@ maven { Maven Central and Ivy local stay available, and CLI `-r` values are appended to the configured list. Lists replace rather than merge, so a project-level `maven.repositories` overrides the user-level -list and `repositories = []` clears it for that project. Configuration holds URLs only — -credentials come from coursier, see [Private repositories](#private-repositories). +list and `repositories = []` clears it for that project. For credentials, see +[Private repositories](#private-repositories). Or via environment: `CELLAR_SBT_BINARY=sbtn cellar get --module core cats.Monad` @@ -327,9 +327,13 @@ Or via environment: `CELLAR_SBT_BINARY=sbtn cellar get --module core cats.Monad` Project-aware commands get the classpath from your build tool, so they use whatever credentials the build already has. -External commands resolve with coursier and read only coursier's credentials — the same ones the -coursier CLI, scala-cli and Metals use. cellar never reads another tool's credentials -(`~/.sbt/.credentials`, `~/.m2/settings.xml`, `~/.gradle/gradle.properties`). Coursier looks in: +External commands resolve with coursier and use coursier's credentials — the same ones the +coursier CLI, scala-cli and Metals read. cellar never reads another tool's credentials +(`~/.sbt/.credentials`, `~/.m2/settings.xml`, `~/.gradle/gradle.properties`). + +#### Credential files and environment + +Coursier looks in: 1. `COURSIER_CREDENTIALS` (or `-Dcoursier.credentials`): inline entries, or a path to a properties file. When set, the files in 2. are ignored. @@ -352,8 +356,9 @@ company.https-only=true Inline, one entry per line: `artifactory.company.com alice:s3cret`, or `artifactory.company.com(Artifactory Realm) alice:s3cret` with a realm. -Credentials apply to every request to a matching host, but are sent only after the server -answers `401`; a repository that rejects anonymous requests some other way never gets them. +These credentials apply to every repository on a matching host, but are held back until the +server rejects an anonymous request with a `4xx` (`401`, `403` or `404`). Most servers do; Azure +Artifacts is the known exception, see below. Coming from sbt, an `~/.sbt/.credentials` file maps one-to-one: @@ -364,8 +369,34 @@ Coming from sbt, an `~/.sbt/.credentials` file maps one-to-one: | `user=…` | `.username=…` | | `password=…` | `.password=…` | -A failed resolution lists every location coursier tried; a missing or wrong credential shows there -as `unauthorized: ()`. +#### Credentials in the repository URL + +Azure Artifacts answers anonymous requests with `203` and a sign-in page rather than a `4xx`, so +held-back credentials never reach it. Put them in the repository URL instead; they are then sent +with every request to that repository: + +```hocon +maven { + repositories = ["https://user:"${AZURE_ARTIFACTS_TOKEN}"@pkgs.dev.azure.com/org/_packaging/feed/maven/v1"] +} +``` + +`${…}` is read from the environment, which keeps the token out of the file. The same URL works +with `-r`. URL credentials are sent over HTTPS only, unless the URL itself is `http://`, and cellar +masks the password in its own messages. + +#### Troubleshooting + +A failed resolution lists every location coursier tried: + +| Shown under `Tried:` | Meaning | +|---------------------------------------------------|------------------------------------------------------------| +| `unauthorized: ()` | No credentials for that host, or the server rejected them | +| `forbidden: ` | No credentials for that host, or they lack access | +| `non-authoritative information: (HTTP 203)` | The server wants credentials up front — put them in the URL | + +To see which credentials coursier sends to each URL, set `COURSIER_HTTP_DEBUG=true`. It names the +user, never the password. Trust a TLS-intercepting proxy's CA. The native binary ignores `JAVA_TOOL_OPTIONS`, so set the trust store in `~/.cellar/cellar.conf` (same format as scala-cli's `java.properties`): diff --git a/cli/src/cellar/cli/RepositoryUrl.scala b/cli/src/cellar/cli/RepositoryUrl.scala index f92cff6..040e622 100644 --- a/cli/src/cellar/cli/RepositoryUrl.scala +++ b/cli/src/cellar/cli/RepositoryUrl.scala @@ -1,6 +1,7 @@ package cellar.cli import coursier.MavenRepository +import coursier.parse.RepositoryParser import java.net.URI import scala.util.Try @@ -8,6 +9,9 @@ import scala.util.Try /** `MavenRepository(...)` accepts any string verbatim, so a typo such as a missing scheme survives * until resolution fails — and it then fails as a coordinate error, pointing at the coordinate * rather than at the repository. Rejecting the URL up front keeps the blame where it belongs. + * + * Credentials in the URL (`https://user:token@host/...`) become the repository's own and are sent + * with the first request; error messages mask the password. */ object RepositoryUrl: @@ -18,18 +22,25 @@ object RepositoryUrl: /** Left carries the reason only; the caller prefixes it with where the URL came from. */ def parse(raw: String): Either[String, MavenRepository] = val trimmed = raw.trim + val shown = masked(trimmed) if trimmed.isEmpty then Left("the URL is empty") else Try(URI(trimmed)).toEither.left - .map(_ => s"'$raw' is not a valid URL") + .map(_ => s"'$shown' is not a valid URL") .flatMap { uri => Option(uri.getScheme).map(_.toLowerCase) match case None => - Left(s"'$raw' has no scheme (expected one of $expected)") + Left(s"'$shown' has no scheme (expected one of $expected)") case Some(scheme) if !supportedSchemes.contains(scheme) => - Left(s"'$raw' has unsupported scheme '$scheme' (expected one of $expected)") + Left(s"'$shown' has unsupported scheme '$scheme' (expected one of $expected)") case Some(scheme) if scheme != "file" && Option(uri.getHost).forall(_.isEmpty) => - Left(s"'$raw' has no host") + Left(s"'$shown' has no host") case Some(_) => - Right(MavenRepository(trimmed)) + RepositoryParser.repository(trimmed).left.map(_ => s"'$shown' is not a valid URL").flatMap { + case maven: MavenRepository => Right(maven) + case _ => Left(s"'$shown' is not a Maven repository URL") + } } + + private def masked(url: String): String = + url.replaceFirst("^([a-zA-Z][a-zA-Z0-9+.-]*://[^:/@]*):[^@/]*@", "$1:****@") diff --git a/cli/test/src/cellar/cli/RepositoryUrlTest.scala b/cli/test/src/cellar/cli/RepositoryUrlTest.scala index 6215f70..232fef1 100644 --- a/cli/test/src/cellar/cli/RepositoryUrlTest.scala +++ b/cli/test/src/cellar/cli/RepositoryUrlTest.scala @@ -38,3 +38,20 @@ object RepositoryUrlTest extends FunSuite: test("rejects a string that is not a URL at all"): reason("not a url")(message => expect(message.contains("not a valid URL"))) + + test("moves URL credentials onto the repository, sent up front and over https only"): + RepositoryUrl.parse("https://alice:s3cret@pkgs.example.com/maven").fold( + err => failure(s"expected the URL to be accepted, got: $err"), + repo => + repo.authentication.fold(failure("expected credentials")) { auth => + expect.eql("https://pkgs.example.com/maven", repo.root) and + expect.eql((Some("alice"), Some("s3cret")), (auth.userOpt, auth.passwordOpt)) and + expect(!auth.optional && auth.httpsOnly) + } + ) + + test("never echoes a URL password in an error"): + reason("ftp://alice:s3cret@pkgs.example.com/maven") { message => + expect(message.contains("ftp://alice:****@pkgs.example.com/maven"), message) and + expect(!message.contains("s3cret"), message) + } diff --git a/lib/src/cellar/CoordinateCompleter.scala b/lib/src/cellar/CoordinateCompleter.scala index 1c7b710..237b918 100644 --- a/lib/src/cellar/CoordinateCompleter.scala +++ b/lib/src/cellar/CoordinateCompleter.scala @@ -8,11 +8,11 @@ object CoordinateCompleter: def suggest(coord: MavenCoordinate, extraRepos: Seq[Repository]): IO[List[String]] = IO.blocking { def release(artifact: String): Option[String] = - Some(Versions().withModule(coord.copy(artifact = artifact).toCoursierModule).addRepositories(extraRepos*).run()) + Some(Versions(CoursierCache.instance).withModule(coord.copy(artifact = artifact).toCoursierModule).addRepositories(extraRepos*).run()) .map(_.release0.asString) .filter(_.nonEmpty) - val complete = Complete().addRepositories(extraRepos*).copy(input = s"${coord.group}:${coord.artifact}") + val complete = Complete(CoursierCache.instance).addRepositories(extraRepos*).copy(input = s"${coord.group}:${coord.artifact}") val completions = complete.complete().unsafeRun()(using complete.cache.ec)._2.toList val exactMatch = completions.contains(coord.artifact) diff --git a/lib/src/cellar/CoursierCache.scala b/lib/src/cellar/CoursierCache.scala new file mode 100644 index 0000000..65c117f --- /dev/null +++ b/lib/src/cellar/CoursierCache.scala @@ -0,0 +1,14 @@ +package cellar + +import coursier.cache.FileCache +import coursier.util.Task + +private[cellar] object CoursierCache: + + /** The cache every coursier call shares. + * + * Some feeds (Azure Artifacts) answer an anonymous request with `203` and a sign-in page instead + * of challenging it. Accepted, that page would be cached as the artifact and break every later + * lookup, so it fails the download instead. + */ + lazy val instance: FileCache[Task] = FileCache().copy(rejectNonAuthoritativeResponses = true) diff --git a/lib/src/cellar/CoursierFetchClient.scala b/lib/src/cellar/CoursierFetchClient.scala index f24bff9..eb5d58d 100644 --- a/lib/src/cellar/CoursierFetchClient.scala +++ b/lib/src/cellar/CoursierFetchClient.scala @@ -19,7 +19,7 @@ object CoursierFetchClient: )(using logger: Logger[IO] = StderrLogger.off): IO[Option[Path]] = logAttempt(coord, "fetching POM for", extraRepositories) *> IO.blocking { - Fetch() + Fetch(CoursierCache.instance) .addDependencies(coord.toCoursierDependency.copy(transitive = false)) .addRepositories(extraRepositories*) .run() @@ -58,7 +58,7 @@ object CoursierFetchClient: tracer.span("coursier.fetch").surround { logAttempt(coord, "resolving", extraRepositories) *> IO.blocking { - val files = Fetch() + val files = Fetch(CoursierCache.instance) .addDependencies(coord.toCoursierDependency) .addRepositories(extraRepositories*) .addClassifiers(Classifier.sources) diff --git a/lib/src/cellar/CoursierResolveClient.scala b/lib/src/cellar/CoursierResolveClient.scala index 795021b..9f5fc20 100644 --- a/lib/src/cellar/CoursierResolveClient.scala +++ b/lib/src/cellar/CoursierResolveClient.scala @@ -14,7 +14,7 @@ object CoursierResolveClient: )(using logger: Logger[IO] = StderrLogger.off): IO[ResolvedDeps] = logger.debug(s"resolving dependency tree for ${coord.render}") *> IO.blocking { - val resolution = Resolve() + val resolution = Resolve(CoursierCache.instance) .addDependencies(coord.toCoursierDependency) .addRepositories(extraRepositories*) .run() diff --git a/lib/src/cellar/DocstringExtractor.scala b/lib/src/cellar/DocstringExtractor.scala index 4ac0498..682d1bc 100644 --- a/lib/src/cellar/DocstringExtractor.scala +++ b/lib/src/cellar/DocstringExtractor.scala @@ -20,7 +20,7 @@ object DocstringExtractor: // yields "" and the fetch below asks for `scala3-library_3:` with no version at all. // Requires the Scala 3 stdlib (3.8+), where `library.properties` carries a 3.x version. val scalaVersion = scala.util.Properties.versionNumberString - coursier.Fetch() + coursier.Fetch(CoursierCache.instance) .addDependencies(MavenCoordinate("org.scala-lang", "scala3-library_3", scalaVersion).toCoursierDependency) .run() .map(file => Path.fromNioPath(file.toPath)) diff --git a/lib/src/cellar/MavenCoordinate.scala b/lib/src/cellar/MavenCoordinate.scala index d69f4c9..d89db77 100644 --- a/lib/src/cellar/MavenCoordinate.scala +++ b/lib/src/cellar/MavenCoordinate.scala @@ -13,7 +13,7 @@ final case class MavenCoordinate(group: String, artifact: String, version: Strin def resolveLatest(extraRepos: Seq[Repository] = Seq.empty): IO[MavenCoordinate] = if version != "latest" then IO.pure(this) else IO.blocking { - val release = Versions().withModule(toCoursierModule).addRepositories(extraRepos*).run().release0.asString + val release = Versions(CoursierCache.instance).withModule(toCoursierModule).addRepositories(extraRepos*).run().release0.asString if release.nonEmpty then copy(version = release) else throw CellarError.CoordinateNotFound(this, new RuntimeException(s"No versions found for '$group:$artifact'")) } diff --git a/lib/src/cellar/SourcesResolver.scala b/lib/src/cellar/SourcesResolver.scala index c8477d4..ec45137 100644 --- a/lib/src/cellar/SourcesResolver.scala +++ b/lib/src/cellar/SourcesResolver.scala @@ -1,6 +1,5 @@ package cellar -import coursier.cache.FileCache import coursier.util.Artifact import coursier.{Classifier, Fetch, Repository} import fs2.io.file.Path @@ -32,7 +31,7 @@ object SourcesResolver: * jar is the same URL with the `-sources.jar` suffix, whichever repository served it. */ private def fromCoursierCache(jar: Path): Option[Path] = - val cache = FileCache() + val cache = CoursierCache.instance val root = Path.fromNioPath(cache.location.toPath) val relative = root.toNioPath.relativize(jar.toNioPath) Option.when(jar.toNioPath.startsWith(root.toNioPath) && relative.getNameCount > 2) { @@ -48,7 +47,7 @@ object SourcesResolver: private def fromPomProperties(jar: Path, repositories: Seq[Repository]): Option[Path] = coordinate(jar).flatMap { (group, artifact, version) => val dependency = MavenCoordinate(group, artifact, version).toCoursierDependency.copy(transitive = false) - Fetch() + Fetch(CoursierCache.instance) .addDependencies(dependency) .addRepositories(repositories*) .addClassifiers(Classifier.sources) From 4371410087182ac0033c66b0a38c3107a6a39f11 Mon Sep 17 00:00:00 2001 From: rochala Date: Wed, 21 Oct 2026 17:43:10 +0200 Subject: [PATCH 3/3] Proxy authentication, mirror-aware version lookups, review fixes - Drop the global 203 rejection: coursier documents most 203s as a proxy's transformed body, and a sign-in page already fails on its checksum (and is cached per user, so authenticated runs never see it). - Proxy credentials: install coursier's proxy Authenticator from http(s).proxyUser/proxyPassword, default proxyProtocol to http so an https.proxy*-only setup is answered, and allow Basic for HTTPS tunnels in the native image, where that setting is fixed at build time. - :latest and "did you mean" suggestions go through the same mirrors as the fetch (scala-cli and Maven settings mirrors included). - Mask everything between scheme and last @ in repository URL errors; explain that credentials with / ? # @ cannot go in a URL. - A duplicate repository carrying credentials wins over one without. - README: credential sources incl. Maven settings mirrors, ${?ENV}, per-project credentials, proxies, troubleshooting. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01ETVFFSQUFsAxXzjVsFwthR --- README.md | 81 ++++++++----- build.mill | 6 +- cli/src/cellar/cli/CellarApp.scala | 2 +- cli/src/cellar/cli/JvmProperties.scala | 23 ++++ cli/src/cellar/cli/RepositoryUrl.scala | 26 +++-- .../cellar/cli/PrivateRepositoryTest.scala | 110 ++++++++++++++++++ .../src/cellar/cli/RepositoryUrlTest.scala | 15 ++- lib/src/cellar/CoordinateCompleter.scala | 12 +- lib/src/cellar/CoursierCache.scala | 14 --- lib/src/cellar/CoursierFetchClient.scala | 4 +- lib/src/cellar/CoursierResolveClient.scala | 2 +- lib/src/cellar/DocstringExtractor.scala | 2 +- lib/src/cellar/ExtraRepositories.scala | 10 +- lib/src/cellar/MavenCoordinate.scala | 26 ++++- lib/src/cellar/SourcesResolver.scala | 5 +- .../src/cellar/ExtraRepositoriesTest.scala | 10 ++ 16 files changed, 273 insertions(+), 75 deletions(-) create mode 100644 cli/src/cellar/cli/JvmProperties.scala create mode 100644 cli/test/src/cellar/cli/PrivateRepositoryTest.scala delete mode 100644 lib/src/cellar/CoursierCache.scala diff --git a/README.md b/README.md index e3c948c..3bdfbf4 100644 --- a/README.md +++ b/README.md @@ -327,20 +327,22 @@ Or via environment: `CELLAR_SBT_BINARY=sbtn cellar get --module core cats.Monad` Project-aware commands get the classpath from your build tool, so they use whatever credentials the build already has. -External commands resolve with coursier and use coursier's credentials — the same ones the -coursier CLI, scala-cli and Metals read. cellar never reads another tool's credentials -(`~/.sbt/.credentials`, `~/.m2/settings.xml`, `~/.gradle/gradle.properties`). +External commands resolve with coursier and use exactly the credentials coursier reads — the same +ones the coursier CLI and scala-cli use. cellar adds no sources of its own and never reads +`~/.sbt/.credentials` or Gradle properties. -#### Credential files and environment +#### Where coursier finds credentials -Coursier looks in: - -1. `COURSIER_CREDENTIALS` (or `-Dcoursier.credentials`): inline entries, or a path to a +1. `COURSIER_CREDENTIALS` (or `-Dcoursier.credentials`): inline entries, or an absolute path to a properties file. When set, the files in 2. are ignored. 2. `credentials.properties` and `credentials/*.properties` in the coursier config directory: `~/.config/coursier` on Linux, `~/Library/Application Support/Coursier` on macOS, or `COURSIER_CONFIG_DIR`. 3. `repositories.credentials` in the scala-cli config (`scala-cli config repositories.credentials …`). +4. Mirrors in Maven's `~/.m2/settings.xml`, with the credentials of the `` sharing the + mirror's `id`. `COURSIER_MAVEN_SETTINGS=false` + turns this off; `COURSIER_MAVEN_SETTINGS=/path/to/settings.xml` picks another file. +5. Credentials in a repository URL, see [below](#credentials-in-the-repository-url). `credentials.properties`: @@ -356,9 +358,9 @@ company.https-only=true Inline, one entry per line: `artifactory.company.com alice:s3cret`, or `artifactory.company.com(Artifactory Realm) alice:s3cret` with a realm. -These credentials apply to every repository on a matching host, but are held back until the +Credentials from 1.–3. apply to every repository on a matching host, but are held back until the server rejects an anonymous request with a `4xx` (`401`, `403` or `404`). Most servers do; Azure -Artifacts is the known exception, see below. +Artifacts is the known exception. Credentials from 4. and 5. are sent with the first request. Coming from sbt, an `~/.sbt/.credentials` file maps one-to-one: @@ -372,44 +374,67 @@ Coming from sbt, an `~/.sbt/.credentials` file maps one-to-one: #### Credentials in the repository URL Azure Artifacts answers anonymous requests with `203` and a sign-in page rather than a `4xx`, so -held-back credentials never reach it. Put them in the repository URL instead; they are then sent -with every request to that repository: +held-back credentials never reach it. Put them in the repository URL instead: ```hocon maven { - repositories = ["https://user:"${AZURE_ARTIFACTS_TOKEN}"@pkgs.dev.azure.com/org/_packaging/feed/maven/v1"] + repositories = ["https://user:"${?AZURE_ARTIFACTS_TOKEN}"@pkgs.dev.azure.com/org/_packaging/feed/maven/v1"] } ``` -`${…}` is read from the environment, which keeps the token out of the file. The same URL works -with `-r`. URL credentials are sent over HTTPS only, unless the URL itself is `http://`, and cellar -masks the password in its own messages. +`${?…}` reads the environment, so the token stays out of the file and commands keep working where +it is unset. The same URL works with `-r`. URL credentials are sent over HTTPS only, unless the +URL itself is `http://`, and cellar never prints them. They cannot contain `/`, `?`, `#` or `@` +(coursier takes them verbatim, so percent-encoding does not help); use a credentials file then. -#### Troubleshooting +#### Per-project credentials -A failed resolution lists every location coursier tried: +Coursier never looks for credentials in the project directory. Either commit the URL form above +to `.cellar/cellar.conf`, with the token taken from the environment, or point +`COURSIER_CREDENTIALS` at a file outside version control — an absolute path, as anything else is +read as inline entries: -| Shown under `Tried:` | Meaning | -|---------------------------------------------------|------------------------------------------------------------| -| `unauthorized: ()` | No credentials for that host, or the server rejected them | -| `forbidden: ` | No credentials for that host, or they lack access | -| `non-authoritative information: (HTTP 203)` | The server wants credentials up front — put them in the URL | +```sh +export COURSIER_CREDENTIALS="$PWD/.coursier-credentials.properties" # e.g. from .envrc; gitignore it +``` -To see which credentials coursier sends to each URL, set `COURSIER_HTTP_DEBUG=true`. It names the -user, never the password. +#### Proxies -Trust a TLS-intercepting proxy's CA. The native binary ignores `JAVA_TOOL_OPTIONS`, so set the -trust store in `~/.cellar/cellar.conf` (same format as scala-cli's `java.properties`): +Proxy settings are JVM system properties. The native binary ignores `JAVA_TOOL_OPTIONS`, so set +them in `~/.cellar/cellar.conf` (same format as scala-cli's `java.properties`): ```hocon java { properties = [ - "javax.net.ssl.trustStore=/path/to/cacerts", - "javax.net.ssl.trustStorePassword=changeit" + "https.proxyHost=proxy.company.com", "https.proxyPort=3128", + "http.proxyHost=proxy.company.com", "http.proxyPort=3128", + "http.nonProxyHosts=localhost|*.company.com", + # Only for a proxy that asks for credentials: + "https.proxyUser=alice", "https.proxyPassword="${?PROXY_PASSWORD}, + # Only for a TLS-intercepting proxy: + "javax.net.ssl.trustStore=/path/to/cacerts", "javax.net.ssl.trustStorePassword=changeit" ] } ``` +To go through a company mirror instead of Maven Central, configure the mirror in coursier +(`mirror.properties`, see the [coursier docs](https://get-coursier.io/docs/other-proxy)) or in +Maven's `settings.xml`. cellar does not read proxy settings from `settings.xml`. + +#### Troubleshooting + +A failed resolution lists every location coursier tried: + +| Shown under `Tried:` | Meaning | +|---------------------------------------------|---------------------------------------------------------------------------| +| `unauthorized: ()` | No credentials for that host, or the server rejected them | +| `forbidden: ` | No credentials for that host, or they lack access | +| `checksum format error: …` | The server sent a page instead of the artifact (e.g. Azure Artifacts' sign-in page): [put the credentials in the URL](#credentials-in-the-repository-url) | +| `… 407 …` / `Unable to tunnel through proxy` | The proxy wants credentials: set `http(s).proxyUser` and `http(s).proxyPassword` | + +To see which credentials coursier sends to each URL, set `COURSIER_HTTP_DEBUG=true`. It names the +user, never the password. + ## Telemetry Cellar collects **anonymous usage telemetry** to help improve the tool. It is opt-in. On first use cellar asks you to choose before running: diff --git a/build.mill b/build.mill index 5e863b5..0878dd0 100644 --- a/build.mill +++ b/build.mill @@ -205,6 +205,9 @@ object cli extends CellarPublishModule with NativeImageModule { // // GC-bound builds on 7 GB macOS runners outlast the default 10-minute deadlock watchdog. // Unsafe access is fixed at build time; otherwise every run warns about Scala's `LazyVals`. + // So is the JDK's list of auth schemes refused for HTTPS proxy tunnels, which makes the runtime + // opt-in in `JvmProperties` a no-op here; Basic must be allowed at build time for a proxy asking + // for credentials. Without a proxy user nothing answers the challenge either way. def nativeImageOptions = Seq( "--no-fallback", "--enable-url-protocols=http,https", @@ -212,7 +215,8 @@ object cli extends CellarPublishModule with NativeImageModule { "-H:+UnlockExperimentalVMOptions", "-H:DeadlockWatchdogInterval=30", "-H:-UnlockExperimentalVMOptions", - "-J--sun-misc-unsafe-memory-access=allow" + "-J--sun-misc-unsafe-memory-access=allow", + "-Djdk.http.auth.tunneling.disabledSchemes=" ) def mvnDeps = Seq( diff --git a/cli/src/cellar/cli/CellarApp.scala b/cli/src/cellar/cli/CellarApp.scala index 06f3470..717d301 100644 --- a/cli/src/cellar/cli/CellarApp.scala +++ b/cli/src/cellar/cli/CellarApp.scala @@ -129,7 +129,7 @@ object CellarApp extends ProfilingIOApp: case Left(error) => IO.blocking(System.err.println(error)).as(ExitCode.Error) case Right(properties) => - IO(properties.foreach(sys.props.getOrElseUpdate(_, _))) *> runCommand(args) + JvmProperties.install(properties) *> runCommand(args) private def runCommand(args: List[String]): IO[ExitCode] = val versionOpt = Opts diff --git a/cli/src/cellar/cli/JvmProperties.scala b/cli/src/cellar/cli/JvmProperties.scala new file mode 100644 index 0000000..387d9d2 --- /dev/null +++ b/cli/src/cellar/cli/JvmProperties.scala @@ -0,0 +1,23 @@ +package cellar.cli + +import cats.effect.IO +import coursier.proxy.SetupProxy + +object JvmProperties: + + /** Sets each `java.properties` entry not already set (e.g. by `-D`) and enables proxy + * authentication from `http(s).proxyUser` / `http(s).proxyPassword`. + * + * Must run before the first HTTP connection: the JDK reads its tunneling settings once. + */ + def install(properties: List[(String, String)]): IO[Unit] = + IO { + properties.foreach(sys.props.getOrElseUpdate(_, _)) + // coursier answers a proxy's challenge only when the protocol the JDK reports matches these, + // and it defaults the https one to "https". The JDK only ever speaks plain HTTP to a proxy, + // so a setup with just `https.proxy*` would never be answered. + List("http.proxyProtocol", "https.proxyProtocol").foreach(sys.props.getOrElseUpdate(_, "http")) + // The JDK ignores the proxy user and password on its own and refuses Basic auth for HTTPS + // tunnels; this installs what the coursier launcher does, and only when a proxy user is set. + SetupProxy.setupAuthenticator(): Unit + } diff --git a/cli/src/cellar/cli/RepositoryUrl.scala b/cli/src/cellar/cli/RepositoryUrl.scala index 040e622..197a480 100644 --- a/cli/src/cellar/cli/RepositoryUrl.scala +++ b/cli/src/cellar/cli/RepositoryUrl.scala @@ -11,7 +11,7 @@ import scala.util.Try * rather than at the repository. Rejecting the URL up front keeps the blame where it belongs. * * Credentials in the URL (`https://user:token@host/...`) become the repository's own and are sent - * with the first request; error messages mask the password. + * with the first request; error messages mask them. */ object RepositoryUrl: @@ -23,10 +23,14 @@ object RepositoryUrl: def parse(raw: String): Either[String, MavenRepository] = val trimmed = raw.trim val shown = masked(trimmed) + // coursier does not URL-decode credentials, so these characters cannot be written in any form. + val credentialsHint = + if shown == trimmed then "" + else "; credentials in a URL cannot contain '/', '?', '#' or '@', use COURSIER_CREDENTIALS instead" if trimmed.isEmpty then Left("the URL is empty") else Try(URI(trimmed)).toEither.left - .map(_ => s"'$shown' is not a valid URL") + .map(_ => s"'$shown' is not a valid URL$credentialsHint") .flatMap { uri => Option(uri.getScheme).map(_.toLowerCase) match case None => @@ -34,13 +38,19 @@ object RepositoryUrl: case Some(scheme) if !supportedSchemes.contains(scheme) => Left(s"'$shown' has unsupported scheme '$scheme' (expected one of $expected)") case Some(scheme) if scheme != "file" && Option(uri.getHost).forall(_.isEmpty) => - Left(s"'$shown' has no host") + Left(s"'$shown' has no host$credentialsHint") case Some(_) => - RepositoryParser.repository(trimmed).left.map(_ => s"'$shown' is not a valid URL").flatMap { - case maven: MavenRepository => Right(maven) - case _ => Left(s"'$shown' is not a Maven repository URL") - } + // coursier's own message would echo the password; a scheme checked above always + // parses to a Maven repository. + RepositoryParser.repository(trimmed).left + .map(_ => s"'$shown' is not a valid URL$credentialsHint") + .map { case maven: MavenRepository => maven } } + /** Hides everything between the scheme and the last `@`. A token may itself contain `/`, `@` or + * `#`, so the usual delimiters cannot tell where the credentials end. + */ private def masked(url: String): String = - url.replaceFirst("^([a-zA-Z][a-zA-Z0-9+.-]*://[^:/@]*):[^@/]*@", "$1:****@") + val schemeEnd = url.indexOf("://") + val at = url.lastIndexOf('@') + if schemeEnd >= 0 && at > schemeEnd then url.take(schemeEnd + 3) + "****" + url.drop(at) else url diff --git a/cli/test/src/cellar/cli/PrivateRepositoryTest.scala b/cli/test/src/cellar/cli/PrivateRepositoryTest.scala new file mode 100644 index 0000000..224f8a0 --- /dev/null +++ b/cli/test/src/cellar/cli/PrivateRepositoryTest.scala @@ -0,0 +1,110 @@ +package cellar.cli + +import cats.effect.{IO, Resource} +import cellar.{CoursierFetchClient, MavenCoordinate} +import com.sun.net.httpserver.{HttpExchange, HttpServer} +import weaver.SimpleIOSuite + +import java.net.{Authenticator, InetAddress, InetSocketAddress} +import java.nio.charset.StandardCharsets.UTF_8 +import java.util.Base64 +import java.util.concurrent.ConcurrentLinkedQueue +import scala.jdk.CollectionConverters.* + +/** Fetches through real HTTP servers standing in for the private-repository setups the README documents. */ +object PrivateRepositoryTest extends SimpleIOSuite: + + // The proxy tests set JVM-wide system properties and the default Authenticator. + override def maxParallelism = 1 + + private val coord = MavenCoordinate("org.example.private", "lib", "1.0.0") + private val dir = "/org/example/private/lib/1.0.0/" + private val content = Map( + s"${dir}lib-1.0.0.pom" -> + "4.0.0org.example.privatelib1.0.0", + s"${dir}lib-1.0.0.jar" -> "not inspected" + ) + + // coursier caches by URL, credentials' user included; a run reusing an earlier URL would be + // answered from the cache without reaching the servers under test. + private val run = java.util.UUID.randomUUID().toString.take(8) + + private def basic(user: String, password: String) = + "Basic " + Base64.getEncoder.encodeToString(s"$user:$password".getBytes(UTF_8)) + + private def respond(exchange: HttpExchange, code: Int, body: String = ""): Unit = + val bytes = body.getBytes(UTF_8) + exchange.sendResponseHeaders(code, if bytes.isEmpty then -1 else bytes.length.toLong) + if bytes.nonEmpty then exchange.getResponseBody.write(bytes) + exchange.close() + + private def serve(exchange: HttpExchange): Unit = + content.get(exchange.getRequestURI.getPath).fold(respond(exchange, 404))(respond(exchange, 200, _)) + + /** Starts `handle` on a free port; yields the port. */ + private def server(handle: HttpExchange => Unit): Resource[IO, Int] = + Resource + .make(IO.blocking { + val server = HttpServer.create(InetSocketAddress("127.0.0.1", 0), 0) + server.createContext("/", exchange => handle(exchange)) + server.start() + server + })(server => IO.blocking(server.stop(0))) + .map(_.getAddress.getPort) + + test("URL credentials reach a feed that answers anonymous requests with a 203 sign-in page"): + val anonymous = ConcurrentLinkedQueue[String]() + val feed = server { exchange => + if exchange.getRequestHeaders.getFirst("Authorization") == basic(s"alice-$run", "s3cret") then serve(exchange) + else + anonymous.add(exchange.getRequestURI.getPath): Unit + respond(exchange, 203, "Sign in") + } + feed.use { port => + RepositoryUrl.parse(s"http://alice-$run:s3cret@127.0.0.1:$port/").fold( + err => IO.pure(failure(err)), + repo => + CoursierFetchClient.fetchPom(coord, Seq(repo)).map { pom => + expect(pom.isDefined) and expect.eql(Nil, anonymous.asScala.toList) + } + ) + } + + test("an authenticating proxy is passed the proxy user and password"): + val proxied = ConcurrentLinkedQueue[String]() + // A forward proxy is sent the full target URL; this one serves the repository itself. + val proxy = server { exchange => + if exchange.getRequestHeaders.getFirst("Proxy-Authorization") == basic("proxyuser", "proxypw") then + proxied.add(exchange.getRequestURI.toString): Unit + serve(exchange) + else + exchange.getResponseHeaders.add("Proxy-Authenticate", """Basic realm="corp-proxy"""") + respond(exchange, 407) + } + val properties = List("http.proxyHost", "http.proxyPort", "http.proxyUser", "http.proxyPassword") + proxy.use { port => + val install = JvmProperties.install( + properties.zip(List("127.0.0.1", port.toString, "proxyuser", "proxypw")) + ) + val repo = RepositoryUrl.parse(s"http://repo-$run.example.invalid/").fold(err => throw AssertionError(err), identity) + (install *> CoursierFetchClient.fetchPom(coord, Seq(repo))) + .guarantee(IO(properties.foreach(sys.props.remove))) + .map { pom => + expect(pom.isDefined) and + expect(proxied.asScala.exists(_.startsWith(s"http://repo-$run.example.invalid/org/example/private/lib/"))) + } + } + + test("a proxy configured only through https.proxy* properties is answered too"): + val properties = List("https.proxyHost", "https.proxyPort", "https.proxyUser", "https.proxyPassword") + val install = JvmProperties.install(properties.zip(List("127.0.0.1", "3128", "proxyuser", "proxypw"))) + // The JDK reports "http" for any proxy challenge, an HTTPS tunnel's included. + val challenge = IO( + Authenticator.requestPasswordAuthentication( + "127.0.0.1", InetAddress.getLoopbackAddress, 3128, "http", "corp-proxy", "basic", null, + Authenticator.RequestorType.PROXY + ) + ) + (install *> challenge) + .guarantee(IO((properties ++ List("http.proxyProtocol", "https.proxyProtocol")).foreach(sys.props.remove))) + .map(auth => expect(auth != null && auth.getUserName == "proxyuser", String.valueOf(auth))) diff --git a/cli/test/src/cellar/cli/RepositoryUrlTest.scala b/cli/test/src/cellar/cli/RepositoryUrlTest.scala index 232fef1..6bb56d8 100644 --- a/cli/test/src/cellar/cli/RepositoryUrlTest.scala +++ b/cli/test/src/cellar/cli/RepositoryUrlTest.scala @@ -50,8 +50,13 @@ object RepositoryUrlTest extends FunSuite: } ) - test("never echoes a URL password in an error"): - reason("ftp://alice:s3cret@pkgs.example.com/maven") { message => - expect(message.contains("ftp://alice:****@pkgs.example.com/maven"), message) and - expect(!message.contains("s3cret"), message) - } + test("never echoes URL credentials in an error, whatever characters they contain"): + def masked(raw: String, secret: String) = + reason(raw)(message => expect(message.contains("****@") && !message.contains(secret), message)) + masked("ftp://alice:s3cret@pkgs.example.com/maven", "s3cret") and + masked("https://alice:ab/cd@pkgs.example.com/maven", "ab/cd") and + masked("https://alice:p@ss@pkgs.example.com/maven", "p@ss") and + masked("ftp://ghp_token@pkgs.example.com/maven", "ghp_token") + + test("explains that credentials with URL delimiters cannot go in a URL"): + reason("https://alice:ab/cd@pkgs.example.com/maven")(message => expect(message.contains("COURSIER_CREDENTIALS"), message)) diff --git a/lib/src/cellar/CoordinateCompleter.scala b/lib/src/cellar/CoordinateCompleter.scala index 237b918..7063df1 100644 --- a/lib/src/cellar/CoordinateCompleter.scala +++ b/lib/src/cellar/CoordinateCompleter.scala @@ -2,17 +2,17 @@ package cellar import cats.effect.IO import coursier.complete.Complete -import coursier.{Repository, Versions} +import coursier.{Repository, Resolve} object CoordinateCompleter: def suggest(coord: MavenCoordinate, extraRepos: Seq[Repository]): IO[List[String]] = IO.blocking { - def release(artifact: String): Option[String] = - Some(Versions(CoursierCache.instance).withModule(coord.copy(artifact = artifact).toCoursierModule).addRepositories(extraRepos*).run()) - .map(_.release0.asString) - .filter(_.nonEmpty) + def release(artifact: String): Option[String] = coord.copy(artifact = artifact).latestRelease(extraRepos) - val complete = Complete(CoursierCache.instance).addRepositories(extraRepos*).copy(input = s"${coord.group}:${coord.artifact}") + // Completion knows nothing of mirrors; hand it the repositories a fetch actually goes to. + val resolve = Resolve().addRepositories(extraRepos*) + val mirrored = resolve.finalRepositories.unsafeRun()(using resolve.cache.ec) + val complete = Complete().copy(repositories = mirrored, input = s"${coord.group}:${coord.artifact}") val completions = complete.complete().unsafeRun()(using complete.cache.ec)._2.toList val exactMatch = completions.contains(coord.artifact) diff --git a/lib/src/cellar/CoursierCache.scala b/lib/src/cellar/CoursierCache.scala deleted file mode 100644 index 65c117f..0000000 --- a/lib/src/cellar/CoursierCache.scala +++ /dev/null @@ -1,14 +0,0 @@ -package cellar - -import coursier.cache.FileCache -import coursier.util.Task - -private[cellar] object CoursierCache: - - /** The cache every coursier call shares. - * - * Some feeds (Azure Artifacts) answer an anonymous request with `203` and a sign-in page instead - * of challenging it. Accepted, that page would be cached as the artifact and break every later - * lookup, so it fails the download instead. - */ - lazy val instance: FileCache[Task] = FileCache().copy(rejectNonAuthoritativeResponses = true) diff --git a/lib/src/cellar/CoursierFetchClient.scala b/lib/src/cellar/CoursierFetchClient.scala index eb5d58d..f24bff9 100644 --- a/lib/src/cellar/CoursierFetchClient.scala +++ b/lib/src/cellar/CoursierFetchClient.scala @@ -19,7 +19,7 @@ object CoursierFetchClient: )(using logger: Logger[IO] = StderrLogger.off): IO[Option[Path]] = logAttempt(coord, "fetching POM for", extraRepositories) *> IO.blocking { - Fetch(CoursierCache.instance) + Fetch() .addDependencies(coord.toCoursierDependency.copy(transitive = false)) .addRepositories(extraRepositories*) .run() @@ -58,7 +58,7 @@ object CoursierFetchClient: tracer.span("coursier.fetch").surround { logAttempt(coord, "resolving", extraRepositories) *> IO.blocking { - val files = Fetch(CoursierCache.instance) + val files = Fetch() .addDependencies(coord.toCoursierDependency) .addRepositories(extraRepositories*) .addClassifiers(Classifier.sources) diff --git a/lib/src/cellar/CoursierResolveClient.scala b/lib/src/cellar/CoursierResolveClient.scala index 9f5fc20..795021b 100644 --- a/lib/src/cellar/CoursierResolveClient.scala +++ b/lib/src/cellar/CoursierResolveClient.scala @@ -14,7 +14,7 @@ object CoursierResolveClient: )(using logger: Logger[IO] = StderrLogger.off): IO[ResolvedDeps] = logger.debug(s"resolving dependency tree for ${coord.render}") *> IO.blocking { - val resolution = Resolve(CoursierCache.instance) + val resolution = Resolve() .addDependencies(coord.toCoursierDependency) .addRepositories(extraRepositories*) .run() diff --git a/lib/src/cellar/DocstringExtractor.scala b/lib/src/cellar/DocstringExtractor.scala index 682d1bc..4ac0498 100644 --- a/lib/src/cellar/DocstringExtractor.scala +++ b/lib/src/cellar/DocstringExtractor.scala @@ -20,7 +20,7 @@ object DocstringExtractor: // yields "" and the fetch below asks for `scala3-library_3:` with no version at all. // Requires the Scala 3 stdlib (3.8+), where `library.properties` carries a 3.x version. val scalaVersion = scala.util.Properties.versionNumberString - coursier.Fetch(CoursierCache.instance) + coursier.Fetch() .addDependencies(MavenCoordinate("org.scala-lang", "scala3-library_3", scalaVersion).toCoursierDependency) .run() .map(file => Path.fromNioPath(file.toPath)) diff --git a/lib/src/cellar/ExtraRepositories.scala b/lib/src/cellar/ExtraRepositories.scala index 2bc6ece..b24c720 100644 --- a/lib/src/cellar/ExtraRepositories.scala +++ b/lib/src/cellar/ExtraRepositories.scala @@ -6,7 +6,13 @@ private[cellar] object ExtraRepositories: /** Configured repositories first, command-line `-r` values appended. * - * Duplicates collapse to their first occurrence, ignoring a trailing slash. + * Duplicates collapse to the position of their first occurrence, ignoring a trailing slash. An + * occurrence carrying credentials wins over one without, so `-r https://user:token@host/...` is + * not lost to the same URL already configured without them. */ def effective(configured: List[MavenRepository], commandLine: List[MavenRepository]): List[Repository] = - (configured ++ commandLine).distinctBy(_.root.stripSuffix("/")) + val all = configured ++ commandLine + def key(repo: MavenRepository) = repo.root.stripSuffix("/") + all.distinctBy(key).map { first => + all.find(repo => key(repo) == key(first) && repo.authentication.nonEmpty).getOrElse(first) + } diff --git a/lib/src/cellar/MavenCoordinate.scala b/lib/src/cellar/MavenCoordinate.scala index d89db77..13b8483 100644 --- a/lib/src/cellar/MavenCoordinate.scala +++ b/lib/src/cellar/MavenCoordinate.scala @@ -1,7 +1,7 @@ package cellar import cats.effect.IO -import coursier.{Dependency, Module, ModuleName, Organization, Repository, VersionConstraint, Versions} +import coursier.{Dependency, Module, ModuleName, Organization, Repository, Resolve, VersionConstraint, Versions} final case class MavenCoordinate(group: String, artifact: String, version: String): def render: String = s"$group:$artifact:$version" @@ -13,11 +13,29 @@ final case class MavenCoordinate(group: String, artifact: String, version: Strin def resolveLatest(extraRepos: Seq[Repository] = Seq.empty): IO[MavenCoordinate] = if version != "latest" then IO.pure(this) else IO.blocking { - val release = Versions(CoursierCache.instance).withModule(toCoursierModule).addRepositories(extraRepos*).run().release0.asString - if release.nonEmpty then copy(version = release) - else throw CellarError.CoordinateNotFound(this, new RuntimeException(s"No versions found for '$group:$artifact'")) + latestRelease(extraRepos).fold( + throw CellarError.CoordinateNotFound(this, new RuntimeException(s"No versions found for '$group:$artifact'")) + )(release => copy(version = release)) } + /** The newest release of `group:artifact`, whatever `version` says; `None` when none is listed. + * Blocks while it queries the repositories. + */ + def latestRelease(extraRepos: Seq[Repository]): Option[String] = + // `Versions` applies only coursier's own mirror files by default, `Resolve` also the scala-cli + // and Maven settings mirrors; a lookup that skips a mirror the fetch goes through fails behind + // a firewall. + Some( + Versions() + .noMirrors + .addMirrors(Resolve.defaultMirrors*) + .withModule(toCoursierModule) + .addRepositories(extraRepos*) + .run() + .release0 + .asString + ).filter(_.nonEmpty) + object MavenCoordinate: def parse(raw: String): Either[String, MavenCoordinate] = raw.split(":", -1).toList match diff --git a/lib/src/cellar/SourcesResolver.scala b/lib/src/cellar/SourcesResolver.scala index ec45137..c8477d4 100644 --- a/lib/src/cellar/SourcesResolver.scala +++ b/lib/src/cellar/SourcesResolver.scala @@ -1,5 +1,6 @@ package cellar +import coursier.cache.FileCache import coursier.util.Artifact import coursier.{Classifier, Fetch, Repository} import fs2.io.file.Path @@ -31,7 +32,7 @@ object SourcesResolver: * jar is the same URL with the `-sources.jar` suffix, whichever repository served it. */ private def fromCoursierCache(jar: Path): Option[Path] = - val cache = CoursierCache.instance + val cache = FileCache() val root = Path.fromNioPath(cache.location.toPath) val relative = root.toNioPath.relativize(jar.toNioPath) Option.when(jar.toNioPath.startsWith(root.toNioPath) && relative.getNameCount > 2) { @@ -47,7 +48,7 @@ object SourcesResolver: private def fromPomProperties(jar: Path, repositories: Seq[Repository]): Option[Path] = coordinate(jar).flatMap { (group, artifact, version) => val dependency = MavenCoordinate(group, artifact, version).toCoursierDependency.copy(transitive = false) - Fetch(CoursierCache.instance) + Fetch() .addDependencies(dependency) .addRepositories(repositories*) .addClassifiers(Classifier.sources) diff --git a/lib/test/src/cellar/ExtraRepositoriesTest.scala b/lib/test/src/cellar/ExtraRepositoriesTest.scala index 4302436..3d9cef0 100644 --- a/lib/test/src/cellar/ExtraRepositoriesTest.scala +++ b/lib/test/src/cellar/ExtraRepositoriesTest.scala @@ -42,6 +42,16 @@ object ExtraRepositoriesTest extends SimpleIOSuite: bases(List("https://repo.example/maven"), List("https://repo.example/maven/", "https://other.example/maven")) ) + pureTest("a duplicate carrying credentials wins over the configured one without"): + val withCredentials = MavenRepository("https://repo.example/maven", Some(coursier.core.Authentication("u", "p"))) + expect.eql( + List(Some("u")), + ExtraRepositories.effective(List(MavenRepository("https://repo.example/maven")), List(withCredentials)).map { + case maven: MavenRepository => maven.authentication.flatMap(_.userOpt) + case _ => None + } + ) + pureTest("no configured repositories leaves command-line values untouched"): expect.eql(List("https://cli.example/maven"), bases(Nil, List("https://cli.example/maven")))