diff --git a/CHANGELOG.md b/CHANGELOG.md index e0402f04d..2aea6bbad 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -159,6 +159,18 @@ Version 5.x is JDK17 LTS bytecode compatible, with Docker and JUnit / direct Jav * Version updates (build dependencies) * TBD +## 5.2.1 + +* Features and fixes + * fix: Restore the ability to persist data to a mounted Docker volume. Since the migration to the Cloud Native Buildpacks OCI image (which runs as the non-root `cnb` user), a mounted named volume was no longer writable. The image now pre-creates a `cnb`-owned `/s3mockroot` directory, so a Docker named volume mounted at `/s3mockroot` (with `COM_ADOBE_TESTING_S3MOCK_STORE_ROOT=/s3mockroot`) is writable without running the container as root. S3Mock now also fails fast with a clear message when the configured store root is not writable, creates nested store-root paths (not just a single missing directory), and `S3MockContainer.withNamedVolume(...)` plus a container restart persistence test were added. The image also restores `/` (rather than the Buildpacks default `/workspace`) as its working directory, so a *relative* `COM_ADOBE_TESTING_S3MOCK_STORE_ROOT` resolves exactly as it did before 5.2 instead of silently resolving under `/workspace` and missing the mounted volume; the resolved absolute root folder is now always logged at startup alongside the configured value. ([#3139](https://github.com/adobe/S3Mock/issues/3139)) +* Version updates (deliverable dependencies) + * N/A +* Version updates (build dependencies) + * Bump actions/setup-java from 5.7.0 to 6.0.0 + * Bump github/codeql-action from 4.37.7 to 4.37.9 + * Bump docker/setup-qemu-action from 4.2.0 to 4.3.0 + * Bump step-security/harden-runner from 2.21.0 to 2.21.1 + ## 5.2.0 * Features and fixes @@ -216,7 +228,7 @@ Version 5.x is JDK17 LTS bytecode compatible, with Docker and JUnit / direct Jav * Bump docker/setup-qemu-action from 4.1.0 to 4.2.0 * Bump github/codeql-action from 4.36.2 to 4.37.7 * Bump ossf/scorecard-action from 2.4.3 to 2.4.4 - * Bump step-security/harden-runner from 2.19.4 to 2.21.1 + * Bump step-security/harden-runner from 2.19.4 to 2.21.0 ## 5.1.0 diff --git a/README.md b/README.md index 1791d4b79..4ebbf7096 100755 --- a/README.md +++ b/README.md @@ -302,20 +302,46 @@ services: - 9191:9191 ``` -**With persistent storage:** +**With persistent storage (named volume):** ```yaml services: s3mock: image: adobe/s3mock:latest environment: - - COM_ADOBE_TESTING_S3MOCK_STORE_ROOT=containers3root + # /s3mockroot is pre-created in the image, owned by the non-root user the container runs as, + # so a mounted named volume is writable without running the container as root. + - COM_ADOBE_TESTING_S3MOCK_STORE_ROOT=/s3mockroot - COM_ADOBE_TESTING_S3MOCK_STORE_RETAIN_FILES_ON_EXIT=true ports: - 9090:9090 volumes: - - ./locals3root:/containers3root + - s3mockdata:/s3mockroot + +volumes: + s3mockdata: ``` +> The store root **must** be `/s3mockroot` for a Docker named volume: the image runs as the +> non-root `cnb` user, and Docker only makes a named volume writable by that user when it is +> mounted onto a directory that already exists in the image with that ownership. A bind mount +> (`./host/path:/s3mockroot`) keeps its host ownership instead, so it must be writable by that +> user (uid 1000). +> +> A Docker named volume (`s3mockdata:/s3mockroot` above) is **not** a host directory you can +> browse directly — Docker manages its storage location internally. Use +> `docker volume inspect _s3mockdata` to find its `Mountpoint`, or +> `docker run --rm -v _s3mockdata:/v alpine ls /v` to list its contents. If you need a +> predictable, browsable host path instead, use a bind mount +> (`./locals3root:/s3mockroot`), keeping in mind the ownership requirement above. +> +> A **relative** `COM_ADOBE_TESTING_S3MOCK_STORE_ROOT` (e.g. `data` instead of `/data`) resolves +> against the container's working directory, which the image sets to `/` — so `data` resolves +> to `/data`, matching a volume mounted at `/data`. This is unrelated to the `/s3mockroot` +> named-volume requirement above: prefer an absolute path in general so the resolved location +> doesn't depend on the image's working directory. The resolved absolute root folder is always +> logged at startup, alongside the configured value, to make a mismatch between the two easy to +> spot. + ### Testcontainers The [`S3MockContainer`](testsupport/testcontainers/src/main/kotlin/com/adobe/testing/s3mock/testcontainers/S3MockContainer.kt) provides a ready-to-use Testcontainers implementation. @@ -443,7 +469,7 @@ Configure S3Mock using environment variables: | Variable | Default | Description | |-------------------------------------------------------|---------------------|---------------------------------------------------------------| -| `COM_ADOBE_TESTING_S3MOCK_STORE_ROOT` | Java temp directory | Base directory for file storage | +| `COM_ADOBE_TESTING_S3MOCK_STORE_ROOT` | Java temp directory | Base directory for file storage (use `/s3mockroot` to persist to a mounted named volume) | | `COM_ADOBE_TESTING_S3MOCK_STORE_REGION` | `us-east-1` | AWS region to mock | | `COM_ADOBE_TESTING_S3MOCK_STORE_INITIAL_BUCKETS` | none | Comma-separated list of buckets to create on startup | | `COM_ADOBE_TESTING_S3MOCK_STORE_RETAIN_FILES_ON_EXIT` | `false` | Keep files after shutdown | diff --git a/docs/SETUP.md b/docs/SETUP.md index 82d42fe1b..f845cde3e 100644 --- a/docs/SETUP.md +++ b/docs/SETUP.md @@ -36,7 +36,7 @@ S3Mock is configured via environment variables (Docker / Testcontainers) or Spri | Environment variable | Spring property | Default | Description | |---|---|---|---| -| `COM_ADOBE_TESTING_S3MOCK_STORE_ROOT` | `com.adobe.testing.s3mock.store.root` | temp dir | Storage root directory | +| `COM_ADOBE_TESTING_S3MOCK_STORE_ROOT` | `com.adobe.testing.s3mock.store.root` | temp dir | Storage root directory (use `/s3mockroot` to persist to a mounted Docker named volume) | | `COM_ADOBE_TESTING_S3MOCK_STORE_RETAIN_FILES_ON_EXIT` | `com.adobe.testing.s3mock.store.retainFilesOnExit` | `false` | Keep files on shutdown | | `COM_ADOBE_TESTING_S3MOCK_STORE_REGION` | `com.adobe.testing.s3mock.store.region` | `us-east-1` | AWS region | | `COM_ADOBE_TESTING_S3MOCK_STORE_INITIAL_BUCKETS` | `com.adobe.testing.s3mock.store.initialBuckets` | _(none)_ | Comma-separated bucket names to create on startup | @@ -56,16 +56,20 @@ Or with Docker directly: docker run -p 9090:9090 -p 9191:9191 adobe/s3mock ``` -With persistent storage: +With persistent storage (named volume): ```bash docker run \ -p 9090:9090 -p 9191:9191 \ - -e COM_ADOBE_TESTING_S3MOCK_STORE_ROOT=/data \ + -e COM_ADOBE_TESTING_S3MOCK_STORE_ROOT=/s3mockroot \ -e COM_ADOBE_TESTING_S3MOCK_STORE_RETAIN_FILES_ON_EXIT=true \ - -v /local/path:/data \ + -v s3mockdata:/s3mockroot \ adobe/s3mock ``` +`/s3mockroot` is pre-created in the image owned by the non-root `cnb` user the container runs as, +so a Docker named volume mounted there is writable without running as root. A bind mount +(`-v /local/path:/s3mockroot`) keeps its host ownership and must be writable by uid 1000 instead. + ## Running Tests ```bash @@ -119,3 +123,5 @@ MANAGEMENT_ENDPOINTS_ACCESS_DEFAULT=unrestricted docker run -p 9090:9090 -p 9191 | `NoClassDefFoundError` in in-process test | Spring Boot version mismatch | Ensure your project is Spring Boot 4.x compatible | | HTTPS connection refused | Client not trusting self-signed cert | Configure trust-all-certs on your AWS SDK client | | Empty bucket after restart | `retainFilesOnExit` defaults to false | Set `COM_ADOBE_TESTING_S3MOCK_STORE_RETAIN_FILES_ON_EXIT=true` and use a fixed `STORE_ROOT` | +| HTTP 500 / `not writable` at startup when mounting a volume | Docker named volume or bind mount not writable by the non-root `cnb` user | Mount a **named volume** at `/s3mockroot` and set `COM_ADOBE_TESTING_S3MOCK_STORE_ROOT=/s3mockroot`; for a bind mount, make the host directory writable by uid 1000 | +| Container starts fine, `2xx` responses, but a mounted volume stays empty | A **relative** `STORE_ROOT` (e.g. `s3root`) resolved against the wrong working directory, missing the mounted volume | Use an **absolute** `STORE_ROOT` (e.g. `/s3root`); check the startup log line showing the resolved absolute root folder against your mount path | diff --git a/pom.xml b/pom.xml index 4acfb931e..c1c719e96 100644 --- a/pom.xml +++ b/pom.xml @@ -76,6 +76,12 @@ 14.0.0 adobe/s3mock + + adobe/s3mock-run 2.2.0 3.6.3 @@ -231,7 +237,7 @@ year bumps scoped to files actually touched (see INVARIANTS.md). --> origin/main - + ${maven.multiModuleProjectDirectory}/etc/license-header-spotless.txt @@ -401,8 +407,8 @@ ${java.version} - - + + @@ -470,15 +476,24 @@ true deploy -DskipTests -Prelease -P!build-docker-image -Ppush-docker-image - -Dspotless.check.skip=true - clean install + spotless:apply clean install + spotless:apply @{project.version} diff --git a/server/pom.xml b/server/pom.xml index 8ea39e80f..d64caadeb 100644 --- a/server/pom.xml +++ b/server/pom.xml @@ -189,6 +189,15 @@ CVEs reported by users' image scanners. Multi-arch (linux/amd64 + linux/arm64). --> bellsoft/buildpacks.builder:musl + + IF_NOT_PRESENT ${java.version} @@ -272,6 +281,33 @@ + + org.codehaus.mojo + exec-maven-plugin + + + + build-run-image + + exec + + prepare-package + + docker + + build + --tag + ${docker.run.image.name}:musl + ${project.basedir}/src/main/docker/run-image + + + + + org.springframework.boot spring-boot-maven-plugin @@ -285,6 +321,7 @@ ${docker.image.name}:${project.version} + ${docker.run.image.name}:musl ${docker.image.name}:${parsedVersion.majorVersion} @@ -303,6 +340,12 @@ Release-only profile: builds both linux/amd64 and linux/arm64 images (arm64 via QEMU emulation on an amd64 runner), publishes each to a temporary per-architecture tag on Docker Hub, then merges them into a single multi-architecture manifest for the final tags. + + Duplicates the build-helper-maven-plugin/exec-maven-plugin declarations from + build-docker-image above rather than sharing them: the two profiles are mutually + exclusive at release time (the root pom's release goals pass + "-P!build-docker-image -Ppush-docker-image"), and this profile's executions build/push + both architectures instead of just the host one. --> push-docker-image @@ -328,6 +371,54 @@ org.codehaus.mojo exec-maven-plugin + + + build-run-image-amd64 + + exec + + prepare-package + + docker + + buildx + build + --platform + linux/amd64 + --load + --tag + ${docker.run.image.name}:musl-amd64 + ${project.basedir}/src/main/docker/run-image + + + + + build-run-image-arm64 + + exec + + prepare-package + + docker + + buildx + build + --platform + linux/arm64 + --load + --tag + ${docker.run.image.name}:musl-arm64 + ${project.basedir}/src/main/docker/run-image + + +