Skip to content

test(storage): run the file store on a real S3 store in CI - #303

Merged
SirLouen merged 11 commits into
mainfrom
fix/302
Oct 26, 2026
Merged

SirLouen merged 11 commits into
mainfrom
fix/302

Conversation

@SirLouen

@SirLouen SirLouen commented Oct 26, 2026 •

Copy link
Copy Markdown
Member

Closes #302.

What

  • A new CI job, s3, starts a pinned Garage store, makes a bucket and a key, and runs the storage tests built with the new s3 tag against it.
  • Those tests cover a save, open and delete, a missing file, another workspace, a file over the cap, a failed upload and a file sent in parts, under both checksum settings. They check through the bucket and the S3 client that nothing is left behind.
  • Fix: an upload sent in parts that failed or ran over the cap left its parts in the bucket, because the abort went out on the request it was cancelling. The abort now goes on its own, bounded by ALPHONE_STORAGE_CLEANUP_TIMEOUT.
  • make test-s3 runs the same tests on a local Garage from a new s3 profile in compose.yaml.
  • The install guide says Garage before v2.5.0 needs ALPHONE_STORAGE_S3_CHECKSUM=when_required.

Why

Nothing tested the S3 driver against a real store, and the first run found the leftover parts.

Testing Instructions

Run the testing toolchain.

Summary by CodeRabbit

  • New Features
    • Added support for cleaning up incomplete S3 uploads, even when the original request is canceled.
    • Added local S3-compatible storage setup and integration tests for file uploads, downloads, and deletion.
  • Documentation
    • Expanded guidance for local S3 testing, cleanup timeouts, and checksum settings when using older Garage releases.

@coderabbitai

coderabbitai Bot commented Oct 26, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

No actionable comments were generated in the recent review.

Recent review info
Run configuration
Commits

Reviewing files that changed from the base of the PR and between c692fee and 0d840e7.

Files selected for processing: 17
  • .env.example
  • .github/workflows/ci.yml
  • .golangci.yml
  • Makefile
  • cmd/alphone/settings_test.go
  • compose.yaml
  • docs/src/content/docs/self-hosting/configuration.md
  • docs/src/content/docs/self-hosting/install.md
  • docs/src/content/docs/start/local-development.md
  • go.mod
  • internal/storage/s3_test.go
  • internal/storage/storage.go
  • internal/storage/storage_internal_test.go
  • internal/testdb/testdb.go
  • internal/testdb/testdb_test.go
  • test/s3/garage.sh
  • test/s3/garage.toml

Included review availability: This review used your included allowance. Your plan provides up to 2 included reviews per hour; 1 remain after this review.

Walkthrough

The pull request adds Garage-backed S3 storage tests and CI coverage. It also applies a configured cleanup timeout to S3 multipart-upload abort requests and adds local test configuration and documentation.

Changes: S3 storage tests and cleanup

Layer / File(s) Summary
Test settings and Garage provisioning
.env.example, internal/testdb/..., cmd/alphone/settings_test.go, test/s3/*, compose.yaml
Adds required test-setting lookup, example S3 settings, and Garage configuration and provisioning for the test bucket and key.
Bounded S3 abort cleanup
internal/storage/storage.go, internal/storage/storage_internal_test.go, go.mod, .env.example, docs/src/content/docs/self-hosting/configuration.md
The S3 client now uses the configured cleanup timeout for multipart-upload abort requests. Tests cover aborts after cancellation and when the store responds after the timeout. Cleanup descriptions now include S3 parts as well as PostgreSQL rows.
S3 storage behavior coverage
internal/storage/s3_test.go
Adds tests for file operations, workspace isolation, oversized and failed uploads, multipart readback, and cleanup under both checksum settings.
Local and CI test runs
Makefile, .github/workflows/ci.yml, .golangci.yml, docs/src/content/docs/start/local-development.md, docs/src/content/docs/self-hosting/install.md
Adds local and CI commands for S3-tagged tests, configures lint to use the s3 build tag, and documents test setup and the Garage checksum caveat.

Priority: Normal

Estimated code review effort: 3 (Moderate) | ~30 minutes

Change: Other

Possibly related PRs

  • gopherium/AlphOne#301: Adds the S3 storage driver that this pull request tests against Garage.
  • gopherium/AlphOne#282: Adds validation for settings in .env.example; this pull request adds the S3 test settings to that validation allowlist.

Suggested reviewers: codescene-access

Merge Risk: Minimal · up to 0d840

No actionable issue remains from the reviewed S3 test and cleanup changes. Merge after normal checks.

Pre-merge checks: Passed 4 · Failed 1

Failed Failed checks: 1 warning

Check name Status Explanation Resolution
Docstring Coverage Warning Docstring coverage is 57.89% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 38 functions across 7 files. (10 skipped:… Write docstrings for the functions missing them to satisfy the coverage threshold.
Passed Passed checks: 4 passed
Check name Status Explanation
Description Check Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check Passed The title clearly and concisely describes the main change: running storage tests against a real S3 store in CI.
Linked Issues check Passed Issue #302 requires a Garage CI job, bucket and key setup, S3 storage tests, both checksum settings, multipart coverage, cleanup checks, and a job under 15 minutes. The reviewed changes add the pinned…
Out of Scope Changes check Passed The changes stay related to #302. The local Garage Compose profile, Makefile target, test helpers, configuration settings, documentation, and multipart cleanup fix support S3 test execution, diagnosis…
Full details: Linked Issues check

Explanation

Issue #302 requires a Garage CI job, bucket and key setup, S3 storage tests, both checksum settings, multipart coverage, cleanup checks, and a job under 15 minutes. The reviewed changes add the pinned Garage job, create the bucket and key, add the requested save/open/delete, missing-file, over-cap, failed-upload, cleanup, checksum, and multipart tests, and set a 10-minute job timeout. The S3 cleanup fix supports the required cleanup behavior.

Full details: Out of Scope Changes check

Explanation

The changes stay related to #302. The local Garage Compose profile, Makefile target, test helpers, configuration settings, documentation, and multipart cleanup fix support S3 test execution, diagnosis, or operation. No unrelated change is established by the reviewed summary.

Full details: Docstring Coverage

Explanation

Docstring coverage is 57.89% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 38 functions across 7 files. (10 skipped: 10 unsupported.)

Resolution

Write docstrings for the functions missing them to satisfy the coverage threshold.

Finishing Touches: 💡 1
📝 Generate docstrings: 💡
  • Commit to this branch
  • Create a new PR
🧪 Generate unit tests (beta)
  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@codescene-access codescene-access Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gates Passed
4 Quality Gates Passed

See analysis details in CodeScene

Quality Gate Profile: Clean Code Collective
Install CodeScene MCP: safeguard and uplift AI-generated code. Catch issues early with our IDE extension and CLI tool.

@gitar-bot

gitar-bot Bot commented Oct 26, 2026 •

Copy link
Copy Markdown
Code Review ✅ Approved

🟡 Medium risk · S3 multipart aborts now outlive canceled upload contexts but are time-bounded, so a slow store could retain abandoned parts after timeout.

Adds a s3 CI job that runs the storage tests against a pinned Garage store under both checksum settings, plus a make test-s3 target and a fix so failed or over-cap multipart uploads abort their parts with a bounded cleanup context. No issues found.

Verification

Rules No rules evaluated

Functional validation Not enabled

Set up

For more verification details, comment gitar display:verbose.

Options

Auto-apply is off → Gitar will not commit updates to this branch.
Display: compact → Showing compact verification results.

Comment with these commands to change the behavior for this request:

Auto-apply Compact
gitar auto-apply:on         
gitar display:verbose         

Was this helpful? React with 👍 / 👎 | Powered by Gitar — free for open source

@codecov

codecov Bot commented Oct 26, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@greptile-apps

greptile-apps Bot commented Oct 26, 2026 •

Copy link
Copy Markdown

RetriggerView in GreptileConfidence Score: 5/5

[Medium impact] Safe to merge within the reviewed scope. The real-store integration suite and focused cancellation and timeout checks passed.

Summary

Adds real S3 storage tests using a pinned Garage container in CI and local development.

  • Failed S3 uploads now send a bounded cleanup abort.
  • Storage tests now run against a real Garage S3 store.

Validation completed successfully:

  • make test-s3 passed against the pinned Garage image with the race detector enabled.
  • The CI-style Garage startup, bootstrap, and S3 test sequence passed under both checksum settings.
  • Focused execution confirmed that cancelled multipart uploads send abort requests and that slow cleanup stops at the configured timeout.

No actionable issues found. The change is safe to merge within the reviewed scope.

Reviews (1) · Last reviewed commit: "docs(start): describe make test-s3" · Reviewed by Greptile

@SirLouen
SirLouen merged commit f0853a3 into main Oct 26, 2026
19 checks passed
@SirLouen
SirLouen deleted the fix/302 branch October 26, 2026 20:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Storage: prove the S3 driver against a real S3 store in CI

1 participant