Skip to content

fix(s3): send explicit Content-Length on every request - #171

Closed
ArnabChatterjee20k wants to merge 1 commit into
utopia-php:mainfrom
ArnabChatterjee20k:s3-fix
Closed

fix(s3): send explicit Content-Length on every request#171
ArnabChatterjee20k wants to merge 1 commit into
utopia-php:mainfrom
ArnabChatterjee20k:s3-fix

Conversation

@ArnabChatterjee20k

Copy link
Copy Markdown
Contributor

Problem

Uploading large (chunked) files to S3-compatible storage — reported against GCS — fails with:

HTTP 411 — POST requests require a Content-length header

Reported downstream in appwrite/appwrite#13548 (self-hosted 2.0, files 15 MB–550 MB).

Cause

S3::call() handed a streamed body to the utopia-php/client transport without a Content-Length header. The cURL adapter then:

  • streams a body of unknown size (StreamInterface::getSize() returns null) with Transfer-Encoding: chunked, and
  • emits no length header at all for an empty-body POST (e.g. createMultipartUpload).

Many S3-compatible services (GCS in particular) reject both with 411.

Fix

hashBody() already reads the whole (seekable) body once to sign it, so it now also counts the bytes and returns the length. call() sets an explicit, signed content-length header on every request (0 for empty bodies). The value is the full body size, so it matches what the transport already sends for a size-known stream — no behavioural change for those, and correct headers for the streamed / empty-body cases.

No new reads of the body are introduced.

Testing

  • php -l clean. Existing signing path unchanged apart from the added (signed) header.
  • Vendor deps not installed in this checkout, so the PHPUnit/PHPStan suites were not run locally — CI will cover them.

Fixes appwrite/appwrite#13548

Uploading large (chunked) files to S3-compatible storage such as GCS
failed with HTTP 411 "POST requests require a Content-length header".

call() handed a streamed body to the transport without a Content-Length
header. The cURL adapter then streams a body of unknown size with
Transfer-Encoding: chunked, and emits no length at all for an empty-body
POST (createMultipartUpload) — both of which S3-compatible services
reject with 411.

hashBody() already reads the whole (seekable) body once to sign it, so
count the bytes there and set an explicit, signed content-length header
for every request (0 for empty bodies). The value equals the full body
size, matching what the transport sends for a size-known stream.

Fixes appwrite/appwrite#13548
@github-actions

Copy link
Copy Markdown

Thanks for contributing! This repository is a read-only mirror; development for this library happens in packages/storage in the utopia-php monorepo. Please open this pull request there instead.

@github-actions github-actions Bot closed this Sep 10, 2026
@greptile-apps

greptile-apps Bot commented Sep 10, 2026

Copy link
Copy Markdown

RetriggerConfidence Score: 4/5

The implementation appears safe to merge, with a non-blocking but important regression-coverage gap around the exact S3 request behavior being fixed.

Fix All in Claude CodeFindings

  1. P2 Content-Length behavior lacks coverage
Fix with agent prompt
### Issue 1
src/Storage/Device/S3.php:756
The new request behavior is not covered by regression tests. Existing request tests do not assert the observable `Content-Length`, and multipart preparation bypasses the real request-building path. Please add request-level tests showing that an unknown-size seekable stream sends its full byte count and that an empty multipart-initiation POST sends `Content-Length: 0`; otherwise either part of the reported failure could return without CI detecting it.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Summary

  • Preserves the existing full-stream MD5 and SHA-256 signing pass.
  • Applies an explicit length to all shared S3-compatible request paths.
  • Lacks observable request-level regression tests for the two cases the change is intended to fix.

Reviews (1) · Last reviewed commit: "fix(s3): send explicit Content-Length on..."

Comment thread src/Storage/Device/S3.php
// or omits the header on an empty POST — which S3-compatible services such
// as GCS reject with HTTP 411. The value is the full body size, so it also
// matches what the transport sends for a size-known stream.
$headers['content-length'] = (string) $length;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Content-Length behavior lacks coverage

The new request behavior is not covered by regression tests. Existing request tests do not assert the observable Content-Length, and multipart preparation bypasses the real request-building path. Please add request-level tests showing that an unknown-size seekable stream sends its full byte count and that an empty multipart-initiation POST sends Content-Length: 0; otherwise either part of the reported failure could return without CI detecting it.

Knowledge Base Used: S3-compatible storage devices

Prompt To Fix With AI
This is a comment left during a code review.
Path: src/Storage/Device/S3.php
Line: 756

Comment:
**Content-Length behavior lacks coverage**

The new request behavior is not covered by regression tests. Existing request tests do not assert the observable `Content-Length`, and multipart preparation bypasses the real request-building path. Please add request-level tests showing that an unknown-size seekable stream sends its full byte count and that an empty multipart-initiation POST sends `Content-Length: 0`; otherwise either part of the reported failure could return without CI detecting it.

**Knowledge Base Used:** [S3-compatible storage devices](https://app.greptile.com/appwrite/-/custom-context/knowledge-base/utopia-php/storage/-/docs/s3-compatible-storage-devices.md)

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

Fix in Claude Code Fix in Codex

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2.0 selfhost: Files fail to upload to s3 compatible storages due to missing Content-length header

1 participant