transport/http: Request.Build discards a caller-set ContentLength for *io.PipeReader, breaking SigV4
- Dominant language
- Java
- Stars
- 255
- Forks
- 81
- Avg merge
- 1d 6h
- Merged PRs (30d)
- 7
Description
### Describe the bug
`Request.Build` overwrites `ContentLength` with -1 when the stream is an
`*io.PipeReader`, even if the caller set it.
transport/http/request.go#L167-L179:
```go
switch stream := r.stream.(type) {
case *io.PipeReader:
req.Body = io.NopCloser(stream)
req.ContentLength = -1
```
The length is not incidental. `httpbinding.Encoder.Encode` deliberately hoists a
modeled `Content-Length` header onto `req.ContentLength`, because net/http
requires it there:
```go
// net/http ignores Content-Length header and requires it to be set on http.Request
if v := e.header.Get(contentLengthHeader); len(v) > 0 {
...
req.ContentLength = iv
e.header.Del(contentLengthHeader)
}
```
`Build` runs in `ClientHandler.Handle`, after Finalize. SigV4 has already signed
by then, and adds `content-length` to `SignedHeaders` whenever
`req.ContentLength > 0` (aws-sdk-go-v2 `aws/signer/v4/v4.go`,
`buildCanonicalHeaders`).
So the request goes out with `content-length` in `SignedHeaders`, no
`Content-Length` header, and `Transfer-Encoding: chunked`. A caller cannot
prevent this: the length is correct at signing time and discarded after.
### Expected behavior
`Build` fills in `ContentLength` only when it is unknown. A caller-set value
stands.
### Steps to reproduce
`Build` alone drops the length:
```go
req := smithyhttp.NewStackRequest().(*smithyhttp.Request)
pr, _ := io.Pipe()
req, _ = req.SetStream(pr)
req.ContentLength = 11
fmt.Println(req.Build(context.Background()).ContentLength) // -1, want 11
```
End to end against iDrive e2 on current `main`. Two PutObject calls, same
11-byte pipe, both `ContentLength: 11`, unsigned payload so the non-seekable
body streams. The second only hides the concrete type behind
`struct{ io.Reader }`:
```
*io.PipeReader -> smithy-probe-pipe
sending: ContentLength=-1 chunked=true
sending: signed=accept-encoding;amz-sdk-invocation-id;amz-sdk-request;content-length;content-type;host;x-amz-content-sha256;x-amz-date
RESULT: operation error S3: PutObject, https response error StatusCode: 411, RequestID: 18D17208B3E9ECAE, api error MissingContentLength: You must provide the Content-Length HTTP header.
wrapped -> smithy-probe-wrapped
sending: ContentLength=11 chunked=false
sending: signed=accept-encoding;amz-sdk-invocation-id;amz-sdk-request;content-length;content-type;host;x-amz-content-sha256;x-amz-date
RESULT: ok
```
Both sign `content-length`. Only the wrapped one sends it. MinIO
RELEASE.2025-09-07T16-13-09Z answers the same 411; a proxy in front of it
confirms the bare pipe arrives as `Transfer-Encoding: chunked` with no
`Content-Length` header. Backends that do not require the header but do verify
the signature reject it as 403 SignatureDoesNotMatch instead.
The script below clones `aws/smithy-go` at `main`, runs the program against it,
then clones a branch carrying a fix and runs the identical program again. The
only thing that changes between the two runs is which checkout is replaced in.
It works in a temp directory and deletes the two objects it writes.
```
S3_ENDPOINT=https://... S3_BUCKET=my-bucket S3_REGION=us-west-1 S3_ACCESS_KEY=... S3_SECRET_KEY=... ./reproduce.sh
```
No S3 account needed: a local MinIO reproduces it identically.
```
docker run -d --name smithy-minio -p 9000:9000 -e MINIO_ROOT_USER=probeuser -e MINIO_ROOT_PASSWORD=probepassword minio/minio:latest server /data
docker run --rm --network host -e MC_HOST_p=http://probeuser:probepassword@127.0.0.1:9000 minio/mc:latest mb -p p/probe-bucket
S3_ENDPOINT=http://127.0.0.1:9000 S3_BUCKET=probe-bucket S3_ACCESS_KEY=probeuser S3_SECRET_KEY=probepassword ./reproduce.sh
```
reproduce.sh
```bash
#!/usr/bin/env bash
#
# Reproduces the *io.PipeReader ContentLength bug against a real
# S3-compatible backend, then re-runs the identical program against a
# smithy-go that carries the fix. The only difference between the two runs is
# which smithy-go checkout is replaced in.
#
# Nothing is left behind: it works in a temp dir and deletes the two objects
# it writes.
#
# Usage:
# S3_ENDPOINT=https://... S3_BUCKET=my-bucket S3_REGION=us-west-1 \
# S3_ACCESS_KEY=... S3_SECRET_KEY=... ./reproduce.sh
#
# SMITHY_BASE/SMITHY_FIX may each be a git URL or a local checkout;
# SMITHY_BASE_REF/SMITHY_FIX_REF select the branch to clone.
set -euo pipefail
: "${S3_ENDPOINT:?set S3_ENDPOINT}"
: "${S3_BUCKET:?set S3_BUCKET}"
: "${S3_ACCESS_KEY:?set S3_ACCESS_KEY}"
: "${S3_SECRET_KEY:?set S3_SECRET_KEY}"
export S3_ENDPOINT S3_BUCKET S3_ACCESS_KEY S3_SECRET_KEY
export S3_REGION=${S3_REGION:-us-east-1}
SMITHY_BASE=${SMITHY_BASE:-https://github.com/aws/smithy-go}
SMITHY_BASE_REF=${SMITHY_BASE_REF:-main}
SMITHY_FIX=${SMITHY_FIX:-https://github.com/afreidah/smithy-go}
SMITHY_FIX_REF=${SMITHY_FIX_REF:-fix/pipereader-content-length}
work=$(mktemp -d)
trap 'rm -rf "$work"' EXIT
cd "$work"
cat >go.mod <<'GOMOD'
module smithyprobe
go 1.24
GOMOD
cat >main.go <<'PROBE'
// Writes the same object twice: once with a bare *io.PipeReader body, once
// with that pipe hidden behind an opaque io.Reader. Both calls pass the same
// ContentLength, so both must be framed the same way on the wire.
package main
import (
"context"
"fmt"
"io"
"net/http"
"os"
"strings"
"github.com/aws/aws-sdk-go-v2/aws"
v4 "github.com/aws/aws-sdk-go-v2/aws/signer/v4"
"github.com/aws/aws-sdk-go-v2/credentials"
"github.com/aws/aws-sdk-go-v2/service/s3"
"github.com/aws/smithy-go/middleware"
smithyhttp "github.com/aws/smithy-go/transport/http"
)
const payload = "hello world"
// reporter reports what the transport was handed after every middleware,
// signing included. Go frames a request with ContentLength -1 as chunked and
// sends no Content-Length header.
type reporter struct{ next http.RoundTripper }
func (t reporter) RoundTrip(r *http.Request) (*http.Response, error) {
if r.Method != http.MethodPut {
return t.next.RoundTrip(r)
}
fmt.Printf(" sending: ContentLength=%d chunked=%t\n", r.ContentLength, r.ContentLength < 0)
fmt.Printf(" sending: signed=%s\n", signedHeaders(r.Header.Get("Authorization")))
return t.next.RoundTrip(r)
}
func signedHeaders(auth string) string {
for _, part := range strings.Split(auth, ",") {
if v, ok := strings.CutPrefix(strings.TrimSpace(part), "SignedHeaders="); ok {
return v
}
}
return "(none)"
}
func pipe() *io.PipeReader {
pr, pw := io.Pipe()
go func() { _, _ = pw.Write([]byte(payload)); _ = pw.Close() }()
return pr
}
// buildLabel shows what Request.Build does with a known length, so the two
// halves of a comparison run cannot be mistaken for each other.
func buildLabel() string {
req := smithyhttp.NewStackRequest().(*smithyhttp.Request)
req, err := req.SetStream(pipe())
if err != nil {
return "unknown: " + err.Error()
}
req.ContentLength = int64(len(payload))
return fmt.Sprintf("set ContentLength=%d -> Build gives %d",
len(payload), req.Build(context.Background()).ContentLength)
}
func main() {
fmt.Printf("smithy-go: %s\n\n", buildLabel())
cli := s3.New(s3.Options{
Region: os.Getenv("S3_REGION"),
Credentials: credentials.NewStaticCredentialsProvider(
os.Getenv("S3_ACCESS_KEY"), os.Getenv("S3_SECRET_KEY"), ""),
BaseEndpoint: aws.String(os.Getenv("S3_ENDPOINT")),
UsePathStyle: true,
HTTPClient: &http.Client{Transport: reporter{next: http.DefaultTransport}},
// Unsigned payload: the only way a non-seekable body streams at all.
APIOptions: []func(*middleware.Stack) error{v4.SwapComputePayloadSHA256ForUnsignedPayloadMiddleware},
RetryMaxAttempts: 1,
})
bucket := os.Getenv("S3_BUCKET")
bare := put(cli, bucket, "smithy-probe-pipe", "*io.PipeReader", pipe())
wrapped := put(cli, bucket, "smithy-probe-wrapped", "wrapped ", struct{ io.Reader }{pipe()})
cleanup(cli, bucket, "smithy-probe-pipe", "smithy-probe-wrapped")
switch {
case !wrapped:
fmt.Println("INCONCLUSIVE: the control upload failed, so the backend or the credentials are at fault")
os.Exit(3)
case !bare:
fmt.Println("REPRODUCED: the same stream is rejected only because it is an *io.PipeReader")
os.Exit(1)
default:
fmt.Println("FIXED: both bodies frame identically and the backend accepts both")
}
}
func put(cli *s3.Client, bucket, key, label string, body io.Reader) bool {
fmt.Printf("%s -> %s\n", label, key)
_, err := cli.PutObject(context.Background(), &s3.PutObjectInput{
Bucket: aws.String(bucket),
Key: aws.String(key),
Body: body,
ContentLength: aws.Int64(int64(len(payload))),
})
if err != nil {
fmt.Printf(" RESULT: %v\n\n", err)
return false
}
fmt.Printf(" RESULT: ok\n\n")
return true
}
// cleanup removes anything the probe managed to write.
func cleanup(cli *s3.Client, bucket string, keys ...string) {
for _, k := range keys {
if _, err := cli.DeleteObject(context.Background(), &s3.DeleteObjectInput{
Bucket: aws.String(bucket), Key: aws.String(k),
}); err != nil {
fmt.Printf("cleanup %s: %v\n", k, err)
}
}
}
PROBE
# checkout resolves a source that is either an existing directory or a git URL
# to clone, and echoes the path to use.
checkout() {
local src=$1 ref=$2 dest=$3
if [ -d "$src" ]; then
echo "$src"
return
fi
git clone --quiet --depth 1 --branch "$ref" "$src" "$dest"
echo "$dest"
}
run_against() {
local title=$1 path=$2
echo
echo "=============================================================="
echo " $title"
echo "=============================================================="
git -C "$path" --no-pager log -1 --format='checkout: %h %s'
go mod edit -replace "github.com/aws/smithy-go=$path"
go mod tidy >/dev/null 2>&1
echo
local rc=0
go run . || rc=$?
return "$rc"
}
base_path=$(checkout "$SMITHY_BASE" "$SMITHY_BASE_REF" "$work/smithy-base")
fix_path=$(checkout "$SMITHY_FIX" "$SMITHY_FIX_REF" "$work/smithy-fix")
before=0
run_against "aws/smithy-go $SMITHY_BASE_REF (expect: the bare pipe is rejected)" "$base_path" || before=$?
after=0
run_against "fork $SMITHY_FIX_REF (expect: both uploads accepted)" "$fix_path" || after=$?
echo
echo "=============================================================="
echo " summary"
echo "=============================================================="
printf ' %-40s exit %d\n' "aws/smithy-go $SMITHY_BASE_REF" "$before"
printf ' %-40s exit %d\n' "fork $SMITHY_FIX_REF" "$after"
echo
if [ "$before" -eq 1 ] && [ "$after" -eq 0 ]; then
echo "PASS: reproduced on upstream, gone with the fix"
exit 0
fi
echo "did not get the expected 1 then 0" >&2
exit 1
```
Output, upstream `main` first and the fix second:
```
============================================================
aws/smithy-go main (expect: the bare pipe is rejected)
============================================================
checkout: 959c6ab Keep a trailing slash when JoinPath right operand is a slash. (#698)
smithy-go: set ContentLength=11 -> Build gives -1
*io.PipeReader sending: ContentLength=-1 chunked=true RESULT: 411 MissingContentLength
wrapped sending: ContentLength=11 chunked=false RESULT: ok
REPRODUCED
============================================================
fork fix/pipereader-content-length (expect: both accepted)
============================================================
checkout: d27f2ff Keep a caller-supplied ContentLength for *io.PipeReader bodies.
smithy-go: set ContentLength=11 -> Build gives 11
*io.PipeReader sending: ContentLength=11 chunked=false RESULT: ok
wrapped sending: ContentLength=11 chunked=false RESULT: ok
FIXED
aws/smithy-go main exit 1
fork fix/pipereader-content-length exit 0
PASS
```
### Workaround
Wrap non-seekable bodies in a `Read`-only type so the SDK cannot see the
concrete type.
```go
type measuredStream struct{ r io.Reader }
func (s measuredStream) Read(p []byte) (int, error) { return s.r.Read(p) }
```
### Versions
github.com/aws/smithy-go main @ 959c6ab (also reproduced on v1.28.1 and v1.27.9)
github.com/aws/aws-sdk-go-v2 v1.43.7
github.com/aws/aws-sdk-go-v2/service/s3 v1.107.3
go1.27.0 linux/amd64
Contributor guide
Research direction
Start in transport/http/request.go at Request.Build and reproduce the issue with the provided *io.PipeReader example, then inspect the surrounding stream handling. Done means a caller-supplied ContentLength remains unchanged for pipe readers and the request is sent with that length instead of chunked transfer encoding; run the reproduction against an S3-compatible backend to verify both body forms.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- api, backend
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 84/100