apache / apache/polaris

Alibaba Cloud OSS Compatibility - AWS Chunked Encoding Header Mismatch

Open
#3,346 10 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Java
Stars
2.1k
Forks
522
Avg merge
1d 22h
Merged PRs (30d)
137

Description

### Is your feature request related to a problem? Please describe.

Apache Polaris fails to create Iceberg tables on Alibaba Cloud OSS due to incompatibility between AWS SDK v2's chunked encoding implementation and OSS's requirements.

aws-chunked encoding is not supported with the specified x-amz-content-sha256 value.
(Service: S3, Status Code: 400, Request ID: 69511DB18FDF233733116693)

InvalidArgument
aws-chunked encoding is not supported with the specified x-amz-content-sha256 value.
x-amz-content-sha256
STREAMING-UNSIGNED-PAYLOAD-TRAILER

### Describe the solution you'd like

### Root Cause

When Polaris writes Iceberg metadata files to OSS, the AWS SDK v2 (version 2.35.0) automatically enables chunked transfer encoding and sets:
```
Content-Encoding: aws-chunked
x-amz-content-sha256: STREAMING-UNSIGNED-PAYLOAD-TRAILER
x-amz-trailer: x-amz-checksum-crc32
```

However, Alibaba Cloud OSS **only accepts**:
```
x-amz-content-sha256: STREAMING-AWS4-HMAC-SHA256-PAYLOAD
```

According to [Alibaba OSS documentation](https://www.alibabacloud.com/help/en/oss/user-guide/0017-00000804), chunked encoding is supported, but only with the `STREAMING-AWS4-HMAC-SHA256-PAYLOAD` signature method.

### Technical Details

**Environment:**
- Apache Polaris: 1.2.0-incubating
- Apache Iceberg: 1.10.0
- AWS SDK for Java: v2.35.0 (embedded)
- Storage: Alibaba Cloud OSS
- Endpoint: `https://bucket-name.oss-region-internal.aliyuncs.com` (virtual hosted style)

**Request Headers Sent by SDK:**
```
PUT /warehouse/namespace/table/metadata/00000-xxx.metadata.json HTTP/1.1
Host: dev-polaris.oss-me-central-1.aliyuncs.com
Content-Encoding: aws-chunked
Content-Type: application/octet-stream
x-amz-content-sha256: STREAMING-UNSIGNED-PAYLOAD-TRAILER
x-amz-decoded-content-length: 934
x-amz-sdk-checksum-algorithm: CRC32
x-amz-trailer: x-amz-checksum-crc32
User-Agent: s3fileio/{iceberg-version=Apache Iceberg 1.10.0} aws-sdk-java/2.35.0
```

**OSS Requires:**
```
x-amz-content-sha256: STREAMING-AWS4-HMAC-SHA256-PAYLOAD

### Describe alternatives you've considered

advancedConfig:
"s3.chunked-encoding-enabled": "false"

### Additional context

Option 1 (Preferred): Add configuration option to control the signing method for chunked encoding:
advancedConfig:
"s3.chunked-encoding-signature-method": "STREAMING-AWS4-HMAC-SHA256-PAYLOAD"
Option 2: Add OSS-specific profile that configures SDK appropriately:
advancedConfig:
"s3.storage-provider": "alibaba-oss" # Auto-configures OSS compatibility
Option 3: Expose AWS SDK ClientOverrideConfiguration to allow custom interceptors:
advancedConfig:
"s3.enable-trailing-checksum": "false" # Prevents STREAMING-UNSIGNED-PAYLOAD-TRAILER
```

### Impact

This affects any organization using:
- Apache Polaris with Alibaba Cloud OSS
- Apache Polaris with any S3-compatible storage that doesn't support `STREAMING-UNSIGNED-PAYLOAD-TRAILER`
- Potentially other S3-compatible services beyond Alibaba Cloud

### References

- Alibaba OSS Chunked Encoding Error: https://www.alibabacloud.com/help/en/oss/user-guide/0017-00000804
- AWS SDK v2 Signing Documentation: https://docs.aws.amazon.com/general/latest/gr/sigv4-streaming.html
- Related: AWS SDK hardcodes this value in `DefaultV4RequestSigner` with no user-facing configuration

### Additional Context

Alibaba Cloud support confirmed the issue and recommended either:
1. Configuring SDK to use `STREAMING-AWS4-HMAC-SHA256-PAYLOAD` (but no config option exists in AWS SDK v2)
2. Disabling chunked encoding (current workaround)
3. Using standard multipart upload instead

The challenge is that AWS SDK v2 provides no public API to control the `x-amz-content-sha256` header value when chunked encoding is enabled.

### Logs

Debug logs showing the error
```
2025-12-28 12:08:17,013 DEBUG [sof.ama.aws.htt.aut.aws.int.sig.DefaultV4RequestSigner] AWS4 Canonical Request: PUT
/warehouse/test_db_pl1/test_tblpl1/metadata/00000-9340564a-0abc-4fc0-92d0-75dfb09e9c91.metadata.json

amz-sdk-invocation-id:e39f76e7-eb4c-3ae1-2ac0-858ccb50d103
amz-sdk-request:attempt=1; max=6
content-encoding:aws-chunked
content-length:977
content-type:application/octet-stream
host:dev-polaris.oss-me-central-1.aliyuncs.com
x-amz-content-sha256:STREAMING-UNSIGNED-PAYLOAD-TRAILER
x-amz-date:20251228T120817Z
x-amz-decoded-content-length:934
x-amz-sdk-checksum-algorithm:CRC32
x-amz-trailer:x-amz-checksum-crc32

2025-12-28 12:08:17,296 DEBUG [sof.ama.aws.request] Received failed response: 400, Request ID: 69511DB18FDF233733116693

2025-12-28 12:08:17,310 INFO [org.apa.pol.ser.exc.IcebergExceptionMapper] Handling runtimeException aws-chunked encoding is not supported with the specified x-amz-content-sha256 value. (Service: S3, Status Code: 400, Request ID: 69511DB18FDF233733116693)
```

---

## Alternative - Shorter Version for GitHub

If you want a more concise version:

---

**Title:** Add support for Alibaba Cloud OSS (chunked encoding compatibility)

**Description:**

Apache Polaris fails to write to Alibaba Cloud OSS with error:
```
aws-chunked encoding is not supported with the specified x-amz-content-sha256 value

Contributor guide

Open the contributing guide

Research direction

Start with the AWS SDK v2 DefaultV4RequestSigner behavior described in the issue and Polaris's advancedConfig handling for S3 settings. Compare the proposed signing-method, trailing-checksum, and disabled-chunked-encoding options, then verify that Iceberg metadata writes succeed against Alibaba Cloud OSS without regressing standard S3-compatible storage.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, java
Domain
cloud
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.