apache / apache/doris

[Bug] AWS S3 Express One Zone MD5 Check

Open
#58,779 3 comments 0 reactions 1 assignee Claimed by @Tech-Circle-48 View on GitHub
Dominant language
Java
Stars
15.9k
Forks
3.9k
Avg merge
2d 23h
Merged PRs (30d)
520

Description

### Search before asking

- [x] I had searched in the [issues](https://github.com/apache/doris/issues?q=is%3Aissue) and found no similar issues.

### Version

4.0.1 and 3.1.3

### What's Wrong?

Md5 Error on Data Load into a storage vault backed by S3 Express One Zone

errCode = 2, detailMessage = (10.11.5.44)[INTERNAL_ERROR][INTERNAL_ERROR]VNodeChannel[1764794939571-1764794803457], load_id=9f044b91381c4505-aaff8ae8126afff7, txn_id=42520010534912, node=10.11.5.199:8060, add batch req success but status isn't ok,

err: [INTERNAL_ERROR]PStatus: (10.11.5.199)[INTERNAL_ERROR]failed to put object: s3://doris-pa-s3/doris/data/1764794939574/020000000000005632439e37df2f380b93b2bae14617a0ae_0.dat: NotImplemented Unable to parse ExceptionName: NotImplemented Message: This bucket does not support Content Md5 header. code=501 type=100, request_id=01b1b43a4a00019ae86f842e0407ffcae4016fab
S3信息:

ctime: 1764794825 mtime: 1764794825 bucket: "doris-pa-s3" prefix: "doris" endpoint: "https://s3express-use1-az4.us-east-1.amazonaws.com/" region: "us-east-1" provider: S3 external_endpoint: "" sse_enabled: false use_path_style: false cred_provider_type: INSTANCE_PROFILE role_arn: "arn:aws:iam::1234567890:role/warehouse-benchmarks-express-doris-s3"

### What You Expected?

Succesfull load of data for use in Doris

### How to Reproduce?

Create new S3 Express One Zone bucket, use Load Command, receive error

### Anything Else?

I believe that the Doris Docker image for 4.0.2 has a patch for this, but I cannot find any binaries to test in this environment.

### Are you willing to submit PR?

- [ ] Yes I am willing to submit a PR!

### Code of Conduct

- [x] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.