[Bug report] Can not create fileset with Ceph RGW
- Dominant language
- Java
- Stars
- 3.2k
- Forks
- 935
- Avg merge
- 1d 16h
- Merged PRs (30d)
- 298
Description
### Version
main branch
### Describe what's wrong
When I create an S3 fileset catalog with the following properties, Gravitino appears that I am using path-style access for S3-compatible (Ceph RGW), but the S3 fileset does not support it
### Error message and/or stacktrace
I encounter the following error: Caused by: org.apache.hadoop.fs.s3a.AWSClientIOException: getFileStatus Name or service not known
### How to reproduce
`
"s3-access-key-id": "...",
"s3-secret-access-key": "...",
"s3-endpoint": ...,
"filesystem-providers": "s3",
"credential-providers": "s3-token",
"s3-region": "us-east-1",
"s3-role-arn": "...",
"s3-path-style-access": "true",
"s3-creds-provider": "s3-token"
`
### Additional context
_No response_
Contributor guide
Research direction
Start by reproducing the S3 fileset catalog setup against Ceph RGW with the listed path-style and credential properties, then trace how those properties reach the Hadoop S3A filesystem. Done means creating and accessing the fileset succeeds without the AWSClientIOException.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- backend, cloud
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100