apache / apache/gravitino

[Bug report] Can not create fileset with Ceph RGW

Open
#7,919 17 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
Java
Stars
3.2k
Forks
935
Avg merge
1d 16h
Merged PRs (30d)
298

Description

### Version

main branch

### Describe what's wrong

When I create an S3 fileset catalog with the following properties, Gravitino appears that I am using path-style access for S3-compatible (Ceph RGW), but the S3 fileset does not support it

### Error message and/or stacktrace

I encounter the following error: Caused by: org.apache.hadoop.fs.s3a.AWSClientIOException: getFileStatus Name or service not known

### How to reproduce

`
"s3-access-key-id": "...",
"s3-secret-access-key": "...",
"s3-endpoint": ...,
"filesystem-providers": "s3",
"credential-providers": "s3-token",
"s3-region": "us-east-1",
"s3-role-arn": "...",
"s3-path-style-access": "true",
"s3-creds-provider": "s3-token"
`

### Additional context

_No response_

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the S3 fileset catalog setup against Ceph RGW with the listed path-style and credential properties, then trace how those properties reach the Hadoop S3A filesystem. Done means creating and accessing the fileset succeeds without the AWSClientIOException.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
backend, cloud
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.