[Bug] Paimon s3 configuration conflicts with Flink s3 configuration
- Dominant language
- Java
- Stars
- 3.4k
- Forks
- 1.4k
- Avg merge
- 1d 11h
- Merged PRs (30d)
- 396
Description
### Search before asking
- [X] I searched in the [issues](https://github.com/apache/incubator-paimon/issues) and found nothing similar.
### Paimon version
paimon-flink-1.17-0.5-20230814.001854-121.jar
### Compute Engine
Flink Version:1.17.1
### Minimal reproduce step
I use paimon with s3 storage and configure s3 access through Flink FlleSystem(flink-s3-fs-hadoop-1.17.1.jar).
I want to use different s3 storage to configure paimon directory and savpoint and checkpoint of flink
My s3 configuration in flink is
s3.access-key=access-key1
s3.secret-key=serect-key1
s3.endpoint=http://endpoint1:80
My paimon cataglog ddl:
```sql
CREATE CATALOG my_catalog WITH (
'type' = 'paimon',
'warehouse' = 's3a://mybucket/',
's3.endpoint' = 'http://endpoint2:80',
's3.access-key' = 'serect-key2',
's3.secret-key' = 'access-key2'
);
```
However, It' seem like some confliction happen.
### What doesn't meet your expectations?


It seems that when paimon accesses s3, it uses the Flink configuration instead of the configuration in the catalog.
### Anything else?
_No response_
### Are you willing to submit a PR?
- [ ] I'm willing to submit a PR!
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with the Paimon catalog DDL and the Flink S3 filesystem configuration described in the reproduction, then trace where catalog options and Flink settings meet during S3 access. Reproduce the setup with separate endpoints and credentials; done means Paimon uses the catalog configuration without conflicting with Flink's checkpoint and savepoint configuration.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws
- Domain
- cloud
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100