apache / apache/iceberg-python

s3 enpoint url in table.io.properties is different than that of catalog.properties

未关闭
#3,163 1 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看
stale
主要语言
Python
星标
1.1k
派生
581
平均合并
1 天 17 小时
30 天内合并 PR
78

描述

### Apache Iceberg version

0.11.0 (latest release)

### Please describe the bug 🐞

I am using polaris as rest catalog and RustFS as S3 client. My problem is that the s3 endpoint that i set in the `pyiceberg.yaml` file is not being used by the table.io (although catalog has it correctly)

The url is changed from `http://192.168.0.72:9008` to `localhost:9008` for the table.io.properties.get("s3.endpoint") property.

Here is a minimal example for it:

```
catalog = load_catalog("default", **catalog_conf)
catalog.create_namespace("Dummy")
schema = pa.schema([
pa.field("a", pa.int32()),
pa.field("b", pa.int32()),
])
catalog.create_table("Dummy.new_table", schema)
table = catalog.load_table("Dummy.new_table")
table.io.properties["s3.endpoint"] == catalog.properties["s3.endpoint"]

```

### Willingness to contribute

- [ ] I can contribute a fix for this bug independently
- [x] I would be willing to contribute a fix for this bug with guidance from the Iceberg community
- [ ] I cannot contribute a fix for this bug at this time

贡献指南

这个仓库没有索引到贡献指南

调研方向

从 load_catalog 和 load_table 使用的 catalog 和 table IO 属性处理开始,然后使用提供的最小示例和 pyiceberg.yaml 配置重现不匹配问题。当 table.io.properties["s3.endpoint"] 保留配置的 http://192.168.0.72:9008 值并与 catalog.properties["s3.endpoint"] 匹配时,即表示完成。

由索引模型根据 Issue 内容生成。

评估

技术栈
python
领域
data-engineering
Issue 类型
缺陷
难度
3/5
预计耗时
1-2 天
活跃度
活跃
描述清晰度
基本清楚
新手友好度
67/100

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。