apache / apache/iceberg-python

Table properties override catalog configuration when constructing FileIO

未关闭
#3,931 0 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看
bug
主要语言
Python
星标
1.1k
派生
581
平均合并
1 天 17 小时
30 天内合并 PR
78

描述

When a table is loaded, its metadata properties are merged into the property map used to construct the table's `FileIO`, and they take precedence over the operator's catalog configuration.

`Catalog._load_file_io` computes:

```python
load_file_io({**self.properties, **properties}, location)
```

where `properties` is `metadata.properties`. Because table properties come last, a value stored in a table overrides the same key configured on the catalog. The REST path does the same at `_response_to_table` / `_response_to_staged_table`, and Glue, Hive, SQL, DynamoDB, BigQuery and `StaticTable.from_metadata` all funnel table metadata into `FileIO` construction the same way — 12 call sites in total.

The keys this reaches include implementation selection (`py-io-impl`, `s3.retry-strategy-impl`) and transport configuration (`s3.endpoint`, `s3.proxy-uri`, `s3.signer` / `s3.signer.uri`, `gcs.service.host`, `hf.endpoint`, the ADLS storage authorities). These are deployment concerns — an operator sets them on the catalog — but any principal who can commit to a table can currently override them for everyone who reads it.

Issue investigation generated via claude, reviewed by Sung, Kevin, Fokko.

贡献指南

这个仓库没有索引到贡献指南

调研方向

从 Catalog._load_file_io 开始,跟踪 REST 的 _response_to_table 和 _response_to_staged_table 路径,然后检查 Glue、Hive、SQL、DynamoDB、BigQuery 和 StaticTable.from_metadata 对应的调用点。验证元数据属性如何合并到 FileIO 构造中,并添加覆盖测试,表明在所有受影响的路径上,catalog 配置对于 deployment 设置仍具有权威性。

由索引模型根据 Issue 内容生成。

评估

技术栈
python
领域
backend, security
Issue 类型
缺陷
难度
4/5
预计耗时
3-5 天
活跃度
活跃
描述清晰度
基本清楚
新手友好度
55/100

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。