apache / apache/iceberg-python

Delete/Overwrite/Upsert with string key containing dot (.) hangs indefinitely

未关闭
#2,512 3 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看
主要语言
Python
星标
1.1k
派生
581
平均合并
1 天 17 小时
30 天内合并 PR
78

描述

### Apache Iceberg version

None

### Please describe the bug 🐞

### Description

Any operation that rewrites data (delete/ overwrite/ upsert) is extremely slow or unresponsive, even on very small tables. Appending rows works as expected.

### Steps to Reproduce

```
from pyiceberg.schema import Schema
from pyiceberg.types import NestedField, StringType
import pyarrow as pa

# Create table
schema = Schema(
NestedField(field_id=1, name="load_id", field_type=StringType(), required=True),
NestedField(field_id=2, name="status", field_type=StringType(), required=True),
identifier_field_ids=[1],
)

catalog.create_table(identifier="test.test_load", schema=schema)

tbl = catalog.load_table("test.test_load")
df = pa.Table.from_pylist(
[
{"load_id": "123.123", "status": "started"},
{"load_id": "456.456", "status": "done"},
],
schema=tbl.schema().as_arrow(),
)
tbl.append(df)

# Delete with dot in string key → hangs indefinitely
tbl.delete(delete_filter="load_id == '123.123'")
```

### Observed Behavior

- append: completes quickly.
- delete with string containing dot (e.g., "123.123"): runs indefinitely or takes 10+ minutes without returning a response.
- overwrite and upsert with string containing dot: same behavior — operation never completes or is extremely slow.

### Environment

- PyIceberg version: `0.10.0`
- Catalog: `REST`
- Python version: `3.12`
- Storage: `AWS S3 tables`

### Willingness to contribute

- [ ] I can contribute a fix for this bug independently
- [ ] I would be willing to contribute a fix for this bug with guidance from the Iceberg community
- [ ] I cannot contribute a fix for this bug at this time

贡献指南

这个仓库没有索引到贡献指南

调研方向

首先针对 PyIceberg 表运行提供的 Python 复现代码,并跟踪带有过滤条件 load_id == '123.123' 的 tbl.delete 入口点。比较带点和不带点时 delete、overwrite 和 upsert 的行为;当这些操作对于包含点的字符串键能够及时完成时,即表示完成。

由索引模型根据 Issue 内容生成。

评估

技术栈
aws, python
领域
data-engineering, databases
Issue 类型
缺陷
难度
3/5
预计耗时
1-2 天
活跃度
冷清
描述清晰度
基本清楚
新手友好度
55/100

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。