apache / apache/iceberg-python
Delete/Overwrite/Upsert with string key containing dot (.) hangs indefinitely
- Dominant language
- Python
- Stars
- 1.1k
- Forks
- 581
- Avg merge
- 1d 17h
- Merged PRs (30d)
- 78
Description
### Apache Iceberg version
None
### Please describe the bug 🐞
### Description
Any operation that rewrites data (delete/ overwrite/ upsert) is extremely slow or unresponsive, even on very small tables. Appending rows works as expected.
### Steps to Reproduce
```
from pyiceberg.schema import Schema
from pyiceberg.types import NestedField, StringType
import pyarrow as pa
# Create table
schema = Schema(
NestedField(field_id=1, name="load_id", field_type=StringType(), required=True),
NestedField(field_id=2, name="status", field_type=StringType(), required=True),
identifier_field_ids=[1],
)
catalog.create_table(identifier="test.test_load", schema=schema)
tbl = catalog.load_table("test.test_load")
df = pa.Table.from_pylist(
[
{"load_id": "123.123", "status": "started"},
{"load_id": "456.456", "status": "done"},
],
schema=tbl.schema().as_arrow(),
)
tbl.append(df)
# Delete with dot in string key → hangs indefinitely
tbl.delete(delete_filter="load_id == '123.123'")
```
### Observed Behavior
- append: completes quickly.
- delete with string containing dot (e.g., "123.123"): runs indefinitely or takes 10+ minutes without returning a response.
- overwrite and upsert with string containing dot: same behavior — operation never completes or is extremely slow.
### Environment
- PyIceberg version: `0.10.0`
- Catalog: `REST`
- Python version: `3.12`
- Storage: `AWS S3 tables`
### Willingness to contribute
- [ ] I can contribute a fix for this bug independently
- [ ] I would be willing to contribute a fix for this bug with guidance from the Iceberg community
- [ ] I cannot contribute a fix for this bug at this time
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by running the provided Python reproduction against the PyIceberg table and trace the tbl.delete entry point with the filter load_id == '123.123'. Compare delete, overwrite, and upsert behavior with and without the dot; done means these operations complete promptly for string keys containing dots.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, python
- Domain
- data-engineering, databases
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 55/100