apache / apache/iceberg-python
`DataScan` `count` method does not respect limit
- 主要语言
- Python
- 星标
- 1.1k
- 派生
- 581
- 平均合并
- 1 天 17 小时
- 30 天内合并 PR
- 78
描述
### Apache Iceberg version
0.9.1 (latest release)
### Please describe the bug 🐞
When calling `count()` on a `DataScan`, limit is not respected. Seems trivial but if I set a limit of 5 I expect 5 or less rows back, at least with a scan-like implementation
The underlying `ArrowScan` does not get passed the limit param
https://github.com/apache/iceberg-python/blob/main/pyiceberg/table/__init__.py#L1940
This results in scans taking longer due to not respecting the limit.
The fix will involve more than just passing the limit to the `ArrowScan`
### Willingness to contribute
- [x] I can contribute a fix for this bug independently
- [ ] I would be willing to contribute a fix for this bug with guidance from the Iceberg community
- [ ] I cannot contribute a fix for this bug at this time
贡献指南
这个仓库没有索引到贡献指南
调研方向
从 pyiceberg/table/__init__.py 第 1940 行附近开始,跟踪 DataScan.count 经过底层 ArrowScan 调用的执行路径。检查 limit 是如何传递的,并确定 issue 注释指出可能需要调整的额外扫描行为。当 count() 返回不超过请求的 limit,同时避免不必要的扫描工作时,即视为完成。
由索引模型根据 Issue 内容生成。
评估
- 技术栈
- python
- 领域
- data-engineering, databases
- Issue 类型
- 缺陷
- 难度
- 3/5
- 预计耗时
- 1-2 天
- 活跃度
- 活跃
- 描述清晰度
- 基本清楚
- 新手友好度
- 68/100