apache / apache/iceberg-python
`DataScan` `count` method does not respect limit
- 主要語言
- Python
- 星號
- 1.1k
- 分支
- 581
- 平均合併
- 1 天 13 小時
- 30 天內合併 PR
- 76
描述
### Apache Iceberg version
0.9.1 (latest release)
### Please describe the bug 🐞
When calling `count()` on a `DataScan`, limit is not respected. Seems trivial but if I set a limit of 5 I expect 5 or less rows back, at least with a scan-like implementation
The underlying `ArrowScan` does not get passed the limit param
https://github.com/apache/iceberg-python/blob/main/pyiceberg/table/__init__.py#L1940
This results in scans taking longer due to not respecting the limit.
The fix will involve more than just passing the limit to the `ArrowScan`
### Willingness to contribute
- [x] I can contribute a fix for this bug independently
- [ ] I would be willing to contribute a fix for this bug with guidance from the Iceberg community
- [ ] I cannot contribute a fix for this bug at this time
貢獻指南
這個儲存庫沒有索引到貢獻指南
研究方向
從 pyiceberg/table/__init__.py 第 1940 行附近開始,追蹤 DataScan.count 經過底層 ArrowScan 呼叫的執行路徑。檢查 limit 如何傳遞,並找出 issue 註記指出可能需要調整的額外掃描行為。當 count() 回傳不超過要求的 limit,同時避免不必要的掃描工作時,即視為完成。
由索引模型根據 Issue 內容生成。
評估
- 技術堆疊
- python
- 領域
- data-engineering, databases
- Issue 類型
- 缺陷
- 難度
- 3/5
- 預估耗時
- 1-2 天
- 活躍度
- 活躍
- 描述清晰度
- 基本清楚
- 新手友好度
- 68/100