A file scan's Limit and Offset accept a negative value
- Dominant language
- Scala
- Stars
- 314
- Forks
- 187
- Avg merge
- 1d 21h
- Merged PRs (30d)
- 214
Description
### Task Summary
Every file-scan source takes a Limit and an Offset from `ScanSourceOpDesc`, and neither field says it cannot be negative. The form accepts `-1` for either one.
Neither value is meaningful below zero, and what happens to one is not the same everywhere it is read. The executors take the window with Scala's `drop` and `take`, where a negative `drop` is a no-op and a negative `take` yields nothing, so `offset = -1` returns every row and `limit = -1` returns none. The Arrow and Parquet sources take the same window in the exported script with `iloc`, where `-1` counts from the end instead: `iloc[-1:]` is the last row. A value nobody means to type lands somewhere different depending on which of the two is reading it.
Declaring `minimum: 0` on both fields states the constraint the operators already assume, and the form then refuses the value rather than passing it on. It reaches CSV, CSVOld, JSONL, Arrow, Parquet and the file scans at once, since all of them inherit the two fields.
This is a bound on the form, not a change to any executor: a plan submitted through the API can still carry a negative, and no operator's behaviour is altered by this task.
### Task Type
- [ ] Refactor / Cleanup
- [ ] DevOps / Deployment / CI
- [ ] Testing / QA
- [ ] Documentation
- [ ] Performance
- [X] Other
Contributor guide
Assessment
This issue has not been assessed yet.