[Feature](lance) Expand file TVF path and multi-dataset support
- Dominant language
- Java
- Stars
- 15.9k
- Forks
- 3.9k
- Avg merge
- 2d 23h
- Merged PRs (30d)
- 520
Description
## Parent
Tracked by #66340. The baseline s3() and local() Lance TVFs were introduced by #65730.
## Scope
- [ ] Expand glob patterns to multiple Lance datasets.
- [ ] Read multiple Lance datasets in one TVF invocation.
- [ ] Support additional file TVF entry points and broader path handling.
- [ ] Preserve dataset boundaries, version pinning, schema compatibility, and secure path handling.
## Completion criteria
- [ ] FE split planning and BE execution support every selected dataset.
- [ ] Tests cover glob expansion, mixed paths, multiple datasets, and incompatible schemas.
- [ ] User documentation describes supported TVFs, paths, and limitations.
- [ ] Representative local and object-storage paths are verified.
Contributor guide
Research direction
Start with the existing s3() and local() Lance TVFs introduced by #65730, then review the parent issue #66340. Trace FE split planning and BE execution for multiple datasets, and identify tests for glob expansion, mixed paths, schema compatibility, and local or object-storage paths. Done includes passing coverage and updated user documentation.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, sql
- Domain
- backend, data-engineering, database
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100