lance-format / lance-format/lance
bug: Session::new eagerly creates a LocalSpillStore temp dir and panics when it cannot
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 7.1k
- Forks
- 852
- Avg merge
- 3d 18h
- Merged PRs (30d)
- 272
Description
Description
Session::new builds a spill store eagerly, and its constructor panics rather than returning an error. rust/lance/src/session.rs:130 and :150 both do Arc::new(LocalSpillStore::default()), and the Default impl is Self::new().expect("failed to create temp directory for LocalSpillStore") (rust/lance-io/src/spill.rs:233). So every session creates a temp directory up front, whether or not anything ever spills, and a machine where that directory cannot be created takes down any code path that opens a dataset.
On Windows CI runners this happens intermittently, and it lands on tests that have nothing to do with spilling. From windows-build, in io::commit::conflict_resolver::tests::test_data_overlay_finish_conflicts_with_row_moving_update::case_2_coverage_disjoint_from_moved_row:
failed to create temp directory for LocalSpillStore: IO { source: Custom { kind: PermissionDenied,
error: PathError { path: "C:\\Users\\RUNNER~1\\AppData\\Local\\Temp\\.tmp9uNyQx",
err: Os { code: 5, kind: PermissionDenied, message: "Access is denied." } } }
Why it matters
The panic comes out of a Default impl, so no caller can handle it: a library user gets a process abort where an error would do. It also fires for sessions that never spill, since nothing about Session::new knows whether a query will need one.
Expected behavior
Either create the spill store on first use, so a session that never spills never touches the temp directory, or keep the eager construction and propagate the failure as an error from the session constructor.
Environment
Windows runners in GitHub Actions, intermittently. Not observed on Linux or macOS.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Read rust/lance/src/session.rs at lines 130 and 150 and rust/lance-io/src/spill.rs around line 233 to trace how Session::new obtains LocalSpillStore. Run the named Windows failure test and verify that sessions which never spill no longer panic on temp-directory creation, or that the failure is returned as an error, with regression coverage for the behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- backend, data-engineering
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 55/100