apache / apache/lucene

Add more checks to DirectIODirectory? [LUCENE-9788]

Open
#10,827 0 comments 0 reactions 0 assignees View on GitHub
legacy-jira-priority:Major type:enhancement
Dominant language
Java
Stars
3.6k
Forks
1.4k
Avg merge
2d 11h
Merged PRs (30d)
88

Description

Followup to #10813 (which just added an assumption to the tests so they will pass on unsupported filesystems).

If you try to use DirectIODirectory on an unsupported FS like tmpfs (I am sure there are others), you'll get some nasty behavior: EINVAL from the kernel (translates to a FilesystemException), leftover files on the filesystem, etc. See LUCENE-9774's PR for some more explanation.

Maybe we could try to make the behavior friendlier/clearer in some way. It will behave like a bug today.

Some options might be:
- improve the javadocs / make javadocs stand out better.
- catch the evil EINVAL from this specific place and add a suppressed "explanation" that helps (I think we've done similar stuff before with the confusing OutOfMemoryError from FileChannel.map)
- probe that the thing can work against the filesystem in DirectIODirectory code itself somehow? This seems a bit scary as it means we have to write files, try to delete the leaked files when it fails, avoid masking other actual errors, etc.
- others...

---
Migrated from [LUCENE-9788](https://issues.apache.org/jira/browse/LUCENE-9788) by Robert Muir (@rmuir)

Contributor guide

Open the contributing guide

Research direction

Start with DirectIODirectory and the changes from #10813, then read LUCENE-9774's PR for the reported EINVAL and leftover-file behavior. The issue needs a chosen approach before implementation; done should make unsupported filesystems fail with clearer behavior without masking other errors or leaving files behind.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
operating-systems
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
28/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.