Support scanning filesystems without building an index
- Dominant language
- Go
- Stars
- 9.6k
- Forks
- 954
- Avg merge
- 23h 27m
- Merged PRs (30d)
- 48
Description
**What would you like to be added**:
My use case is to scan my host filesystem and get the Linux distro information alongside with the host packages and their versions. Unfortunately, indexing the entire filesystem takes too much time (~7 minutes to call `directorySource.FileResolver` from `directory_source.go`).
I would like to avoid the indexing step and directly read the necessary files required by the `os` cataloger. I noticed there is an `unindexed_directory.go` file in the `fileresolver` package, but that's an internal package and the functions there cannot be used.
Is there a plan to expose the functionality from `unindexed_directory.go` as a public API? Or is there another way to speed up the scanning process / avoid indexing the entire filesystem beforehand?
**Why is this needed**:
This would bring a significant performance improvement to the filesystem scanning, especially when the filesystem is very large and thus building the entire index would take too much time.
**Additional context**:
Contributor guide
Research direction
Start by reading directory_source.go, the directorySource.FileResolver call, and fileresolver/unindexed_directory.go, then trace how the os cataloger obtains Linux distro and package files. Determine whether exposing that functionality or another scan path meets the performance goal, and define done as scanning the requested filesystem data without indexing the entire filesystem.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- operating-systems, tooling
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100