Multithreaded file scan
Open
Nobody has claimed this yet.
feature request
- Dominant language
- C
- Stars
- 14.1k
- Forks
- 2.2k
- Avg merge
- 7h 35m
- Merged PRs (30d)
- 51
Description
Description
- When I scan files using the "Scan Directory" feature, retroarch doesn't seem to do any parallelization or efficiently use the CPU. This should be a highly parallelizable task, Im not sure why is implemented linearly. I'm a developer but I have zero experience with the retroarch code, if someone could point me out to where I have to look to implement this feature I will gladly take a look myself and try to implement it.
Expected behavior
Scan Directory should be parallelizable.
Actual behavior
Very low CPU usage, making the Scan Directory extremely slow.
Steps to reproduce the bug
- Scan a directory with thousands of small files (ideally more than 10k).
Bisect Results
[Try to bisect and tell us when this started happening]
Version/Commit
You can find this information under Information/System Information
- RetroArch: 1.10.2
Environment information
- OS: Windows 10
- Compiler: [In case you are running local builds]
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by locating RetroArch's implementation of the Scan Directory feature and reproduce the low-CPU scan with a directory containing thousands of small files on Windows 10. Measure the current behavior before investigating a parallel design. Done means directory scanning uses parallel work safely and the large-directory scan completes faster without changing its results.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- c
- Domain
- desktop, performance
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100