Parallel, distributed file interaction using mpifileutils
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 24
- Forks
- 15
- Avg merge
- 10d 3h
- Merged PRs (30d)
- 1
Description
What would you like to see added?
For interacting with a large number of files, standard tools aren't performant and can cause GPFS overload depending on how they are run. Distributing the work across multiple nodes both vastly increases performance and reduces stress on GPFS. mpifileutils is great because it can be run by the users themselves as opposed to needing admins to submit an MMLS policy to perform the action.
We should add a page going over some of the benefits of mpifileutils with examples on some of the basic use cases with scripts users can copy-paste to run themselves
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reviewing the existing MkDocs documentation structure and the basic mpifileutils workflows described in the issue. Done means a page explains the benefits of distributed file interaction and provides copy-paste scripts for several basic use cases.
Written by the indexing model from the issue text.
Assessment
- Domain
- documentation, hpc
- Issue type
- Documentation
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100