Collect MaxRSS/Elapsed from sacct when using SLURM?
Nobody has claimed this yet.
- Dominant language
- R
- Stars
- 184
- Forks
- 53
- Avg merge
- 7d 2h
- Merged PRs (30d)
- 1
Description
I imagine a wide problem in using HPC systems is not knowing how much memory or walltime to allocate. I imagine this not knowing the right amounts (of memory especially) causes huge amounts of wasted resources on HPC systems. I know it does in my workflow.
SLURM returns MaxRSS/Elapsed value for each job after completion which batchtools could store somewhere at the completion of jobs.
I'm eventually imagining a nice function exported from future.batchtools or drake that could report on these somehow.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
No files, tests, or entry points are named. Start by tracing how batchtools handles completed SLURM jobs and whether sacct data is currently collected; done would require a clearly defined way to retain MaxRSS and Elapsed values without expanding into the proposed future.batchtools or drake reporting functions.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- r
- Domain
- hpc
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100