hpc / hpc/mpifileutils

dfind slurm job proccess end too far apart

Open
#653 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C
Stars
200
Forks
85
Avg merge
3d 21h
Merged PRs (30d)
2

Description

When running a modified dfind that printed out extra data (uid, gid, blocks, size) on a whole file system czlustre3 (680 million items), the processes ended far enough apart that slurm killed the remaining processes. One option is to just tell slurm to allow a larger difference in process end times, but I think this issue is already solved for dwalk, such that dwalk can do a whole file system and do a --text printout and the processes all end at approximately the same time.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by comparing dfind's process completion behavior with dwalk's whole-filesystem --text path, then investigate how the differing completion times interact with Slurm's process-end tolerance. Reproduce the issue on a large filesystem and define done as having all processes finish close enough together that Slurm does not kill the remaining work.

Written by the indexing model from the issue text.

Assessment

Tech stack
c, hpc
Domain
distributed-systems, performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.