NVIDIA / NVIDIA/deepops

Make Slurm epilog cleanup job-scoped

Open
#1,407 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Shell
Stars
1.5k
Forks
362
Avg merge
4d 7h
Merged PRs (30d)
23

Description

Track a focused follow-up for the existing Slurm job-end cleanup behavior.

40-lastuserjob-processes terminates processes by user, 42-lastuserjob-cleanup removes files by user ownership, and 50-lastuserjob-all-enroot-dirs removes configured Enroot paths. Those scopes do not necessarily belong only to the job that just ended.

Acceptance criteria:

  • Limit cleanup to processes and paths owned by the ending job.
  • Preserve unrelated login sessions, concurrent or suspended jobs, and shared paths used on other nodes.
  • Add regression tests for missing/erroring ownership information and overlapping workloads.
  • Preserve useful behavior; do not broadly disable tuning, SSH handling, or cleanup as a shortcut. Any necessary behavior change should be reviewed explicitly.

Related: #1391 and #1404. Historical #1315 concerns PID parsing; this follow-up concerns cleanup scope.

Keep this as a separate backlog item, not a prerequisite for unrelated fixes in the current PR queue.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reading the 40-lastuserjob-processes, 42-lastuserjob-cleanup, and 50-lastuserjob-all-enroot-dirs scripts, then review related issues #1391 and #1404. Add regression coverage for missing or failing ownership data and overlapping workloads, and verify that cleanup affects only the ending job while preserving unrelated sessions, jobs, and shared paths.

Written by the indexing model from the issue text.

Assessment

Tech stack
shell
Domain
infrastructure
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.