pyiron / pyiron/executorlib

[Documentation] Explain calling of external executables in more detail

Open
#733 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

documentation
Dominant language
Python
Stars
77
Forks
7
Avg merge
10h 32m
Merged PRs (30d)
12

Description

It is important to use srun or flux run when calling external executables from Python functions submitted to executorlib. Still unless nested executors are used, these jobs appear as additional tasks in the list of tasks from the queuing system. This should be explained in more detail, for example by extending the following sections:

This becomes even more important when calling GPU applications like nvidia-smi, without flux run or srun these applications have access to all GPUs of a given compute node. This can result in unintended resource sharing and slower performance. In the same way the relation to block_allocation=True should be explained in more detail.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Read the quantum-espresso page and the nested-executors section linked in the issue, focusing on how external executables are launched from submitted Python functions. Update those sections to explain srun/flux run, extra queue tasks, GPU resource access, and the relationship to block_allocation=True. Done means the documented behavior and resource implications are clear to executorlib users.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
documentation, hpc
Issue type
Documentation
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
50/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.