snakemake / snakemake/snakemake-executor-plugin-slurm
Installing via bioconda installs incompatible version of slurm-jobstep executor
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 55
- Forks
- 57
- Avg merge
- 40m
- Merged PRs (30d)
- 1
Description
Software Versions
snakemake: 9.21.0
conda: miniforge3/25.3.1
snakemake-executor-plugin-slurm: 2.6.1
slurm-wlm: 22.05.8
Full version info after installing using both conda and pip contained in env_list_<pip/conda>.txt.
Describe the bug
When installing the snakemake-executor-plugin-slurm and snakemake-executor-plugin-slurm-jobstep directly using conda, snakemake-executor-plugin-slurm-jobstep is installed in with version 0.4.0.
This leads to all array jobs crashing, as they try to call --slurm-jobstep-array-execs which was not introduced in that version yet.
When explicitly asking for a later version of snakemake-executor-plugin-slurm-jobstep, conda claims that snakemake-executor-plugin-slurm 2.6.1 requires snakemake-executor-plugin-slurm-jobstep 0.4.0.
When using pip to install snakemake-executor-plugin-slurm and snakemake-executor-plugin-slurm-jobstep, snakemake-executor-plugin-slurm-jobstep is correctly installed with version 0.6.0.
Logs
From logs/slurm_logs/60196711_1.log:
__main__.py: error: unrecognized arguments: --slurm-jobstep-array-execs=...
Minimal example
Create environments snakemake-slurm-debug-<conda/pip> using
conda create -f environment_<conda/pip>.yml
To run the error producing version run
conda activate snakemake-slurm-debug-conda
snakemake --profile ./slurm_profile.yml --config pkgmgr="conda"
which runs an array job consisting of two jobs executing test.py from the rule_run_test and an additional rule which prints the package information.
Additional context
I've added the slurm log files of both the conda installed version (logs/slurm_logs/60196711_1/2.log) and the pip installed version (logs/slurm_logs/60190718_1/2.log).
env_list_<conda/pip>.txt and env_info_<conda/pip>.txt contain the output of the system_information rule, env_list_<conda/pip>.txt specifically the conda list dump.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by creating the environment from environment_<conda/pip>.yml and reproduce the failure with the supplied snakemake command and slurm_profile.yml. Compare the conda and pip package lists with env_list_<conda/pip>.txt, then use logs/slurm_logs/60196711_1.log to verify that the conda-installed array job accepts the required argument and matches the working pip setup.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- devops
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 55/100