nextflow-io / nextflow-io/nextflow
conda + pip install create tmp files in yml file path
Nobody has claimed this yet.
- Dominant language
- Groovy
- Stars
- 3.5k
- Forks
- 811
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 61
Description
Bug report
Expected behavior and actual behavior
when nextflow run use conda yml file which contains pip, (eg https://www.nextflow.io/docs/latest/conda.html#python-packages-from-pypi). The pip create tmp files (requirements.txt) in worflow dir under config dir ($baseDir/config/condaenv.sobah5ol.requirements.txt).
The problem only happens when pip is used in conda yml
For local pipeline, where only dev team has write access, downstream users of such pipeline without write permission to $baseDir worflow dir, will fail to run pipeline.
using conda directly install rather than pip install solved the error, but I hope to find a systematic fix
name: my-env-2
channels:
- conda-forge
- bioconda
dependencies:
- numpy
Steps to reproduce the problem
use the official example, but make the worflow dir 744 or drwxr--r--, when execute from user account in the same group, will has write permission errors.
(eg https://www.nextflow.io/docs/latest/conda.html#python-packages-from-pypi)
name: my-env-2
channels:
- conda-forge
- bioconda
dependencies:
- pip
- pip:
- numpy
- pandas
- matplotlib
Program output
(Copy and paste here output produced by the failing execution. Please highlight it as a code block. Whenever possible upload the .nextflow.log file.)
Environment
- Nextflow version: [?]
- Java version: [?]
- Operating system: [Linux]
- Bash version: (use the command
$SHELL --version)
Additional context
I think the issue description is sufficient, if more details are needed, please let me know. Thank you.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start at the nextflow run conda environment setup using the official Python-packages-from-PyPI example, and reproduce it with the workflow directory made non-writable. Trace where condaenv.sobah5ol.requirements.txt is created; done means pip-backed conda environments run successfully without writing temporary requirements files under $baseDir.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- groovy, python
- Domain
- devops, tooling
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100