Allow task workdir to lie outside workflow.workdir.

New feature

I would like to propose that a process workDir should, in certain circumstances, be allowed to lie outside the workflow.workDir.

Use case

My use case is describe here and here. The current configuration is a workflow with a process NFCORE_RUN ( @mahesh-panchal ) that launches a singularity container and runs a nextflow workflow. It generates workdirs work_1, work_2, ...,. that are currently required to lie under workflow.workDir. In order for cache to resolve, the squashfs images work_1.sqfs, work_2.sqfs,..., must each be mounted at its path of origin. This causes strange observed behavior, such as,

• some code of some tasks accesses mounted image while other code attempts to access the path outside the container. This causes files staged by nxf_stage() that refer to files in mounted images, to be accessed outside the container in some cases, resulting in file-not-found errors.
• In an attempt to hide workflow.workdir from the nested workflow I mounted an empty directory lying outside workflow.wordir at workflow.workdir in the task container. Then I noticed that the main workflow used this mounted directory as the workdir of the workflow.

In order for this project to have any chance of success, the workDirs of the nested workflows must lie outside the workDir of the main workflow.

Suggested implementation

I don't have any suggested solution.

No source file, entry point, or test is named. Start by tracing how workflow.workDir and process workDir are validated and resolved, then reproduce the nested-workflow case with Singularity and squashfs images. Done means a task workDir can be outside the workflow workDir without breaking cache resolution, staging, or nested workflow behavior.