Allow the use of SLURM clusters which forbid --mem and --mem-per-cpu options

New feature

Allow the use of SLURM clusters which forbid --mem and --mem-per-cpu options.

Use case

We have been unable to use nextflow of some national supercomputers from tier-0 which do not allow the use of the --mem and --mem-per-cpu options. This SLURM configuration is made on purpose to make easier the accounting of the CPU time which have been used the users. As this is the policy of these supercomputer, we have to deal with this setting that they will not change.

As a consequence, any nextflow process which sets the memory directive just fails since the --mem option is added in the .command.runfile. Therefore, there is no way to run a nextflow pipeline unless we rewrite the it without the memory directive, which is not a realistic solution

Suggested implementation

We introduced the environment variable NXF_EXECUTOR_MEM_GIGA_PER_CORE which sets how many memory in GB is available per core on the supercomputer. For example, if this variable is set to 4, any nextflow process requiring 40 GB of memory will make sure that the job requests a least 10 cores. The file modules/nextflow/src/main/groovy/nextflow/executor/SlurmExecutor.groovy is modified to consider thi variable if defined.

Start in modules/nextflow/src/main/groovy/nextflow/executor/SlurmExecutor.groovy and inspect how the memory directive becomes --mem or --mem-per-cpu. Trace the proposed NXF_EXECUTOR_MEM_GIGA_PER_CORE environment variable and verify that a 40 GB request with a value of 4 requests at least 10 cores without forbidden memory options. Done means SLURM processes using memory can run under this cluster policy.