nextflow-io / nextflow-io/nextflow

If memory is specified, tasks will not run on the UGE engine

Open
#2,449 15 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

executor/uge stale
Dominant language
Groovy
Stars
3.5k
Forks
811
Avg merge
2d 11h
Merged PRs (30d)
61

Description

Bug report

Parameters in config h_rt/h_rss/mem_free should not be specified by Nextflow.
This is because some supercomputers are unable to accept these parameters. I am not sure if this is common or not.
However, at least on NIG supercomputer system( https://sc.ddbj.nig.ac.jp/ ), ques never run.
We need to use h_vmem instead of those parameters.

https://nfcore.slack.com/archives/CGRTMASKY/p1629489188025100

I solved the problem by this conversation's methods.

In config file
memory = null

Is there any way to replace this, for example by creating a new option?
Null sometimes causes errors in downstream workflows.

Expected behavior and actual behavior

Ques run immediately.

Ques never run.

Steps to reproduce the problem

Not works.

  • memory = '16G'
  • memory was not specified
    Works
  • memory = null
Environment
  • Nextflow version: 21.04.3
  • Java version: 11
  • Operating system: CentOS7.5

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by tracing the UGE engine's handling of the h_rt, h_rss, mem_free, h_vmem, and memory configuration mentioned in the report. Reproduce the difference between memory='16G', an omitted memory setting, and memory=null on the stated environment, then verify that a supported configuration lets queued tasks run without introducing downstream null errors.

Written by the indexing model from the issue text.

Assessment

Tech stack
groovy
Domain
hpc
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.