nf-core / nf-core/tools

Error install subworkflow with specific module

Open
#4,173 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
Python
Stars
322
Forks
255
Avg merge
2d 3h
Merged PRs (30d)
5

Description

Description of the bug

I created a customized nf modules where it store common modules and subworkflows. The issue is that when I installed them in my pipeline, nf-core failed to solve the location of the package needed by subworkflows.

1 How to reproduce the issue

I have a module like this but I have a minor change by adding a subfolder below fastp module

fastp
└── trim
    ├── main.nf
    ├── meta.yml
    └── tests
        ├── main.nf.test
        └── tags.yml

Then add this module on a subworkflow. On your pipeline, install them by

nf-core subworkflows --git-remote <remote_url> install <subworkflow contain fastp>

It will show: ERROR: fastp but does not show anything else on log

2 What happen

After invesitgation, I found that the issue is on this line, where component_name is reconstructed by path after splitting by _. As a result, it will be failed if the name of the module does not match the convention
https://github.com/nf-core/tools/blob/main/nf_core/components/components_utils.py#L165-L166

def get_components_to_install(
    subworkflow_dir: str | Path,
) -> tuple[list[dict[str, str]], list[dict[str, str]]]:
    """
    Parse the subworkflow main.nf file to retrieve all imported modules and subworkflows.
    """
    modules: dict[str, dict[str, str]] = {}
    subworkflows: dict[str, dict[str, str]] = {}

    with open(Path(subworkflow_dir, "main.nf")) as fh:
        for line in fh:
            regex = re.compile(
                r"include(?: *{ *)([a-zA-Z\_0-9]*)(?: *as *)?(?:[a-zA-Z\_0-9]*)?(?: *})(?: *from *)(?:'|\")(.*)(?:'|\")"
            )
            match = regex.search(line)
            if match and len(match.groups()) == 2:
                name, link = match.groups()
                if link.startswith("../../../"):
                    name_split = name.lower().split("_")
                    component_name = "/".join(name_split)
                    component_dict: dict[str, str] = {
                        "name": component_name,
                    }
                    modules[component_name] = component_dict
                elif link.startswith("../"):
                    component_name = name.lower()
                    component_dict = {"name": component_name}
                    subworkflows[component_name] = component_dict

3. How to fix ?

There are 2 methods:

  • Force to name the module by convention. In this case, it should be FASTP_TRIM. I used nf-core lint for this module but it does not show any error
  • Modify the way to construct the component name
Command used and terminal output

System information

No response

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in nf_core/components/components_utils.py, especially get_components_to_install and the component-name reconstruction around lines 165-166. Reproduce the issue with the shown nf-core subworkflows install command using a nested fastp/trim module. Done means installation resolves the nested module location instead of reporting only “ERROR: fastp”.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
cli, tooling
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
58/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.