Error install subworkflow with specific module
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 322
- Forks
- 255
- Avg merge
- 2d 3h
- Merged PRs (30d)
- 5
Description
Description of the bug
I created a customized nf modules where it store common modules and subworkflows. The issue is that when I installed them in my pipeline, nf-core failed to solve the location of the package needed by subworkflows.
1 How to reproduce the issue
I have a module like this but I have a minor change by adding a subfolder below fastp module
fastp
└── trim
├── main.nf
├── meta.yml
└── tests
├── main.nf.test
└── tags.yml
Then add this module on a subworkflow. On your pipeline, install them by
nf-core subworkflows --git-remote <remote_url> install <subworkflow contain fastp>
It will show: ERROR: fastp but does not show anything else on log
2 What happen
After invesitgation, I found that the issue is on this line, where component_name is reconstructed by path after splitting by _. As a result, it will be failed if the name of the module does not match the convention
https://github.com/nf-core/tools/blob/main/nf_core/components/components_utils.py#L165-L166
def get_components_to_install(
subworkflow_dir: str | Path,
) -> tuple[list[dict[str, str]], list[dict[str, str]]]:
"""
Parse the subworkflow main.nf file to retrieve all imported modules and subworkflows.
"""
modules: dict[str, dict[str, str]] = {}
subworkflows: dict[str, dict[str, str]] = {}
with open(Path(subworkflow_dir, "main.nf")) as fh:
for line in fh:
regex = re.compile(
r"include(?: *{ *)([a-zA-Z\_0-9]*)(?: *as *)?(?:[a-zA-Z\_0-9]*)?(?: *})(?: *from *)(?:'|\")(.*)(?:'|\")"
)
match = regex.search(line)
if match and len(match.groups()) == 2:
name, link = match.groups()
if link.startswith("../../../"):
name_split = name.lower().split("_")
component_name = "/".join(name_split)
component_dict: dict[str, str] = {
"name": component_name,
}
modules[component_name] = component_dict
elif link.startswith("../"):
component_name = name.lower()
component_dict = {"name": component_name}
subworkflows[component_name] = component_dict
3. How to fix ?
There are 2 methods:
- Force to name the module by convention. In this case, it should be
FASTP_TRIM. I used nf-core lint for this module but it does not show any error - Modify the way to construct the component name
Command used and terminal output
System information
No response
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in nf_core/components/components_utils.py, especially get_components_to_install and the component-name reconstruction around lines 165-166. Reproduce the issue with the shown nf-core subworkflows install command using a nested fastp/trim module. Done means installation resolves the nested module location instead of reporting only “ERROR: fastp”.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- cli, tooling
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 58/100