nf-core / nf-core/tools

nf-core pipelines sync overwrites newly generated `.nf-core.yml` content

Open
#3,391 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
Python
Stars
322
Forks
255
Avg merge
2d 3h
Merged PRs (30d)
5

Description

Description of the bug

When we first create a pipeline we add a lint section to .nf-core.yml with all the correct checks ignored for the chosen skipped template sections.

When we do a sync this method runs and if you have updated the skipped sections (added or removed any) then fix_linting is called again and the updated lint section is written to the .nf-core.yml file in the TEMPLATE branch.

However at the end of the sync call we then update the .nf-core.yml based on the CreateConfig object in pipeline_create_obj.config. source code

This overwrites the already written updated whole config with the config from before the new template update and just updates the template subsection of it with the new config for that.

I believe the solution here is to replace the linked source above like so:

Current:

# set force to false to avoid overwriting files in the future
if self.config_yml.template is not None:
    self.config_yml.template = pipeline_create_obj.config
    # Set force true in config to overwrite existing files
    self.config_yml.template.force = False
    # Set outdir as the current directory to avoid local info leaking
    self.config_yml.template.outdir = "."
    # Update nf-core version
    self.config_yml.nf_core_version = nf_core.__version__
    dump_yaml_with_prettier(self.config_yml_path, self.config_yml.model_dump(exclude_none=True))

New:

# Load the updated config
_, updated_config = nf_core.utils.load_tools_config(self.pipeline_dir)
# set force to false to avoid overwriting files in the future
if updated_config.template is not None:
    updated_config.template = pipeline_create_obj.config
    # Set force true in config to overwrite existing files
    updated_config.template.force = False
    # Set outdir as the current directory to avoid local info leaking
    updated_config.template.outdir = "."
    # Update nf-core version
    updated_config.nf_core_version = nf_core.__version__
    dump_yaml_with_prettier(self.config_yml_path, updated_config.model_dump(exclude_none=True))
Command used and terminal output

System information

nf-core/tools v3.1.1

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in nf_core/pipelines/sync.py around lines 286-294 and trace how the sync entry point reloads and writes .nf-core.yml. Compare that flow with fix_linting in nf_core/pipelines/create/create.py, focusing on preservation of newly generated lint settings. Done means sync retains updated lint content while applying the refreshed template configuration.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
cli, tooling
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.