nf-core pipelines sync overwrites newly generated `.nf-core.yml` content
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 322
- Forks
- 255
- Avg merge
- 2d 3h
- Merged PRs (30d)
- 5
Description
Description of the bug
When we first create a pipeline we add a lint section to .nf-core.yml with all the correct checks ignored for the chosen skipped template sections.
When we do a sync this method runs and if you have updated the skipped sections (added or removed any) then fix_linting is called again and the updated lint section is written to the .nf-core.yml file in the TEMPLATE branch.
However at the end of the sync call we then update the .nf-core.yml based on the CreateConfig object in pipeline_create_obj.config. source code
This overwrites the already written updated whole config with the config from before the new template update and just updates the template subsection of it with the new config for that.
I believe the solution here is to replace the linked source above like so:
Current:
# set force to false to avoid overwriting files in the future
if self.config_yml.template is not None:
self.config_yml.template = pipeline_create_obj.config
# Set force true in config to overwrite existing files
self.config_yml.template.force = False
# Set outdir as the current directory to avoid local info leaking
self.config_yml.template.outdir = "."
# Update nf-core version
self.config_yml.nf_core_version = nf_core.__version__
dump_yaml_with_prettier(self.config_yml_path, self.config_yml.model_dump(exclude_none=True))
New:
# Load the updated config
_, updated_config = nf_core.utils.load_tools_config(self.pipeline_dir)
# set force to false to avoid overwriting files in the future
if updated_config.template is not None:
updated_config.template = pipeline_create_obj.config
# Set force true in config to overwrite existing files
updated_config.template.force = False
# Set outdir as the current directory to avoid local info leaking
updated_config.template.outdir = "."
# Update nf-core version
updated_config.nf_core_version = nf_core.__version__
dump_yaml_with_prettier(self.config_yml_path, updated_config.model_dump(exclude_none=True))
Command used and terminal output
System information
nf-core/tools v3.1.1
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in nf_core/pipelines/sync.py around lines 286-294 and trace how the sync entry point reloads and writes .nf-core.yml. Compare that flow with fix_linting in nf_core/pipelines/create/create.py, focusing on preservation of newly generated lint settings. Done means sync retains updated lint content while applying the refreshed template configuration.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- cli, tooling
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Clearly specified
- Newbie friendliness
- 55/100