bigbio / bigbio/quantmsdiann

Multi-run SDRF?

Closed
#86 5 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Nextflow
Stars
19
Forks
35
Avg merge
1d 14h
Merged PRs (30d)
5

Description

### Description of feature

Could we from a single `nextflow run --input example.sdrf.tsv`, spawn different `quantms` instances?

For example:
I collected 3 different kinds of cell types from the same tissue. I prefer searching for these tissues using a biology-specific library. Therefore I would like to create 3 instances of `quantmsdiann`, one for each celltype. This entails, 3x scripts, 3x sdrf tables, 3x params.yml, etc.
What if from a single SDRF, we could have 2 additional columns: `session_id` where it describes to which pipeline run instance that row should be separated to; and `param_path` where one can designate which parameters should apply for that row.

**Example:**

| sample_id | celltype | session_id | param_path |
| ---------- | ---------- | ---------- | ---------- |
| sample_1 | Cancer_cells | session_1 | Cancer_params.yml |
| sample_2 | Cancer_cells | session_1 | Cancer_params.yml |
| sample_3 | T-cells | session_2 | T_cell_params.yml |
| sample_4 | T-cells | session_2 | T_cell_params.yml |

This would create 2 different instances of the pipeline.

Besides this biological vignette, which would be great to have, at least for our lab. I think this also has applications for technical experiments.
I could imagine running the same samples with two different sets of parameters, or with two different libraries.

At the moment I am using `job arrays` from SLURM, and while they work, it is clunkier, and requires plenty of file staging.
Having a single SDRF file could abstract entire experimental setups into a single table.

Thoughts?

Contributor guide

Open the contributing guide

Research direction

No files, tests, or entry points are named in the issue. Start by locating the workflow's SDRF input handling and define how session_id and param_path would create separate runs; done means one SDRF can reproduce the requested grouped instances and parameter sets.

Written by the indexing model from the issue text.

Assessment

Domain
data-engineering, devops
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.