nextflow-io / nextflow-io/nextflow
Feature request: pubish per-channel entry index files
Nobody has claimed this yet.
- Dominant language
- Groovy
- Stars
- 3.5k
- Forks
- 811
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 61
Description
New feature
Currently, when publishing an index with the new output block, the output index is an index across all the channel entries published. However, especially with the new Record types coming, it would be nice to be able to publish a per-channel index, as the record/map being published may have its own complicated metadata structure that is worth being recorded at a sample level.
Use case
Say you are publishing a record/map that agglomerates a bunch of information about a sample, and you want to publish the sample files to a specific sample directory in the output. One might also want to publish a sample-level index file in that captures sample-level metadata and files together in that directory.
Suggested implementation
Maybe this would need an extension to add an entryIndex or similar directive to a publishing block:
output {
sample {
path { s ->
s.fastq >> "${s.id}/fastq/"
s.bam >> "${s.id}/bam/{s.sorted ? "sorted" : "not_sorted"}.bam"
}
entryIndex { s ->
path "${s.id}/index.json"
}
}
}
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by locating the existing output-block index publishing path and its tests; the issue does not name files or entry points. Define and validate per-channel entryIndex behavior for sample metadata and files, including the proposed path syntax, with tests covering the published index contents.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- groovy
- Domain
- tooling
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100