nextflow-io / nextflow-io/nextflow
Allow a list of keys in the `by` option of the v2 version of the `join` operator
Nobody has claimed this yet.
- Dominant language
- Groovy
- Stars
- 3.5k
- Forks
- 811
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 61
Description
New feature
Currently the by option in the .join operator for typed workflows allows only a single key to be specified. This on it's own is a very useful feature and will greatly improve how we write pipelines. It would however also be helpful if this option could take a list of multiple keys and only join entries where all of these keys match. This would prevent pipeline developers from having to do a .map on both channels before joining them. It would also increase the readability of the pipeline as it's clear from the join operator what keys are used exactly (instead of one custom key created just for joining)
Use case
Joining channels with records that have multiple keys in common
Suggested implementation
ch1.join(ch2, by['id', 'type']) or similar
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start at the v2 typed-workflow .join operator and trace how its current single-key by option is parsed and matched. Review the existing join tests for one key, then add coverage showing that a list such as ['id', 'type'] joins entries only when every key matches; done means the multi-key form works without requiring preceding .map operations.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- groovy
- Domain
- data-engineering
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100