nextflow-io / nextflow-io/nextflow

Generate output schema from output definition

Open
#5,213 1 comment 2 reactions 1 assignee View on GitHub

@bentsherman is already working on this.

Since Apr 28, 2025.

Dominant language
Groovy
Stars
3.5k
Forks
811
Avg merge
2d 11h
Merged PRs (30d)
61

Description

Spun off from #5103

Nextflow should be able to generate an output schema from the workflow output definition, essentially as a list of index file schemas. See https://github.com/nf-core/fetchngs/pull/312 for a concrete example (schema_outputs.yml).

For now, the index file should specify a schema file (using fetchngs as example):

output {
  'fastq' {
    index {
      path 'samplesheet.csv'
      schema 'assets/schema_samplesheet.yml'
    }
  }
}

This output definition would produce the following output schema:

$schema: 'http://json-schema.org/draft-07/schema'
$id: 'https://raw.githubusercontent.com/nf-core/fetchngs/master/schema_outputs.yml'
title: 'nf-core/fetchngs pipeline outputs'
description: ''
type: object
properties:
  samplesheet:
    type: string
    format: file-path
    mimetype: text/csv
    schema: assets/schema_samplesheet.yml

Extra notes:

  • Schema file should be JSON or YAML
  • Can be extended in the future to define index schema in the DSL directly instead of referencing an external schema file
  • See #4042 for discussion of other relevant metadata, which can be defined in the output target DSL
  • Should be generated on each run or via some separate command.
  • Should eventually be used with parameter schema for chaining pipelines.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.