nextflow-io / nextflow-io/nf-schema

Support `patternProperties` for data/samplesheet validation

Open
#164 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Groovy
Stars
57
Forks
31
Avg merge
3d 17h
Merged PRs (30d)
1

Description

Hi,

I would like to validate a csv file that has dynamic columns that are generated based on some sample identifier.

Example CSV file (input.csv):

fieldA, sample1, sample2, sample3,
rowA, 1, 2, 3,
rowB, 4, 5, 6

In JSON schema, I cannot use the regular items.properties to describe these columns because they can change between runs.
Instead I am using the items.patternProperties to validate fields based on regex match (csv_schema.json):

{
    "$schema": "https://json-schema.org/draft/2020-12/schema",
    "title": "my title",
    "type": "array",
    "items": {
        "type": "object",
        "patternProperties": {
            "^sample": {
                "type": "integer"
            }
        },
        "properties" : {
            "fieldA" : {
                ...
            }
        }
    }
}

When using the samplesheetToList function I get warnings that these columns are unidentified:

WARN: Found the following unidentified headers in testfile.csv:
	- sample1
	- ...

I think that nf-core should either:

  • treat these fields in the patternProperties as defined
  • or add an option to disable the warning for unidentified headers

Also, I think this issue is somewhat related to: https://github.com/nextflow-io/nf-schema/issues/126

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the samplesheetToList function and trace how it identifies headers against the JSON schema. Check how patternProperties is handled for item objects and compare that with the warning in the example. Done means matching dynamic sample columns are treated as defined, or the requested warning-suppression option is supported with tests for the provided schema.

Written by the indexing model from the issue text.

Assessment

Tech stack
groovy, json
Domain
data
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.