galaxyproject / galaxyproject/foundry

Galaxy sample-sheet column-value charset gate rejects nf-core canonical values

Open
#191 0 comments 0 reactions 0 assignees View on GitHub
priority/v2 roadmap/off upstream/galaxy
Dominant language
TypeScript
Stars
2
Forks
3
Avg merge
1d 22m
Merged PRs (30d)
63

Description

Surfaced by gap analysis in [content/research/nf-schema-samplesheet-galaxy-gaps.md](https://github.com/jmchilton/foundry/blob/main/content/research/nf-schema-samplesheet-galaxy-gaps.md) (PR #190), work item W1.

## Problem

Galaxy's sample-sheet collection validator gates column **values** with `[\w\-_ ?]*` via `has_special_characters` in [`lib/galaxy/model/dataset_collections/types/sample_sheet_util.py:116-119`](https://github.com/galaxyproject/galaxy/blob/dev/lib/galaxy/model/dataset_collections/types/sample_sheet_util.py#L116-L119). The gate rejects values containing `;`, `.`, `:`, `,`, `/`, `=`, `'`, `"`, etc. — many of which are routine in nf-core sample-sheet enums and free-text columns.

**Concrete corpus blocker.** nf-core/taxprofiler `assets/schema_database.json` declares `db_type` with `enum: [short, long, short;long]` and `default: "short;long"`. The literal value `short;long` contains `;` and is silently rejected at row submission. A canonical nf-core enum value cannot be stored in a Galaxy `column_definition` cell.

Also affected: taxprofiler `db_params` (CLI flag strings — spaces, `.`, `=`), and any nf-core column whose values are paths, version strings, or configurations.

## Proposed Galaxy-side fix

Distinguish three charset gates in `sample_sheet_util.py`:

1. Column **name** — current strict gate. Keep (serializes into TSV header).
2. `element_identifier` value — current strict gate. Keep (must serialize cleanly into TSV).
3. Arbitrary cell value — relax to "no control characters, no embedded newline/tab; CSV-escapable." Use `strip_control_characters` plus a CSV-escapability check, not the current word-boundary regex.

Implementation sketch:

- Split `validate_no_special_characters` into `validate_identifier_charset` (current strict) and `validate_value_charset` (relaxed).
- `validate_column_value` (`sample_sheet_util.py:155-162`) calls the strict gate only when the column type is `element_identifier`.
- Verify `lib/galaxy/tools/sample_sheet_to_tabular.xml` escapes `\t`, `\n` correctly in the relaxed regime — introduce CSV mode or a quote rule if needed.

Risk: Medium. Touches collection-build-time validation and downstream TSV writers. Size: S–M. No gxformat2 schema rev needed.

Existing Galaxy issues: #20831 (Sample Sheets follow-up tracking) is the umbrella. No PR found that addresses the value-charset gate specifically.

## Why this matters for the Foundry

Without this fix, `nextflow-summary-to-galaxy-interface` casts must rewrite nf-core enum values (e.g. `short;long` → `short_long`) to land them in Galaxy `column_definitions`. That introduces a CLI-mismatch loss recorded as `loss_class: galaxy_value_charset_overrestrictive` — see the cast-Mold loss vocabulary in the gap-analysis note. Resolving this gate upstream removes the most common blocking-severity loss class.

## Next steps

- File a corresponding upstream Galaxy issue once this is triaged here.
- Update [[nf-schema-samplesheet-galaxy-gaps]] when the gate is loosened (downgrade `galaxy_value_charset_overrestrictive` from `blocking` to `none`).

Related: #178, #185, #186, PR #190.

Contributor guide

No contributing guide indexed for this repository

Research direction

Read lib/galaxy/model/dataset_collections/types/sample_sheet_util.py, especially validate_no_special_characters and validate_column_value, then inspect lib/galaxy/tools/sample_sheet_to_tabular.xml. Trace how relaxed cell values are serialized, using short;long and the described db_params values as examples. Done means arbitrary CSV-escapable values are accepted while column names and element_identifier values retain strict validation.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend, data
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.