galaxyproject / galaxyproject/gxformat2

Native schema: `steps`, `input_connections`, `post_job_actions`, and `creator` cannot be fully typed

Open
#151 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
14
Forks
7
Avg merge
2h 49m
Merged PRs (30d)
8

Description

## Context

The `schema/native_v0_1/workflow.yml` schema describes Galaxy's native `.ga` workflow format for documentation generation and basic validation. Several key fields are typed as `Any?` instead of their actual structure due to schema-salad limitations.

## Issue 1: Dict-keyed fields cannot use `mapSubject`

### Affected Fields

**`NativeGalaxyWorkflow.steps`** — dict keyed by string integers (`"0"`, `"1"`, ...) with `NativeStep` values.

**Why:** Schema-salad's `mapSubject: id` requires the subject field to use `jsonldPredicate: "@id"`. But the metaschema globally registers `name` as `@id`, and `NativeStep` must have a `name` field (it's the tool display name, present in all real `.ga` files). Two `@id` fields on one record causes broken Python codegen (`NameError` in `fromDoc`). Removing `name` from the schema causes validation failures since schema-salad rejects unknown fields.

**`NativeStep.input_connections`** — dict keyed by input parameter names (arbitrary strings like `"input1"`, `"library|input_1"`). Values are `NativeInputConnection` objects or arrays of them.

**Why:** Schema-salad has no `Map` type. `mapSubject` requires a field on the value record to serve as the key, but the key (the target input name) is external metadata about the connection, not a field on the connection object itself.

**`NativeStep.post_job_actions`** — dict keyed by compound strings (`"HideDatasetActionout_pairs"`). Values are `NativePostJobAction` objects.

**Why:** Same `Map` limitation. Additionally, the key is a concatenation of two fields (`action_type` + `output_name`), not a single field value.

### Current State

All three value types (`NativeStep`, `NativeInputConnection`, `NativePostJobAction`) are fully defined as records in the schema with proper fields and documentation. They appear in the generated HTML docs as standalone sections under `NativeGalaxyWorkflow`. The parent fields just can't reference them as typed values.

### Impact

- **Docs:** The `steps` field shows as `Any` in the docs instead of linking to `NativeStep`. Users must discover the `NativeStep` section separately.
- **Validation:** Step contents aren't validated — any dict is accepted.
- **Codegen:** Generated Python code types these as `Any`, losing type safety.

### Upstream

Two potential schema-salad improvements that would help:
- Support for `mapSubject` on records with multiple `@id`-predicated fields
- A `Map` type for dicts with arbitrary string keys and typed values

## Issue 2: Union dispatch requires schema-salad type names as discriminator values

### Affected Field

**`NativeGalaxyWorkflow.creator`** — array of schema.org Person/Organization objects, discriminated by `class` field.

Real `.ga` data:
```json
"creator": [
{"class": "Person", "name": "Jane Doe", "identifier": "https://orcid.org/..."},
{"class": "Organization", "name": "Galaxy Project", "url": "https://galaxyproject.org"}
]
```

### Why it can't be typed

Schema-salad union dispatch via `class`/`@type` resolves the discriminator value against the schema's own vocabulary. When we define:

```yaml
- name: NativeCreatorPersonType
type: enum
symbols: [Person]

- name: NativeCreatorPerson
type: record
fields:
- name: class
type: NativeCreatorPersonType
jsonldPredicate: {_id: "@type", _type: "@vocab"}
```

The enum symbol `Person` gets namespaced to `gxnative:NativeCreatorPersonType/Person`. Schema-salad's union loader then expects `"class": "NativeCreatorPerson"` (the schema-salad type name) in the input data, not `"class": "Person"` (the schema.org type name). There is no way to alias a discriminator value to a different type name.

This contrasts with our comment union which works because the discriminator values (`text`, `markdown`, `frame`, `freehand`) happen to be both the domain values AND the enum symbol names — the `type` field matches naturally.

### Current State

`NativeCreatorPerson` and `NativeCreatorOrganization` are fully defined records with all schema.org fields (name, identifier, url, email, image, givenName, familyName, honorificPrefix, honorificSuffix, jobTitle, address, alternateName, telephone, faxNumber). They appear in the generated docs. The `creator` field is `Any?` for validation.

### Impact

Same as Issue 1 — records exist for documentation but can't be referenced for validation or codegen.

### Upstream

Schema-salad would need one of:
- Discriminator value aliasing (map `"Person"` → `NativeCreatorPerson`)
- Support for external vocabularies in `@type` resolution (resolve `"Person"` against schema.org, not the schema's own namespace)
- A way to define types whose names match the domain discriminator values without conflicting with the schema's naming conventions

## Workaround

For all affected fields, the record types exist in the schema for documentation purposes. Field-level docs reference them by name (e.g. "Values are NativeInputConnection objects"). The records appear in generated docs via `docParent` relationships.

Contributor guide

Open the contributing guide

Research direction

Start by reading schema/native_v0_1/workflow.yml and the definitions of NativeStep, NativeInputConnection, NativePostJobAction, NativeCreatorPerson, and NativeCreatorOrganization. Compare the affected Any? fields with the generated documentation, then investigate whether schema-salad supports the required map and discriminator behavior; done means typed validation and code generation work for the real .ga structures.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
documentation, tooling
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
28/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.