galaxyproject / galaxyproject/gxformat2
Native schema: `steps`, `input_connections`, `post_job_actions`, and `creator` cannot be fully typed
- Dominant language
- Python
- Stars
- 14
- Forks
- 7
- Avg merge
- 2h 49m
- Merged PRs (30d)
- 8
Description
## Context
The `schema/native_v0_1/workflow.yml` schema describes Galaxy's native `.ga` workflow format for documentation generation and basic validation. Several key fields are typed as `Any?` instead of their actual structure due to schema-salad limitations.
## Issue 1: Dict-keyed fields cannot use `mapSubject`
### Affected Fields
**`NativeGalaxyWorkflow.steps`** — dict keyed by string integers (`"0"`, `"1"`, ...) with `NativeStep` values.
**Why:** Schema-salad's `mapSubject: id` requires the subject field to use `jsonldPredicate: "@id"`. But the metaschema globally registers `name` as `@id`, and `NativeStep` must have a `name` field (it's the tool display name, present in all real `.ga` files). Two `@id` fields on one record causes broken Python codegen (`NameError` in `fromDoc`). Removing `name` from the schema causes validation failures since schema-salad rejects unknown fields.
**`NativeStep.input_connections`** — dict keyed by input parameter names (arbitrary strings like `"input1"`, `"library|input_1"`). Values are `NativeInputConnection` objects or arrays of them.
**Why:** Schema-salad has no `Map` type. `mapSubject` requires a field on the value record to serve as the key, but the key (the target input name) is external metadata about the connection, not a field on the connection object itself.
**`NativeStep.post_job_actions`** — dict keyed by compound strings (`"HideDatasetActionout_pairs"`). Values are `NativePostJobAction` objects.
**Why:** Same `Map` limitation. Additionally, the key is a concatenation of two fields (`action_type` + `output_name`), not a single field value.
### Current State
All three value types (`NativeStep`, `NativeInputConnection`, `NativePostJobAction`) are fully defined as records in the schema with proper fields and documentation. They appear in the generated HTML docs as standalone sections under `NativeGalaxyWorkflow`. The parent fields just can't reference them as typed values.
### Impact
- **Docs:** The `steps` field shows as `Any` in the docs instead of linking to `NativeStep`. Users must discover the `NativeStep` section separately.
- **Validation:** Step contents aren't validated — any dict is accepted.
- **Codegen:** Generated Python code types these as `Any`, losing type safety.
### Upstream
Two potential schema-salad improvements that would help:
- Support for `mapSubject` on records with multiple `@id`-predicated fields
- A `Map` type for dicts with arbitrary string keys and typed values
## Issue 2: Union dispatch requires schema-salad type names as discriminator values
### Affected Field
**`NativeGalaxyWorkflow.creator`** — array of schema.org Person/Organization objects, discriminated by `class` field.
Real `.ga` data:
```json
"creator": [
{"class": "Person", "name": "Jane Doe", "identifier": "https://orcid.org/..."},
{"class": "Organization", "name": "Galaxy Project", "url": "https://galaxyproject.org"}
]
```
### Why it can't be typed
Schema-salad union dispatch via `class`/`@type` resolves the discriminator value against the schema's own vocabulary. When we define:
```yaml
- name: NativeCreatorPersonType
type: enum
symbols: [Person]
- name: NativeCreatorPerson
type: record
fields:
- name: class
type: NativeCreatorPersonType
jsonldPredicate: {_id: "@type", _type: "@vocab"}
```
The enum symbol `Person` gets namespaced to `gxnative:NativeCreatorPersonType/Person`. Schema-salad's union loader then expects `"class": "NativeCreatorPerson"` (the schema-salad type name) in the input data, not `"class": "Person"` (the schema.org type name). There is no way to alias a discriminator value to a different type name.
This contrasts with our comment union which works because the discriminator values (`text`, `markdown`, `frame`, `freehand`) happen to be both the domain values AND the enum symbol names — the `type` field matches naturally.
### Current State
`NativeCreatorPerson` and `NativeCreatorOrganization` are fully defined records with all schema.org fields (name, identifier, url, email, image, givenName, familyName, honorificPrefix, honorificSuffix, jobTitle, address, alternateName, telephone, faxNumber). They appear in the generated docs. The `creator` field is `Any?` for validation.
### Impact
Same as Issue 1 — records exist for documentation but can't be referenced for validation or codegen.
### Upstream
Schema-salad would need one of:
- Discriminator value aliasing (map `"Person"` → `NativeCreatorPerson`)
- Support for external vocabularies in `@type` resolution (resolve `"Person"` against schema.org, not the schema's own namespace)
- A way to define types whose names match the domain discriminator values without conflicting with the schema's naming conventions
## Workaround
For all affected fields, the record types exist in the schema for documentation purposes. Field-level docs reference them by name (e.g. "Values are NativeInputConnection objects"). The records appear in generated docs via `docParent` relationships.
Contributor guide
Research direction
Start by reading schema/native_v0_1/workflow.yml and the definitions of NativeStep, NativeInputConnection, NativePostJobAction, NativeCreatorPerson, and NativeCreatorOrganization. Compare the affected Any? fields with the generated documentation, then investigate whether schema-salad supports the required map and discriminator behavior; done means typed validation and code generation work for the real .ga structures.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- documentation, tooling
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 28/100