OvertureMaps / OvertureMaps/schema
[ENHANCEMENT](packaging) Umbrella offers no route to codegen or pyspark
@sethfitz is already working on this.
Since Sep 11, 2026.
- Dominant language
- Python
- Stars
- 213
- Forks
- 22
- Avg merge
- 3d 3h
- Merged PRs (30d)
- 31
Description
Problem
The overture-schema umbrella depends on the six themes, overture-schema-validation, and overture-schema-cli. It offers no route to overture-schema-codegen or overture-schema-pyspark, which publish to PyPI at the same version and are reachable only by someone who already knows their names.
Documentation generation and PySpark expression generation are invisible from the package people install. pip install overture-schema gives overture-schema validate and overture-schema json-schema; the markdown and pyspark targets live behind a separate console script, overture-codegen, in a distribution the umbrella never mentions. A reader following AUTHORING.md — the tooling works on any Pydantic model — installs the umbrella, runs --help, and finds no way to generate anything.
Blocked on #449
Adding the extras today makes the CLI worse. The three distributions ship three console scripts: overture-schema, overture-codegen, and overture-validate. An umbrella extra that pulls in overture-schema-pyspark puts overture-validate on PATH beside overture-schema validate, where the two names do different things. Pulling in codegen adds a third binary carrying its own list, overlapping overture-schema list-types.
Leaving the packages unreachable from the umbrella keeps that collision out of sight. #449 removes it: one overture-schema entry point, subcommands contributed through entry points by whichever distributions are installed. The extras land after it, as the supported way to get those subcommands.
Proposal
[project.optional-dependencies]
codegen = ["overture-schema-codegen>=2.0.0"]
pyspark = ["overture-schema-pyspark>=2.0.0"]
The default install is unchanged: themes, validation, CLI, no jinja2 and no Spark. Someone validating a GeoParquet file has no use for a template engine, which is why these are extras rather than dependencies.
Extra naming
pyspark, not spark, at this layer. overture-schema-pyspark[spark] gates the PySpark runtime (#659); an umbrella extra gates the distribution. Spelling both spark makes overture-schema[spark] and overture-schema-pyspark[spark] read as one switch when one installs code and the other installs a runtime — a standalone environment wants both, a Glue or EMR job wants only the first. Naming each umbrella extra after the sibling distribution it adds keeps codegen and pyspark parallel.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.