OvertureMaps / OvertureMaps/schema
Code generation mishandles set, variable-length tuple, and Sequence collection types
Open
@sethfitz is already working on this.
Since Jul 22, 2026.
- Dominant language
- Python
- Stars
- 213
- Forks
- 22
- Avg merge
- 3d 3h
- Merged PRs (30d)
- 31
Description
Reported by @vcschapp during review of #518.
The code generator "supports" several collection types that Pydantic accepts, but the generated output is wrong for both the Markdown and PySpark targets. No Overture model uses these types today, so this does not block current work -- filing for later.
Affected annotations:
set, e.g.set[str]- variable-length tuples (list-equivalent), e.g.
tuple[str, ...] Sequence, e.g.Sequence[str](Pydantic accepts it)
A bonus case: fixed-length tuples whose element Spark types are all the same base type, e.g. tuple[str, str] or tuple[str, SnakeCaseString].
Current behavior
| Annotation | FieldShape | Spark schema type | Markdown | Correction |
|---|---|---|---|---|
list[str] |
ArrayOf(Primitive('str')) |
ArrayType(StringType()) (correct) |
list<str> (correct) |
N/A |
set[str] |
Primitive('set') |
StringType() (wrong) |
set (wrong) |
Spark array, MD set<str>? |
tuple[str, ...] |
Primitive('tuple') |
StringType() (wrong) |
tuple (wrong) |
Spark array, MD list<str>? |
tuple[str, str, str] |
Primitive('tuple') |
StringType() (wrong) |
tuple (wrong) |
Spark array, MD ??? |
tuple[str, int] |
Primitive('tuple') |
StringType() (wrong) |
tuple (wrong) |
fail codegen |
Sequence[str] |
raises TypeError |
aborts run (wrong) | aborts run (wrong) | Spark array, MD list<str>? |
This is a general code-generation issue (FieldShape extraction), not specific to the PySpark target.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.