[Improvement] Define read and write DataType contracts
- Dominant language
- Java
- Stars
- 3.2k
- Forks
- 935
- Avg merge
- 1d 16h
- Merged PRs (30d)
- 298
Description
### What would you like to be improved?
`DataType` is currently reused by read responses and metadata write requests. Its shared deserializer preserves unknown strings and objects as `UnparsedType`. This is required for forward-compatible reads, but write APIs can silently persist unresolved type metadata.
Behavior also differs by surface:
- Function definitions are managed metadata.
- Table writes may pass through catalog-specific converters, while managed tables store types directly.
- Literal and default expressions also carry `DataType`.
This makes it unclear which inputs are structurally valid at the REST boundary and which are supported by a specific catalog.
### How should we improve?
Define three roles for the REST contract:
- Read-compatible `DataType`: native types, `ExternalType`, and `UnparsedType`.
- `WritableDataType`: recursively resolved native types or an explicit, non-blank `ExternalType`; never `UnparsedType`.
- Legacy selector inputs: retain read-compatible types when identifying existing metadata.
Keep wire-level resolvedness separate from catalog-specific capability validation. Accepting `WritableDataType` must not imply that every catalog supports the represented type. `ExternalType` should be the canonical form for intentionally supplied catalog- or engine-specific types; `UnparsedType` should remain a read-compatibility representation.
Roll this out by surface:
1. Function register and `AddDefinition`.
2. Table create and alter, including managed catalogs and connector compatibility.
3. Literal, default, and partition expressions and clients.
Existing metadata must remain readable and manageable, without automatic `UnparsedType` to `ExternalType` conversion.
Contributor guide
Assessment
This issue has not been assessed yet.