Type variation in substrait: Hardcoding the numbers isn't really the proper way to do type variations.
- Dominant language
- Rust
- Stars
- 9.3k
- Forks
- 2.4k
- Avg merge
- 3d 7h
- Merged PRs (30d)
- 344
Description
> FWIW, hardcoding the numbers isn't really the proper way to do type variations. (Rather we should add the variation as an extension and refer to the extension's id.) However, given this is already used for default vs large, I guess adding view makes sense - and they can all be migrated at once to the proper way someday.
_Originally posted by @Blizzara in https://github.com/apache/datafusion/pull/12199#discussion_r1734657314_
The idea here is for someone to research how this is supposed to work and then change the datafusion substrait implementation to do it
Contributor guide
Research direction
Start by researching how Substrait type variations are represented as extensions, then inspect DataFusion's Substrait implementation for the existing default, large, and view handling. Done means the implementation uses extension IDs rather than hardcoded variation numbers, with all three existing variations accounted for.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- data-engineering
- Issue type
- Refactor
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100