[Feature Request]: Some pyarrow numeric types are not supported in `arrow_type_compatibility`
- Dominant language
- Java
- Stars
- 8.7k
- Forks
- 4.7k
- Avg merge
- 2d 2h
- Merged PRs (30d)
- 205
Description
### What would you like to happen?
Beam schemas do not support unsigned ints or float16s, so we currently refuse to map those in `arrow_type_compatibility`. If these types can't be added to core Beam schemas we might consider adding them as logical types.
### Issue Priority
Priority: 2
### Issue Component
Component: sdk-py-core
Contributor guide
Research direction
Start by locating the `arrow_type_compatibility` entry point and the Beam schema handling it uses. Determine whether unsigned integers and float16 should be represented by core schemas or logical types, then identify the relevant compatibility tests. Done means the supported numeric types map consistently without being refused.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100