Harden variant decoding
- Dominant language
- Java
- Stars
- 3.1k
- Forks
- 1.6k
- Avg merge
- 3d 12h
- Merged PRs (30d)
- 33
Description
### Describe the enhancement requested
org.apache.parquet.variant.Variant is brittle against malformed variants, whereas the arrow variant is very strict and includes an optional deep validate phase
* Harden the java implementation, where it doesn't impose any performance penalty
* add tests which can also generate malformed files to add to the parquet test files
The work in #3481 should include some of this or this should be an immediate followon (preferred)
### Component(s)
parquet-variant
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reading org.apache.parquet.variant.Variant and the parquet-variant component, then compare its malformed-input handling with the Arrow variant and review the work in #3481. Done means Java variant decoding is hardened without unnecessary performance cost, with tests that generate malformed files and add them to the Parquet test files.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- data
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 42/100