apache / apache/arrow-nanoarrow
Nested decoders do not propagate some types of required parameters into dictionary sub-decoders
- Dominant language
- C
- Stars
- 249
- Forks
- 68
- Avg merge
- 2d 21h
- Merged PRs (30d)
- 11
Description
Found from a codex review of https://github.com/apache/arrow-nanoarrow/pull/935:
- `ArrowIpcSetDecompressor()` won't propagate into dictionary decoders. We might need to add the ability for decompressors to clone themselves for this to work (or allow a reference to the parent or parent decompressor)
- Dictionary sub-decoders don't inherit the metadata version read by the parent decoder. All one tests files that have ever been written with a dictionary encoded union pre 1.0 will be read incorrectly (but it's a good point worth handling).
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by tracing ArrowIpcSetDecompressor() into dictionary sub-decoders and locating where the parent decoder reads metadata version information. Determine how dictionary decoders should receive decompressor and metadata-version state, then verify that decompressed data and pre-1.0 dictionary-encoded union files are read correctly.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- c
- Domain
- data-engineering
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Needs clarification
- Newbie friendliness
- 38/100