lance-format / lance-format/lance
Clarify purpose of `list.struct`, `large_list.struct` and `fixed_size_list:{elem_type}`
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 7.1k
- Forks
- 852
- Avg merge
- 3d 18h
- Merged PRs (30d)
- 272
Description
Do we need logical list types with child in the type? In Arrow, the type of the child is just in the child field. We should look into if we can deprecate it.
Regardless of whether we can or can't, we should document in detail examples of how we currently store these data types, and how they are mapped to Arrow fields.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by locating the implementations and usages of list.struct, large_list.struct, and fixed_size_list:{elem_type}, then inspect how their child types map to Arrow fields. Done means documenting detailed storage and Arrow-mapping examples and recording whether these logical types can be deprecated.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- data, documentation
- Issue type
- Documentation
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100