[parquet_derive] Add inner struct and enum support for writing and reading
- Dominant language
- Rust
- Stars
- 3.6k
- Forks
- 1.3k
- Avg merge
- 2d 18h
- Merged PRs (30d)
- 169
Description
**Is your feature request related to a problem or challenge? Please describe what you are trying to do.**
Basically I need to be able to write large vectors of data to/from parquet files. And my record struct type has an enum in it with fields. I would like to be able to do this by just adding `#[derive(ParquetRecordWriter, ParquetRecordReader)]` into my types. This should also support structs within structs, or enums within structs, etc.
It would also be nice if I didn't have to hardcode the number of records to be read. It should be able to know when to stop.
**Additional context**
My development journal can be found [here]() for my problem context.
Contributor guide
Research direction
Start by reading the parquet_derive implementations behind ParquetRecordWriter and ParquetRecordReader. Determine how nested structs and enums, including enums with fields, should map to Parquet data and how reading can detect the end without a hardcoded record count. Done means these derive macros support the requested nested types and automatic stopping behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- data-engineering
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100