avro_to_arrow: Support in memory apache_avro Value's
- Lingua principale
- Rust
- Stelle
- 9.3k
- Fork
- 2.4k
- Merge medio
- 3g 11h
- PR unite (30g)
- 360
Descrizione
### Is your feature request related to a problem or challenge?
I have Avro values in memory parsed using the apache_avro crate. I would like to convert these and get a RecordBatch.
### Describe the solution you'd like
Currently AvroArrowArrayReader is private and the avro_to_arrow::Reader only accepts a io reader and calls the apache_avro reader internally. I would have to currently serialize my in memory values and pass the bytes to avro_to_arrow::Reader.
### Describe alternatives you've considered
_No response_
### Additional context
I have a fork with a hacky fix, I can make a draft PR if but the API was a bit weird with the lifetime the apache_avro::Reader required
Guida per i contributori
Apri la guida per i contributori
Direzione di ricerca
Start by reading the private AvroArrowArrayReader and the public avro_to_arrow::Reader, focusing on how the apache_avro::Reader lifetime is currently handled. Define an API that accepts in-memory apache_avro values and produces a RecordBatch without serializing them; done means callers can use parsed values directly.
Scritto dal modello di indicizzazione a partire dal testo della issue.
Valutazione
- Stack tecnologico
- rust
- Ambito
- data-engineering
- Tipo di issue
- Funzionalità
- Difficoltà
- 4/5
- Tempo stimato
- 3-5 giorni
- Stato di attività
- Ferma
- Chiarezza
- Abbastanza chiara
- Idoneità per principianti
- 35/100