avro_to_arrow: Support in memory apache_avro Value's
- Lenguaje dominante
- Rust
- Estrellas
- 9.3k
- Forks
- 2.4k
- Merge medio
- 3 d 11 h
- PR fusionados (30 d)
- 360
Descripción
### Is your feature request related to a problem or challenge?
I have Avro values in memory parsed using the apache_avro crate. I would like to convert these and get a RecordBatch.
### Describe the solution you'd like
Currently AvroArrowArrayReader is private and the avro_to_arrow::Reader only accepts a io reader and calls the apache_avro reader internally. I would have to currently serialize my in memory values and pass the bytes to avro_to_arrow::Reader.
### Describe alternatives you've considered
_No response_
### Additional context
I have a fork with a hacky fix, I can make a draft PR if but the API was a bit weird with the lifetime the apache_avro::Reader required
Guía de contribución
Línea de trabajo
Start by reading the private AvroArrowArrayReader and the public avro_to_arrow::Reader, focusing on how the apache_avro::Reader lifetime is currently handled. Define an API that accepts in-memory apache_avro values and produces a RecordBatch without serializing them; done means callers can use parsed values directly.
Escrito por el modelo de indexación a partir del texto del issue.
Evaluación
- Stack tecnológico
- rust
- Área
- data-engineering
- Tipo de issue
- Nueva funcionalidad
- Dificultad
- 4/5
- Tiempo estimado
- 3-5 días
- Estado de actividad
- Estancado
- Claridad
- Bastante claro
- Aptitud para principiantes
- 35/100