apache / apache/datafusion

avro_to_arrow: Support in memory apache_avro Value's

Abierto
#7,690 7 comentarios 2 reacciones 0 asignados Ver en GitHub
enhancement
Lenguaje dominante
Rust
Estrellas
9.3k
Forks
2.4k
Merge medio
3 d 11 h
PR fusionados (30 d)
360

Descripción

### Is your feature request related to a problem or challenge?

I have Avro values in memory parsed using the apache_avro crate. I would like to convert these and get a RecordBatch.

### Describe the solution you'd like

Currently AvroArrowArrayReader is private and the avro_to_arrow::Reader only accepts a io reader and calls the apache_avro reader internally. I would have to currently serialize my in memory values and pass the bytes to avro_to_arrow::Reader.

### Describe alternatives you've considered

_No response_

### Additional context

I have a fork with a hacky fix, I can make a draft PR if but the API was a bit weird with the lifetime the apache_avro::Reader required

Guía de contribución

Abrir la guía de contribución

Línea de trabajo

Start by reading the private AvroArrowArrayReader and the public avro_to_arrow::Reader, focusing on how the apache_avro::Reader lifetime is currently handled. Define an API that accepts in-memory apache_avro values and produces a RecordBatch without serializing them; done means callers can use parsed values directly.

Escrito por el modelo de indexación a partir del texto del issue.

Evaluación

Stack tecnológico
rust
Área
data-engineering
Tipo de issue
Nueva funcionalidad
Dificultad
4/5
Tiempo estimado
3-5 días
Estado de actividad
Estancado
Claridad
Bastante claro
Aptitud para principiantes
35/100

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.