apache / apache/parquet-java

Avoid Copy of Bytes in Protobuf BinaryWriter

Abierto
#2,527 1 comentario 0 reacciones 0 asignados Ver en GitHub
Component: Parquet Priority: Minor Type: enhancement
Lenguaje dominante
Java
Estrellas
3.1k
Forks
1.6k
Merge medio
3 d 12 h
PR fusionados (30 d)
33

Descripción

```java

class BinaryWriter extends FieldWriter {
@Override
final void writeRawValue(Object value) {
ByteString byteString = (ByteString) value;
Binary binary = Binary.fromConstantByteArray(byteString.toByteArray());
recordConsumer.addBinary(binary);
}
}
```

`toByteArray()` creates a copy of the buffer. There is already support with Parquet and Protobuf to pass instead a ByteBuffer which avoids the copy.

**Reporter**: [David Mollitor](https://issues.apache.org/jira/secure/ViewProfile.jspa?name=belugabehr) / @belugabehr
**Assignee**: [David Mollitor](https://issues.apache.org/jira/secure/ViewProfile.jspa?name=belugabehr) / @belugabehr
#### Related issues:
- [Move Support for ByteBuffer into TTransport](https://issues.apache.org/jira/browse/THRIFT-5288) (is blocked by)

**Note**: *This issue was originally created as [PARQUET-1918](https://issues.apache.org/jira/browse/PARQUET-1918). Please see the [migration documentation](https://issues.apache.org/jira/browse/PARQUET-2502) for further details.*

Guía de contribución

No hay ninguna guía de contribución indexada para este repositorio

Línea de trabajo

Start at BinaryWriter.writeRawValue, where ByteString.toByteArray() is used, and review the existing Parquet and Protobuf ByteBuffer support. Check the related THRIFT-5288 issue because the requested support is blocked there. Done means the binary value can be passed without copying the ByteString buffer.

Escrito por el modelo de indexación a partir del texto del issue.

Evaluación

Stack tecnológico
java
Área
backend
Tipo de issue
Refactorización
Dificultad
4/5
Tiempo estimado
3-5 días
Estado de actividad
Estancado
Claridad
Bastante claro
Aptitud para principiantes
35/100

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.