FasterXML / FasterXML/jackson-dataformats-binary

Add Apache Parquet format backend (`jackson-dataformat-parquet`)

Abierto
#766 0 comentarios 0 reacciones 0 asignados Ver en GitHub
Lenguaje dominante
Java
Estrellas
347
Forks
156
Merge medio
3 d 3 h
PR fusionados (30 d)
22

Descripción

Apache Parquet has become the de-facto interchange format for analytical/columnar data (Spark, Trino, Athena, DuckDB, Arrow, Iceberg/Delta table formats all read and write it).

Today, JVM applications that want to produce or consume Parquet go through the parquet-java libraries directly (`org.apache.parquet:parquet-hadoop` and friends), which means hand-writing `WriteSupport`/`ReadSupport` plumbing or converting through another object model (`parquet-avro`, `parquet-protobuf`). Point solutions exist for narrow slices (see prior art below), but there is no first-class Jackson backend: no `JsonFactory`-level parser/generator pair giving bidirectional streaming, databind (POJOs, `Map`s) and tree-model (`JsonNode`) access under the `ObjectMapper` programming model applications already use for JSON, CBOR, Avro, etc.

Additionally, we've internally observed that adding Hadoop as a dependency ends up blowing the dependencies' size, forcing a lot of teams to drop AWS Lambdas (or using containerised lambdas)

A `jackson-dataformat-parquet` backend would close that gap: the same `ObjectMapper`
programming model, applied to the dominant analytics file format.

## Prior art

* https://github.com/getyourguide/parquet-json — Parquet `WriteSupport` driven by Jackson `JsonNode`:
overlaps the write path proposed here (no read path, and no `JsonFactory`-level
parser/generator or databind integration)
* https://github.com/jerolba/parquet-carpet — Java-records read/write binding over parquet-java,
demonstrating demand for a friendlier binding layer (records only; not a general
Jackson backend)
* `parquet-avro` in parquet-java itself — the "bind through another object model"
pattern this module would generalize to Jackson's
* Searched this tracker, `jackson-future-ideas` and the org: no existing
Parquet request (only a passing mention in [#9](https://github.com/FasterXML/jackson-future-ideas/issues/9)).

Guía de contribución

No hay ninguna guía de contribución indexada para este repositorio

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.