apache / apache/paimon-rust

Tracking issues of 0.1.0 version for Apache Paimon Rust

Abierto
#3 21 comentarios 21 reacciones 0 asignados Ver en GitHub
Lenguaje dominante
Rust
Estrellas
197
Forks
92
Merge medio
1 d 20 h
PR fusionados (30 d)
108

Descripción

Hello everyone, this is the tracking update for the paimon rust 0.1.0 release.

## Goal

Before we outline the tasks for the 0.1.0 release, let me clarify the project's goal:

**Developing a complete implementation of Paimon in pure Rust.**

- Users can read/write paimon table like they do in java API.
- Users can read/write paimon table in arrow format.
- Native support for DataFusion
- But also enable users to implement their own query engines based on this project.
- Native WASM support (a.k.a `paimon-wasm`)
- Native Python binding based on rust core (a.k.a `paimon-py`)
- Hive catalog support

## Tasks

This will be our initial release, and I aim to include basic read support in it.

- [Spec](https://paimon.apache.org/docs/master/concepts/specification/): Implement types that needed by paimon.
- [x] https://github.com/apache/paimon-rust/pull/5
- [x] Datatypes
- [x] https://github.com/apache/paimon-rust/issues/6
- [x] https://github.com/apache/paimon-rust/issues/14
- [x] https://github.com/apache/paimon-rust/issues/7
- [x] https://github.com/apache/paimon-rust/issues/10
- [ ] Global Index
- [ ] Data File Index
- Catalog
- [x] https://github.com/apache/paimon-rust/pull/62
- [x] https://github.com/apache/paimon-rust/issues/70 (leave hive catalog in next release)
- Arrow Integration:
- [x] https://github.com/apache/paimon-rust/issues/71
- Basic Read Process: No schema evolution, no merging, no deletion.
- [x] https://github.com/apache/paimon-rust/issues/72 (filter & project push down are not required in v0.1)
- IO: Integrate with [Apache OpenDAL](https://github.com/apache/opendal) for IO.
- Release Utils: Impelment scripts to help generate and verify ASF releases.

After all those tasks, I expect users can read an existing paimon table from storage services.

Guía de contribución

Abrir la guía de contribución

Línea de trabajo

Revisa la especificación enlazada, los Issues 6, 7, 10, 14, 70, 71 y 72, junto con las tareas sin marcar de Global Index, Data File Index, IO y Release Utils. Empieza por identificar qué tarea sigue abierta y cuáles son sus dependencias. Se considera terminado cuando las tareas de 0.1.0 están completas y los usuarios pueden leer una tabla de Paimon existente desde servicios de almacenamiento.

Escrito por el modelo de indexación a partir del texto del issue.

Evaluación

Stack tecnológico
python, rust, wasm
Área
data-engineering, release
Tipo de issue
Nueva funcionalidad
Dificultad
5/5
Tiempo estimado
Más de una semana
Estado de actividad
Estancado
Claridad
Bastante claro
Aptitud para principiantes
25/100

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.