apache / apache/iceberg-rust

[EPIC] Support for appending data to iceberg table.

Open
#1,382 17 comments 20 reactions 0 assignees View on GitHub
epic not-stale
Dominant language
Rust
Stars
1.4k
Forks
567
Avg merge
2d 2h
Merged PRs (30d)
93

Description

### What's the feature are you trying to implement?

This issue is used to track implementing appending data to iceberg table. In summary, after we complete all subtasks in this issue, we should be able to execution following insert sql statment in datafusion to insert table to iceberg table:
```sql
insert into t value (1, 'a');
```

For more details about insert statement, please refer to [datafusion doc](https://datafusion.apache.org/user-guide/sql/dml.html#insert).

## Commit path
- [x] #1386
- [x] #1387
- [x] #1388
- [ ] #1389

## Writer path

- [x] #1572
- [x] #1573
- [ ] Implement data writer for non partitioned table.
- [x] https://github.com/apache/iceberg-rust/issues/1541

## Datafusion integration

- [ ] [Implement `insert_into` method for `IcebergTableProvider`](https://github.com/apache/iceberg-rust/issues/1540)
- [ ] https://github.com/apache/iceberg-rust/issues/2027

### Willingness to contribute

I would be willing to contribute to this feature with guidance from the Iceberg Rust community

Contributor guide

Open the contributing guide

Research direction

Review the remaining work in #1389, the non-partitioned data writer, and the DataFusion integration issues #1540 and #2027. Start with those issue descriptions and the linked DataFusion INSERT documentation. Done means the remaining writer and integration subtasks are implemented and the stated INSERT statement works for an Iceberg table.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust, sql
Domain
backend, databases
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.