[EPIC] Support for appending data to iceberg table.
- Dominant language
- Rust
- Stars
- 1.4k
- Forks
- 567
- Avg merge
- 2d 2h
- Merged PRs (30d)
- 93
Description
### What's the feature are you trying to implement?
This issue is used to track implementing appending data to iceberg table. In summary, after we complete all subtasks in this issue, we should be able to execution following insert sql statment in datafusion to insert table to iceberg table:
```sql
insert into t value (1, 'a');
```
For more details about insert statement, please refer to [datafusion doc](https://datafusion.apache.org/user-guide/sql/dml.html#insert).
## Commit path
- [x] #1386
- [x] #1387
- [x] #1388
- [ ] #1389
## Writer path
- [x] #1572
- [x] #1573
- [ ] Implement data writer for non partitioned table.
- [x] https://github.com/apache/iceberg-rust/issues/1541
## Datafusion integration
- [ ] [Implement `insert_into` method for `IcebergTableProvider`](https://github.com/apache/iceberg-rust/issues/1540)
- [ ] https://github.com/apache/iceberg-rust/issues/2027
### Willingness to contribute
I would be willing to contribute to this feature with guidance from the Iceberg Rust community
Contributor guide
Research direction
Review the remaining work in #1389, the non-partitioned data writer, and the DataFusion integration issues #1540 and #2027. Start with those issue descriptions and the linked DataFusion INSERT documentation. Done means the remaining writer and integration subtasks are implemented and the stated INSERT statement works for an Iceberg table.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust, sql
- Domain
- backend, databases
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100