apache / apache/iceberg-rust

[Discussion] Reduce the need for `iceberg-rust` forks / how can we grow the community?

Open
#1,797 32 comments 45 reactions 0 assignees View on GitHub
not-stale
Dominant language
Rust
Stars
1.4k
Forks
567
Avg merge
2d 2h
Merged PRs (30d)
93

Description

First of all, I hope it goes without saying thank you so much for all your work on this crate 🙏

# Problem Statement

Recently, I have heard from multiple people and projects that they are maintaining (or considering maintaining) forks or reimplementations of `iceberg-rust` because `iceberg-rust` is not yet suitable for them for some reason.

I think the speaks to a great demand for iceberg functionality in Rust. Having multiple different forks / implementations is of course an option, but I also believe it means the same amount of effort will be split across multiple projects, meaning the available capacity for each will inevitably be smaller than if our efforts were combined somehow

Selfishly, from my DataFusion 🎩 perspective, many projects are interested (understandably) in building systems that use DataFusion to read data stored in Apache Iceberg. I feel that the lack of a super-easy-to-use Iceberg integration crate is holding many of these projects back

Unfortunately I don't have a list of things that would allow these projects to use `iceberg-rust`, but I hope to generate that list using this ticket.

# Ideas to fix:

As I am only relaying this need second hand I don't have many specific ideas to share, and I hope people with firsthand challenges will comment directly on this issue.

One strong strong suspicion is looking at the project [commits](https://github.com/apache/iceberg-rust/commits/main/) is that it might help to get some more capacity for reviews / docs / whatever (it appears that @liurenjie1024 is doing the bulk of reviews recently, and @Xuanwo has been helping too)

As always in open source projects, I think the most productive conversation is NOT "people asking the maintainers for things" but rather "how can the people asking for things also contribute to the solution"

# Call to Action
Thus I would like to ask:

**To people considering a fork**:
1. What specific challenges are you facing that means you can't use iceberg-rust
2. What, if anything, are you willing to do / contribute to help get iceberg-rust to the point where you could use it directly

**To the existing maintainers**:
1. What would help you the most maintaining / moving this crate forward? How can we help?

Contributor guide

Open the contributing guide

Research direction

Start by reading the issue and its 32 comments, then collect the specific blockers preventing projects from using iceberg-rust directly. The issue names no source files, tests, or entry points. Done would be an agreed list of challenges, contributions, and actionable priorities for the project and community.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
data-engineering
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.