apache / apache/iceberg-go

Dependency on github.com/apache/arrow-go/v18/arrow/compute greatly increases binary size

Open
#399 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Go
Stars
463
Forks
232
Avg merge
2d 11h
Merged PRs (30d)
121

Description

### Feature Request / Improvement

We've noticed that including the compute package from arrow-go in our binary contributes ~7mb to the binary size.

At one point (bab83126b2bd5f42949b47b8299d1f28dac86e17) this package was pulled in only by the function `ToRequestedSchema` in arrow_utils.go, which was only used by tests. In newer versions, it appears this is now used in a few more places (arrow_scanner.go and writer.go).

It may not be possible to easily remove the dependency at this point, but just wanted to make you aware.

Contributor guide

Open the contributing guide

Research direction

Start by tracing ToRequestedSchema in arrow_utils.go and its uses in arrow_scanner.go and writer.go. Measure the binary-size impact of the arrow/compute dependency and determine whether the dependency can be removed or isolated; done means preserving the affected behavior while reducing the reported binary-size increase.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
performance
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.