apache / apache/iceberg-go

Enormous dependency tree — consider "drivers"?

Open
#696 22 comments 0 reactions 1 assignee Claimed by @laskoviymishka View on GitHub
Dominant language
Go
Stars
463
Forks
232
Avg merge
2d 11h
Merged PRs (30d)
121

Description

### Feature Request / Improvement

Given an empty project with Iceberg-go as its only dependency:

``` sh
cat main.go
```
```go
package main

import (
_ "github.com/apache/iceberg-go"
_ "github.com/apache/iceberg-go/catalog"
_ "github.com/apache/iceberg-go/catalog/rest"
_ "github.com/apache/iceberg-go/io"
_ "github.com/apache/iceberg-go/table"
)

func main() {
}
```

```sh
head go.mod
```
```
module github.com/brandur/iceberg-go-test

go 1.25.6

require github.com/apache/iceberg-go v0.4.1-0.20260123180146-3ce3b4bb51c0

require (
atomicgo.dev/cursor v0.2.0 // indirect
atomicgo.dev/keyboard v0.2.9 // indirect
atomicgo.dev/schedule v0.1.0 // indirect
```

Iceberg-go adds ~1s of compile overhead (on my commodity laptop). This might not be the end of the world, but you end up soaking it up on every test run compile and test run, which is fairly painful for any projects using Iceberg-go:

```
$ go test .
ok github.com/brandur/iceberg-go-test 0.781s [no tests to run]
```

The project has about 130 dependencies. Manually inspecting the list, a big number of them are from the three major cloud providers (AWS, Azure, Google). Being able to jettison those would probably help a lot.

An approach that we took in one of our projects (River) is to use a "driver" system wherein you have sub-modules for major dependencies so users can pull them in piece by piece. e.g.

https://github.com/riverqueue/river/tree/master/riverdriver

It's far from perfect as Go's multi-module handling leaves a lot to be desired, but it works reasonably well, and a similar approach would probably help Iceberg-go in the longer run. Thoughts?

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.