apache / apache/datafusion-comet

Explore MUSL to avoid GLIBC requirement

Open
#138 6 comments 0 reactions 0 assignees View on GitHub
build enhancement
Dominant language
Scala
Stars
1.3k
Forks
373
Avg merge
2d 4h
Merged PRs (30d)
198

Description

### What is the problem the feature request solves?

Currently we require dynamically linked glibc on the host machine which isn't desirable since it requires certain version of glibc, otherwise the native Comet lib won't be able to load. In particular, the glibc version used to compile and build Comet must be compatible (i.e., older) than the version used to execute Comet.

Another option is to explore `x86_64-unknown-linux-musl` compile target, which statically links against the [MUSL libc](https://musl.libc.org/) and can effectively remove the above constraints. However, t[here](https://andygrove.io/2020/05/why-musl-extremely-slow/) are some reports that MUSL can cause performance degradation, see [here](https://www.reddit.com/r/rust/comments/oh2k8l/rust_musl_and_glibc_in_2021/) and here. We may need to do some benchmark to evaluate.

### Describe the potential solution

Evaluate MUSL and check if it can be a potential candidate to co-exist (or replace) with the current GLIBC approach.

### Additional context

_No response_

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.