oceanbase / oceanbase/seekdb

Replace source code dependency for compression libraries with header/static library dependency

Open
#431 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

dima type: task
Dominant language
C++
Stars
2.9k
Forks
339
Avg merge
1d 19h
Merged PRs (30d)
52

Description

Description

Currently, several compression algorithms in use are integrated by directly including their source code into the project (over 100,000 lines of code). This practice has historical roots: in the early stages, there was a desire to avoid dependency on dynamic library versions in the runtime environment to prevent data inconsistency across distributed nodes. At that time, the static library build system for the observer was not mature, so source code dependencies were adopted. Subsequent introductions of other compression libraries have 'inherited' this habit.

This approach is suboptimal for two main reasons:

  1. Non-standard Practice: It is not a standard method. Dependencies should ideally be managed via header files and .a (static library) files.
  2. Bloat: It leads to project bloat, which manifests in increased codebase size and longer compilation times.

Technical Details

  • Source code location: deps/oblib/src/lib/compress
  • Total lines of code for compression libraries: 104,844 (as counted by find . -name '*' -type f -exec cat {} \; | wc -l)

Proposed Solution

Transition the dependency model for compression libraries from direct source code inclusion to a standard library-based approach (e.g., using header files and pre-built static libraries .a). This will improve build hygiene, reduce compilation overhead, and align with standard software engineering practices.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by inventorying the compression libraries and their inclusion points under deps/oblib/src/lib/compress, then trace how the observer build currently compiles them. Determine the header and static-library dependency arrangement needed for each library. Done means the libraries are no longer directly included as source, the build remains functional, and compilation overhead and source bloat are reduced.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
build-system
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.