es-ude / es-ude/OnDeviceTraining

build: consolidate circular micro static-libraries to remove link-line duplication at the source

Open
#238 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
C
Stars
1
Forks
3
Avg merge
1d 1h
Merged PRs (30d)
8

Description

## Context

Follow-up to #234, which applied the *proportionate, cosmetic* fix: a feature-detected `-Wl,-no_warn_duplicate_libraries` that silences Apple `ld-prime`'s "ignoring duplicate libraries" noise on macOS dev builds. This issue tracks the **structural root cause** that #234 deliberately did **not** address.

## Root cause

The framework splits each operation into its own static archive (~14 in `src/arithmetic`, ~13 in `src/layer`, plus `src/tensor`). The C-vtable dispatcher pattern then creates genuine *circular* dependencies **between** archives:

- `Arithmetic` ↔ `Add` / `Sub` / `Mul` / `Div` (dispatcher calls the ops; the ops call back for shared helpers)
- `Layer` ↔ `Linear` / `Relu` / `Conv1d` / `Softmax` / `Flatten` / …
- `Tensor` ↔ `DataStorage` ↔ `MinMax`

When archives mutually reference each other, CMake repeats the entire strongly-connected component on the link line so that single-pass GNU `ld` (the Linux CI linker) can resolve the back-references. Each test executable's link line therefore lists the SCC clusters **twice** (verified: every cyclic archive appears exactly 2×). macOS `ld-prime` is multi-pass, doesn't need the repeats, and emits the warnings #234 silenced (59 per full clean build).

## Proposal

Consolidate each cyclic cluster into a single archive per `src/` subdirectory, turning the cycle *intra-archive* (resolved by the archive's own symbol table — no link-line repeats on any platform):

- merge the `src/arithmetic` ops into one `Arithmetic` archive
- merge the `src/layer` dispatcher + implementations into one `Layer` archive
- merge the `src/tensor` core into one `Tensor` archive

**Benefits:** no SCC repetition on macOS *or* Linux, shorter/simpler link lines, faster configure, and the per-test `MORE_LIBS` lists shrink dramatically. `CLAUDE.md` already permits "one or more static libraries" per subdirectory.

## Caveats

- Large refactor touching nearly every `src/**/CMakeLists.txt` and most `test/unit/**/CMakeLists.txt` `MORE_LIBS` lists.
- Crosses code-ownership (the quantize-training layers are Jan's).
- Payoff is cosmetic on top of #234 — **low priority**.

Surfaced during the #234 investigation.

Contributor guide

Open the contributing guide

Research direction

Start with the CMakeLists.txt files under src/arithmetic, src/layer, and src/tensor, then inspect the test/unit/**/CMakeLists.txt MORE_LIBS lists. Consolidate each listed cyclic cluster into one static archive per subdirectory and update the affected test link lists. Done means the clusters no longer repeat on link lines while Linux CI and macOS builds continue to link successfully.

Written by the indexing model from the issue text.

Assessment

Tech stack
c, cmake
Domain
build-system
Issue type
Refactor
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.