apple / apple/coremltools

PyPI sdist missing C++ source — cannot build native libraries from source

Open
#2,847 4 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
Python
Stars
5.4k
Forks
850
Avg merge
4d 5h
Merged PRs (30d)
10

Description

## 🐞Describing the bug

The PyPI source distribution (sdist) does not include the C++ source directories needed to build the native libraries (`libmilstoragepython`, `libmodelpackage`, `kmeans1d/_core`). Building from the sdist produces a wheel without these libraries, so model conversion crashes at runtime.

The Git repository contains everything needed — `CMakeLists.txt`, `deps/`, `mlmodel/`, `cmake/`, `modelpackage/`, `milstoragepython/` — but these are excluded from the sdist tarball. The `setup.py` packages the resulting `.so` files via `package_data`, but since the CMake build never runs during `pip install`, they don't exist.

## Stack Trace

WARNING: No module named 'coremltools.libmilstoragepython'
WARNING: No module named 'coremltools.libcoremlpython'

RuntimeError: BlobWriter not loaded

## To Reproduce

pip install coremltools --no-binary coremltools
python -c "
import coremltools as ct
import torch

model = torch.nn.Linear(10, 5)
traced = torch.jit.trace(model, torch.randn(1, 10))
ct.convert(traced) # RuntimeError: BlobWriter not loaded
"

## System environment (please complete the following information):
- coremltools version: 8.3 and 9.0 (likely all versions)
- OS (e.g. MacOS version or Linux type): Linux x86_64 (RHEL 9), but affects any platform when building from sdist
- Any other relevant version information (e.g. PyTorch or TensorFlow version): Python 3.12, PyTorch 2.6.0

## Additional context

Building from the Git repository with CMake works perfectly — all three native libraries compile and model conversion succeeds. The published wheels on PyPI (both macOS and manylinux) include the native libraries and work fine — the issue is only with the sdist.

The fix would be to include the C++ source directories in `MANIFEST.in` (or equivalent) and wire the CMake build into the PEP 517 build backend so `pip install` from the sdist produces a working package.

Contributor guide

Open the contributing guide

Research direction

Start with setup.py, MANIFEST.in, CMakeLists.txt, and the PEP 517 build configuration, then reproduce the failure with pip install --no-binary coremltools from the sdist. Verify that the sdist contains the listed C++ source directories, that installation builds the native libraries, and that the provided model-conversion command no longer raises BlobWriter not loaded.

Written by the indexing model from the issue text.

Assessment

Tech stack
cmake, cpp, python
Domain
build-system
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
58/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.