intake / intake/python-snappy

support pep517 builds - poetry install on Mac fails

Open
#133 9 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
490
Forks
104
PR merge metrics
No merged PRs in 30d

Description

On MacOS 14.3, python 3.11.7, in a project that needs python-snappy as a dependency of another library (in this case, localstack). After installing snappy with brew, the attempt to build python-snappy fails with:

```sh
• Installing python-snappy (0.6.1): Failed

ChefBuildError

Backend subprocess exited when trying to invoke build_wheel

/private/var/folders/s1/ls1zsdv57yj_hxx0ldh9h6jh0000gn/T/tmp2shjyr97/.venv/lib/python3.11/site-packages/setuptools/_distutils/dist.py:265: UserWarning: Unknown distribution option: 'cffi_modules'
warnings.warn(msg)
running bdist_wheel
running build
running build_py
creating build
creating build/lib.macosx-14-x86_64-cpython-311
creating build/lib.macosx-14-x86_64-cpython-311/snappy
copying src/snappy/snappy.py -> build/lib.macosx-14-x86_64-cpython-311/snappy
copying src/snappy/snappy_cffi.py -> build/lib.macosx-14-x86_64-cpython-311/snappy
copying src/snappy/__init__.py -> build/lib.macosx-14-x86_64-cpython-311/snappy
copying src/snappy/hadoop_snappy.py -> build/lib.macosx-14-x86_64-cpython-311/snappy
copying src/snappy/snappy_formats.py -> build/lib.macosx-14-x86_64-cpython-311/snappy
copying src/snappy/__main__.py -> build/lib.macosx-14-x86_64-cpython-311/snappy
copying src/snappy/snappy_cffi_builder.py -> build/lib.macosx-14-x86_64-cpython-311/snappy
running build_ext
building 'snappy._snappy' extension
creating build/temp.macosx-14-x86_64-cpython-311
creating build/temp.macosx-14-x86_64-cpython-311/src
creating build/temp.macosx-14-x86_64-cpython-311/src/snappy
clang -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX14.sdk -I/private/var/folders/s1/ls1zsdv57yj_hxx0ldh9h6jh0000gn/T/tmp2shjyr97/.venv/include -I/usr/local/opt/python@3.11/Frameworks/Python.framework/Versions/3.11/include/python3.11 -c src/snappy/crc32c.c -o build/temp.macosx-14-x86_64-cpython-311/src/snappy/crc32c.o
clang -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX14.sdk -I/private/var/folders/s1/ls1zsdv57yj_hxx0ldh9h6jh0000gn/T/tmp2shjyr97/.venv/include -I/usr/local/opt/python@3.11/Frameworks/Python.framework/Versions/3.11/include/python3.11 -c src/snappy/snappymodule.cc -o build/temp.macosx-14-x86_64-cpython-311/src/snappy/snappymodule.o
src/snappy/snappymodule.cc:33:10: fatal error: 'snappy-c.h' file not found
#include
^~~~~~~~~~~~
1 error generated.
error: command '/usr/bin/clang' failed with exit code 1

at /usr/local/lib/python3.10/site-packages/poetry/installation/chef.py:164 in _prepare
160│
161│ error = ChefBuildError("\n\n".join(message_parts))
162│
163│ if error is not None:
→ 164│ raise error from None
165│
166│ return path
167│
168│ def _prepare_sdist(self, archive: Path, destination: Path | None = None) -> Path:

Note: This error originates from the build backend, and is likely not a problem with poetry but with python-snappy (0.6.1) not supporting PEP 517 builds. You can verify this by running 'pip wheel --no-cache-dir --use-pep517 "python-snappy (==0.6.1)"'.

```

I can manually cause python-snappy to be built with pip, outside of the pep517 / poetry infrastructure, with a workaround command such as the one in the README, e.g. for me:

```sh
CPPFLAGS="-I$(brew --prefix)/include -L$(brew --prefix)/lib" poetry run pip install python-snappy
```

This works but is really frustrating - I see there are quite a lot of MacOS-related build issues already, and am trying to avoid raising a duplicate.

Obviously I'd prefer that the build script just worked entirely, but even if I had to manually set `CPPFLAGS` it would be a big improvement if python-snappy were to support PEP 517 builds directly.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.