MagicStack / MagicStack/httptools

Custom build_ext breaks parallel build/wheel

Aberta
#126 0 comentários 0 reações 0 responsáveis Ver no GitHub

Ninguém assumiu esta issue ainda.

Linguagem predominante
Python
Estrelas
1.3k
Forks
107
Métricas de merge de PRs
Nenhum PR com merge em 30d

Descrição

Our build system typically uses `setup.py` to generate a wheel via `bdist_wheel` but to speed up compilation of extensions we also explictly call `build` and pass `-j`:

```
python3 ./setup.py build -j 8 bdist_wheel
```

This works for 99% of modules but fails with httptools:
```

running build
running build_py
creating build/lib.linux-aarch64-cpython-312/httptools
copying httptools/_version.py -> build/lib.linux-aarch64-cpython-312/httptools
copying httptools/__init__.py -> build/lib.linux-aarch64-cpython-312/httptools
creating build/lib.linux-aarch64-cpython-312/httptools/parser
copying httptools/parser/errors.py -> build/lib.linux-aarch64-cpython-312/httptools/parser
copying httptools/parser/__init__.py -> build/lib.linux-aarch64-cpython-312/httptools/parser
running egg_info
writing httptools.egg-info/PKG-INFO
writing dependency_links to httptools.egg-info/dependency_links.txt
writing requirements to httptools.egg-info/requires.txt
writing top-level names to httptools.egg-info/top_level.txt
/usr/lib/python3/dist-packages/pbr/git.py:28: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81.
import pkg_resources
reading manifest file 'httptools.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
adding license file 'LICENSE'
writing manifest file 'httptools.egg-info/SOURCES.txt'
copying httptools/parser/cparser.pxd -> build/lib.linux-aarch64-cpython-312/httptools/parser
copying httptools/parser/parser.pyx -> build/lib.linux-aarch64-cpython-312/httptools/parser
copying httptools/parser/python.pxd -> build/lib.linux-aarch64-cpython-312/httptools/parser
copying httptools/parser/url_cparser.pxd -> build/lib.linux-aarch64-cpython-312/httptools/parser
copying httptools/parser/url_parser.pyx -> build/lib.linux-aarch64-cpython-312/httptools/parser
running build_ext
building 'httptools.parser.parser' extension
creating build/temp.linux-aarch64-cpython-312/httptools/parser
building 'httptools.parser.url_parser' extension
creating build/temp.linux-aarch64-cpython-312/vendor/llhttp/src
creating build/temp.linux-aarch64-cpython-312/vendor/http-parser
aarch64-linux-gnu-gcc -fno-strict-overflow -Wsign-compare -DNDEBUG -g -O2 -Wall -fPIC -I/work/ross/build/tmp/work/core2-64-poky-linux/python3-httptools/0.6.4/sources/httptools-0.6.4/vendor/llhttp/include -I/work/ross/build/tmp/work/core2-64-poky-linux/python3-httptools/0.6.4/sources/httptools-0.6.4/vendor/llhttp/src -I/usr/include/python3.12 -c httptools/parser/parser.c -o build/temp.linux-aarch64-cpython-312/httptools/parser/parser.o -O2
aarch64-linux-gnu-gcc -fno-strict-overflow -Wsign-compare -DNDEBUG -g -O2 -Wall -fPIC -I/work/ross/build/tmp/work/core2-64-poky-linux/python3-httptools/0.6.4/sources/httptools-0.6.4/vendor/http-parser -I/usr/include/python3.12 -c httptools/parser/url_parser.c -o build/temp.linux-aarch64-cpython-312/httptools/parser/url_parser.o -O2
aarch64-linux-gnu-gcc -fno-strict-overflow -Wsign-compare -DNDEBUG -g -O2 -Wall -fPIC -I/work/ross/build/tmp/work/core2-64-poky-linux/python3-httptools/0.6.4/sources/httptools-0.6.4/vendor/http-parser -I/usr/include/python3.12 -c vendor/http-parser/http_parser.c -o build/temp.linux-aarch64-cpython-312/vendor/http-parser/http_parser.o -O2
aarch64-linux-gnu-gcc -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -Wl,-z,relro -g -fwrapv -O2 build/temp.linux-aarch64-cpython-312/httptools/parser/url_parser.o build/temp.linux-aarch64-cpython-312/vendor/http-parser/http_parser.o -L/usr/lib/aarch64-linux-gnu -o build/lib.linux-aarch64-cpython-312/httptools/parser/url_parser.cpython-312-aarch64-linux-gnu.so
aarch64-linux-gnu-gcc -fno-strict-overflow -Wsign-compare -DNDEBUG -g -O2 -Wall -fPIC -I/work/ross/build/tmp/work/core2-64-poky-linux/python3-httptools/0.6.4/sources/httptools-0.6.4/vendor/llhttp/include -I/work/ross/build/tmp/work/core2-64-poky-linux/python3-httptools/0.6.4/sources/httptools-0.6.4/vendor/llhttp/src -I/usr/include/python3.12 -c vendor/llhttp/src/api.c -o build/temp.linux-aarch64-cpython-312/vendor/llhttp/src/api.o -O2
aarch64-linux-gnu-gcc -fno-strict-overflow -Wsign-compare -DNDEBUG -g -O2 -Wall -fPIC -I/work/ross/build/tmp/work/core2-64-poky-linux/python3-httptools/0.6.4/sources/httptools-0.6.4/vendor/llhttp/include -I/work/ross/build/tmp/work/core2-64-poky-linux/python3-httptools/0.6.4/sources/httptools-0.6.4/vendor/llhttp/src -I/usr/include/python3.12 -c vendor/llhttp/src/http.c -o build/temp.linux-aarch64-cpython-312/vendor/llhttp/src/http.o -O2
aarch64-linux-gnu-gcc -fno-strict-overflow -Wsign-compare -DNDEBUG -g -O2 -Wall -fPIC -I/work/ross/build/tmp/work/core2-64-poky-linux/python3-httptools/0.6.4/sources/httptools-0.6.4/vendor/llhttp/include -I/work/ross/build/tmp/work/core2-64-poky-linux/python3-httptools/0.6.4/sources/httptools-0.6.4/vendor/llhttp/src -I/usr/include/python3.12 -c vendor/llhttp/src/llhttp.c -o build/temp.linux-aarch64-cpython-312/vendor/llhttp/src/llhttp.o -O2
aarch64-linux-gnu-gcc -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -Wl,-z,relro -g -fwrapv -O2 build/temp.linux-aarch64-cpython-312/httptools/parser/parser.o build/temp.linux-aarch64-cpython-312/vendor/llhttp/src/api.o build/temp.linux-aarch64-cpython-312/vendor/llhttp/src/http.o build/temp.linux-aarch64-cpython-312/vendor/llhttp/src/llhttp.o -L/usr/lib/aarch64-linux-gnu -o build/lib.linux-aarch64-cpython-312/httptools/parser/parser.cpython-312-aarch64-linux-gnu.so

running bdist_wheel
installing to build/bdist.linux-aarch64/wheel
running install
running install_lib
running build_ext
error: don't know how to compile C/C++ code on platform 'posix' with '' compiler
```

This behaviour isn't specific to the use of `-j`, simply doing `python3 setup.py build bdist_wheel` results in the build failing.

Guia de contribuição

Nenhum guia de contribuição indexado para este repositório

Primeiros passos

  1. Leia a issue inteira e depois o guia de contribuição do projeto.
  2. Comente na issue dizendo que vai assumir — evita que duas pessoas façam o mesmo trabalho.
  3. Faça um fork do repositório e trabalhe em uma branch.
  4. Abra um pull request que referencie o número da issue.

Direção de pesquisa

Comece com setup.py e reproduza a falha usando `python3 setup.py build bdist_wheel`; em seguida, compare a execução inicial de build_ext com a invocação de build_ext durante bdist_wheel. O trabalho estará concluído quando esse comando for concluído com sucesso e produzir um wheel sem o erro do compilador.

Escrita pelo modelo de indexação a partir do texto da issue.

Avaliação

Stack de tecnologia
python
Domínio
build-system
Tipo de issue
Bug
Dificuldade
3/5
Tempo estimado
1-2 dias
Status de atividade
Estagnada
Clareza
Razoavelmente clara
Facilidade para iniciantes
48/100

Receba novas issues na sua caixa de entrada

Um resumo curto de issues do GitHub para quem está começando.