indygreg / indygreg/PyOxidizer

brotli

Aperta
#167 4 commenti 0 reazioni 0 assegnatari Vedi su GitHub
compatibility
Lingua principale
Rust
Stelle
6.2k
Fork
256
Metriche di merge delle PR
Nessuna PR unita negli ultimi 30g

Descrizione

Adding `brotlipy` fails looking for `-llibbrotli`, which is its own internal copy of the library.

```console
Running command git clone -q https://github.com/jayvdb/brotlipy /tmp/pip-req-build-0y42i1hu
Running command git submodule update --init --recursive -q
Compiling pyembed v0.4.0 (/path/to/project/brotli/pyembed)
Compiling brotli v0.1.0 (/path/to/project/brotli)
error: linking with `cc` failed: exit code: 1
|
= note: "cc" "-Wl,--as-needed" "-Wl,-z,noexecstack" "-m64" "-L" "/usr/lib/rustlib/x86_64-unknown-linux-gnu/lib" "/path/to/project/brotli/build/target/x86_64-unknown-linux-gnu/debug/deps/brotli-f90a18d4864a2a16.15kbg8p4dc6ijdu1.rcgu.o" "/path/to/project/brotli/build/target/x86_64-unknown-linux-gnu/debug/deps/brotli-f90a18d4864a2a16.1jc1u2iuh94lh70o.rcgu.o" "/path/to/project/brotli/build/target/x86_64-unknown-linux-gnu/debug/deps/brotli-f90a18d4864a2a16.22q3drr76jbpis3w.rcgu.o" "/path/to/project/brotli/build/target/x86_64-unknown-linux-gnu/debug/deps/brotli-f90a18d4864a2a16.2syjn2icu1vwbxik.rcgu.o" "/path/to/project/brotli/build/target/x86_64-unknown-linux-gnu/debug/deps/brotli-f90a18d4864a2a16.3bu23526ltwdhtc7.rcgu.o" "/path/to/project/brotli/build/target/x86_64-unknown-linux-gnu/debug/deps/brotli-f90a18d4864a2a16.4y1e3ncj44u9ud8.rcgu.o" "-o" "/path/to/project/brotli/build/target/x86_64-unknown-linux-gnu/debug/deps/brotli-f90a18d4864a2a16" "/path/to/project/brotli/build/target/x86_64-unknown-linux-gnu/debug/deps/brotli-f90a18d4864a2a16.1fz3gb6hjy58le1w.rcgu.o" "-Wl,--gc-sections" "-pie" "-Wl,-zrelro" "-Wl,-znow" "-nodefaultlibs" "-L" "/path/to/project/brotli/build/target/x86_64-unknown-linux-gnu/debug/deps" "-L" "/path/to/project/brotli/build/target/debug/deps" "-L" "/path/to/project/brotli/build/target/x86_64-unknown-linux-gnu/debug/build/jemalloc-sys-6a2dcd7182e77aca/out/build/lib" "-L" "/path/to/project/brotli/build/target/x86_64-unknown-linux-gnu/debug/pyoxidizer" "-L" "/usr/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-Wl,-Bstatic" "/path/to/project/brotli/build/target/x86_64-unknown-linux-gnu/debug/deps/libpyembed-f6d8ce5fcfbe2b59.rlib" "/path/to/project/brotli/build/target/x86_64-unknown-linux-gnu/debug/deps/libuuid-40f161259e10575f.rlib" "/path/to/project/brotli/build/target/x86_64-unknown-linux-gnu/debug/deps/librand-4587c77d50bf082f.rlib" "/path/to/project/brotli/build/target/x86_64-unknown-linux-gnu/debug/deps/librand_xorshift-e19a683b8360bb74.rlib" "/path/to/project/brotli/build/target/x86_64-unknown-linux-gnu/debug/deps/librand_pcg-161749b1ee75ec4d.rlib" "/path/to/project/brotli/build/target/x86_64-unknown-linux-gnu/debug/deps/librand_hc-90ed91a123f3b276.rlib" "/path/to/project/brotli/build/target/x86_64-unknown-linux-gnu/debug/deps/librand_chacha-bb755f08c6cd079a.rlib" "/path/to/project/brotli/build/target/x86_64-unknown-linux-gnu/debug/deps/librand_isaac-bcff91db4f001673.rlib" "/path/to/project/brotli/build/target/x86_64-unknown-linux-gnu/debug/deps/librand_core-2c32e48f67c62942.rlib" "/path/to/project/brotli/build/target/x86_64-unknown-linux-gnu/debug/deps/librand_os-47519e8a6d5b29ed.rlib" "/path/to/project/brotli/build/target/x86_64-unknown-linux-gnu/debug/deps/librand_jitter-4cb1ba9a013dd964.rlib" "/path/to/project/brotli/build/target/x86_64-unknown-linux-gnu/debug/deps/librand_core-3c552559ee9910d6.rlib" "/path/to/project/brotli/build/target/x86_64-unknown-linux-gnu/debug/deps/libjemalloc_sys-ceb86c614fe13a93.rlib" "/path/to/project/brotli/build/target/x86_64-unknown-linux-gnu/debug/deps/liblazy_static-03ee1ee8a2342a8e.rlib" "/path/to/project/brotli/build/target/x86_64-unknown-linux-gnu/debug/deps/libcpython-d6d9ce6fb32c53cc.rlib" "/path/to/project/brotli/build/target/x86_64-unknown-linux-gnu/debug/deps/libnum_traits-76cec740f4fe2b5d.rlib" "/path/to/project/brotli/build/target/x86_64-unknown-linux-gnu/debug/deps/libbyteorder-9bfb225a74a4f789.rlib" "/path/to/project/brotli/build/target/x86_64-unknown-linux-gnu/debug/deps/libpython3_sys-539fc76e435ddd51.rlib" "/path/to/project/brotli/build/target/x86_64-unknown-linux-gnu/debug/deps/liblibc-dae06a64a5f75d6d.rlib" "-Wl,--start-group" "/usr/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd-d655238cc7536e49.rlib" "/usr/lib/rustlib/x86_64-unknown-linux-gnu/lib/libpanic_unwind-a1048e3b4b8487f7.rlib" "/usr/lib/rustlib/x86_64-unknown-linux-gnu/lib/libbacktrace_sys-e303706b808d25cf.rlib" "/usr/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_demangle-7c4c4a242f593b83.rlib" "/usr/lib/rustlib/x86_64-unknown-linux-gnu/lib/libunwind-3a3b1d4df4be24e8.rlib" "/usr/lib/rustlib/x86_64-unknown-linux-gnu/lib/liblibc-9782daa5de8380d8.rlib" "/usr/lib/rustlib/x86_64-unknown-linux-gnu/lib/liballoc-58ece968a1125984.rlib" "/usr/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_std_workspace_core-97498a9a22285b62.rlib" "/usr/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcore-ecc3ad89a49704cf.rlib" "-Wl,--end-group" "/usr/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcompiler_builtins-aeea5fc255482634.rlib" "-Wl,-Bdynamic" "-ldl" "-lm" "-llibbrotli" "-lstdc++" "-lpthread" "-lutil" "-lutil" "-ldl" "-lrt" "-lpthread" "-lgcc_s" "-lc" "-lm" "-lrt" "-lpthread" "-lutil" "-lutil"
= note: /usr/lib64/gcc/x86_64-suse-linux/9/../../../../x86_64-suse-linux/bin/ld: cannot find -llibbrotli
collect2: error: ld returned 1 exit status

error: aborting due to previous error

error: Could not compile `brotli`.

To learn more, run the command again with --verbose.
```

Using git+https://github.com/jayvdb/brotlipy@libbrotli-v1.0-support with envvar `USE_SHARED_BROTLI=1` works better, but then fails with

```py
>>> import brotli
Traceback (most recent call last):
File "", line 1, in
File "brotli", line 3, in
File "brotli.brotli", line 5, in
ModuleNotFoundError: No module named 'brotli._brotli'
```

The brotlipy package shadows the real package `Brotli` from Google, sharing the same installed package name. Luckily they share a similar Python API.
If I can install `brotlipy`, and then install `Brotli` afterwards, the result should be the real Brotli is embedded.

It looks like they are processed in the correct order
```
adding embedded module source: brotli
adding embedded module bytecode: brotli
adding embedded module source: brotli.brotli
adding embedded module bytecode: brotli.brotli
adding embedded module source: brotli.build
adding embedded module bytecode: brotli.build
adding embedded module source: brotli
adding embedded module bytecode: brotli
```
I know brotli.build only exists in brotlipy, so that was included first, and then hopefully overwritten by google Brotli.

It looks like the build is going well

```
resolving inputs for 3 built extension modules...
library brotlienc required by _brotli
library brotlidec required by _brotli
library stdc++ required by _brotli
library ffi required by _cffi_backend
```
I have a static libs build of brotli locally, but I do not see the two brotli static libs being included.

Despite that, it builds ok, but then the error is

```
File "/usr/lib/python3.7/site-packages/requests_mock/adapter.py", line 15, in
from requests.adapters import BaseAdapter
File "requests", line 43, in
File "urllib3", line 7, in
File "urllib3.connectionpool", line 29, in
File "urllib3.connection", line 40, in
File "urllib3.util", line 5, in
File "urllib3.util.request", line 9, in
File "brotli", line 12, in
AttributeError: module '_brotli' has no attribute '__version__'
```
Which looks like the real Google brotli was used, as it does the following during init,
```
import _brotli

# The library version.
__version__ = _brotli.__version__
```
But that suggests something is also wrong with the build of the real Google brotli library.

Guida per i contributori

Nessuna guida per i contributori indicizzata per questo repository

Direzione di ricerca

Inizia riproducendo gli errori di installazione e importazione di `brotlipy`, quindi esamina gli input di build di `pyembed` e `brotli` e le dipendenze di `_brotli` registrate (`brotlienc`, `brotlidec`, `stdc++`). Confronta i moduli incorporati `brotli`, `brotli.brotli` e `brotli.build` con `_brotli` di Google; il lavoro è completo quando il pacchetto si installa e viene importato senza gli errori segnalati del linker, di modulo mancante o di `__version__`.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Valutazione

Stack tecnologico
python, rust
Ambito
build-system, tooling
Tipo di issue
Bug
Difficoltà
5/5
Tempo stimato
Più di una settimana
Stato di attività
Ferma
Chiarezza
Da chiarire
Idoneità per principianti
25/100

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.