conda-forge / conda-forge/python-feedstock

python 3.14 compiled with zlib instead of zlib-ng -- 30% slower on windows for zlib

Offen
#821 8 Kommentare 0 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen
bug
Vorherrschende Sprache
Shell
Sterne
47
Forks
112
Ø Merge
2 T. 9 Std.
Gemergte PRs (30 T.)
18

Beschreibung

### Solution to issue cannot be found in the documentation.

- [x] I checked the documentation.

### Issue

Hello,

I am trying the new release of python 3.14.
This is a short script to compress a log file that's 58 MB.
Running on my laptop with turbo boost off to have stable results.

official python release on windows (you can download from python.org)
```
.\Python314\python.exe .\bench.py
Data size: 58.82 MB

compress: 0.618 seconds (95.13 MB/s)
decompress: 0.059 seconds (1003.92 MB/s)
crc32: 0.002 seconds (26193.28 MB/s)
```

python release on conda forge (newly created with `conda create ... python=3.14`)

```
.\myconda314\python.exe .\bench.py
Data size: 58.82 MB

compress: 1.056 seconds (55.72 MB/s)
decompress: 0.107 seconds (551.28 MB/s)
crc32: 0.022 seconds (2656.00 MB/s)
```

PROBLEM: conda python is 30%-50% slower than the official release of python, 1000% slower on crc 😢

REASON: With the official release of python 3.14 on Windows, the python interpreter has switched to zlib-ng as the compression library. This provides an enormous performance boost to compression/decompression/crc routines.
This affects all usage of compression including zlib, gzip, tarfile, pip install, etc...
https://github.com/python/cpython/issues/91349?reload=1

Conda is still using the old zlib library, that's slow and mostly unmaintained.

Do you think you people could update conda to build with zlib-ng in compatibility mode?
It's a drop in replacement for zlib.

Regards.

### Installed packages

```shell
python=3.14 on windows
```

### Environment info

windows laptop

bench.py

```shell
import zlib
import time

# Load data
with open('access.log', 'rb') as f:
data = f.read()

size_mb = len(data) / 1024 / 1024
print(f"Data size: {size_mb:.2f} MB\n")

# Compress
start = time.perf_counter()
compressed = zlib.compress(data)
compress_time = time.perf_counter() - start

print(f"compress: {compress_time:.3f} seconds ({size_mb/compress_time:.2f} MB/s)")

# Decompress
start = time.perf_counter()
decompressed = zlib.decompress(compressed)
decompress_time = time.perf_counter() - start

print(f"decompress: {decompress_time:.3f} seconds ({size_mb/decompress_time:.2f} MB/s)")

# CRC32
start = time.perf_counter()
checksum = zlib.crc32(data)
crc_time = time.perf_counter() - start

print(f"crc32: {crc_time:.3f} seconds ({size_mb/crc_time:.2f} MB/s)")
```

Beitragsleitfaden

Beitragsleitfaden öffnen

Bewertung

Dieses Issue wurde noch nicht bewertet.

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.