conda-forge / conda-forge/python-feedstock
python 3.14 compiled with zlib instead of zlib-ng -- 30% slower on windows for zlib
- Dominant language
- Shell
- Stars
- 47
- Forks
- 112
- Avg merge
- 2d 9h
- Merged PRs (30d)
- 18
Description
### Solution to issue cannot be found in the documentation.
- [x] I checked the documentation.
### Issue
Hello,
I am trying the new release of python 3.14.
This is a short script to compress a log file that's 58 MB.
Running on my laptop with turbo boost off to have stable results.
official python release on windows (you can download from python.org)
```
.\Python314\python.exe .\bench.py
Data size: 58.82 MB
compress: 0.618 seconds (95.13 MB/s)
decompress: 0.059 seconds (1003.92 MB/s)
crc32: 0.002 seconds (26193.28 MB/s)
```
python release on conda forge (newly created with `conda create ... python=3.14`)
```
.\myconda314\python.exe .\bench.py
Data size: 58.82 MB
compress: 1.056 seconds (55.72 MB/s)
decompress: 0.107 seconds (551.28 MB/s)
crc32: 0.022 seconds (2656.00 MB/s)
```
PROBLEM: conda python is 30%-50% slower than the official release of python, 1000% slower on crc 😢
REASON: With the official release of python 3.14 on Windows, the python interpreter has switched to zlib-ng as the compression library. This provides an enormous performance boost to compression/decompression/crc routines.
This affects all usage of compression including zlib, gzip, tarfile, pip install, etc...
https://github.com/python/cpython/issues/91349?reload=1
Conda is still using the old zlib library, that's slow and mostly unmaintained.
Do you think you people could update conda to build with zlib-ng in compatibility mode?
It's a drop in replacement for zlib.
Regards.
### Installed packages
```shell
python=3.14 on windows
```
### Environment info
windows laptop
bench.py
```shell
import zlib
import time
# Load data
with open('access.log', 'rb') as f:
data = f.read()
size_mb = len(data) / 1024 / 1024
print(f"Data size: {size_mb:.2f} MB\n")
# Compress
start = time.perf_counter()
compressed = zlib.compress(data)
compress_time = time.perf_counter() - start
print(f"compress: {compress_time:.3f} seconds ({size_mb/compress_time:.2f} MB/s)")
# Decompress
start = time.perf_counter()
decompressed = zlib.decompress(compressed)
decompress_time = time.perf_counter() - start
print(f"decompress: {decompress_time:.3f} seconds ({size_mb/decompress_time:.2f} MB/s)")
# CRC32
start = time.perf_counter()
checksum = zlib.crc32(data)
crc_time = time.perf_counter() - start
print(f"crc32: {crc_time:.3f} seconds ({size_mb/crc_time:.2f} MB/s)")
```
Contributor guide
Assessment
This issue has not been assessed yet.