conda-forge / conda-forge/python-feedstock
python 3.14 compiled with zlib instead of zlib-ng -- 30% slower on windows for zlib
- Dominant language
- Shell
- Stars
- 47
- Forks
- 112
- Avg merge
- 2d 9h
- Merged PRs (30d)
- 18
Description
### Solution to issue cannot be found in the documentation.
- [x] I checked the documentation.
### Issue
Hello,
I am trying the new release of python 3.14.
This is a short script to compress a log file that's 58 MB.
Running on my laptop with turbo boost off to have stable results.
official python release on windows (you can download from python.org)
```
.\Python314\python.exe .\bench.py
Data size: 58.82 MB
compress: 0.618 seconds (95.13 MB/s)
decompress: 0.059 seconds (1003.92 MB/s)
crc32: 0.002 seconds (26193.28 MB/s)
```
python release on conda forge (newly created with `conda create ... python=3.14`)
```
.\myconda314\python.exe .\bench.py
Data size: 58.82 MB
compress: 1.056 seconds (55.72 MB/s)
decompress: 0.107 seconds (551.28 MB/s)
crc32: 0.022 seconds (2656.00 MB/s)
```
PROBLEM: conda python is 30%-50% slower than the official release of python, 1000% slower on crc 😢
REASON: With the official release of python 3.14 on Windows, the python interpreter has switched to zlib-ng as the compression library. This provides an enormous performance boost to compression/decompression/crc routines.
This affects all usage of compression including zlib, gzip, tarfile, pip install, etc...
https://github.com/python/cpython/issues/91349?reload=1
Conda is still using the old zlib library, that's slow and mostly unmaintained.
Do you think you people could update conda to build with zlib-ng in compatibility mode?
It's a drop in replacement for zlib.
Regards.
### Installed packages
```shell
python=3.14 on windows
```
### Environment info
windows laptop
bench.py
```shell
import zlib
import time
# Load data
with open('access.log', 'rb') as f:
data = f.read()
size_mb = len(data) / 1024 / 1024
print(f"Data size: {size_mb:.2f} MB\n")
# Compress
start = time.perf_counter()
compressed = zlib.compress(data)
compress_time = time.perf_counter() - start
print(f"compress: {compress_time:.3f} seconds ({size_mb/compress_time:.2f} MB/s)")
# Decompress
start = time.perf_counter()
decompressed = zlib.decompress(compressed)
decompress_time = time.perf_counter() - start
print(f"decompress: {decompress_time:.3f} seconds ({size_mb/decompress_time:.2f} MB/s)")
# CRC32
start = time.perf_counter()
checksum = zlib.crc32(data)
crc_time = time.perf_counter() - start
print(f"crc32: {crc_time:.3f} seconds ({size_mb/crc_time:.2f} MB/s)")
```
Contributor guide
Research direction
Start by reviewing the python-feedstock build configuration and dependency metadata for Windows Python 3.14, then compare how the official Python release selects zlib-ng. Confirm the compatibility-mode dependency is used for the conda build and rerun the issue's compression, decompression, and CRC32 benchmark to verify the performance difference.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- build-system, performance
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100