conda-forge / conda-forge/python-feedstock

python 3.14 compiled with zlib instead of zlib-ng -- 30% slower on windows for zlib

Open
#821 8 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
Shell
Stars
47
Forks
112
Avg merge
2d 9h
Merged PRs (30d)
18

Description

### Solution to issue cannot be found in the documentation.

- [x] I checked the documentation.

### Issue

Hello,

I am trying the new release of python 3.14.
This is a short script to compress a log file that's 58 MB.
Running on my laptop with turbo boost off to have stable results.

official python release on windows (you can download from python.org)
```
.\Python314\python.exe .\bench.py
Data size: 58.82 MB

compress: 0.618 seconds (95.13 MB/s)
decompress: 0.059 seconds (1003.92 MB/s)
crc32: 0.002 seconds (26193.28 MB/s)
```

python release on conda forge (newly created with `conda create ... python=3.14`)

```
.\myconda314\python.exe .\bench.py
Data size: 58.82 MB

compress: 1.056 seconds (55.72 MB/s)
decompress: 0.107 seconds (551.28 MB/s)
crc32: 0.022 seconds (2656.00 MB/s)
```

PROBLEM: conda python is 30%-50% slower than the official release of python, 1000% slower on crc 😢

REASON: With the official release of python 3.14 on Windows, the python interpreter has switched to zlib-ng as the compression library. This provides an enormous performance boost to compression/decompression/crc routines.
This affects all usage of compression including zlib, gzip, tarfile, pip install, etc...
https://github.com/python/cpython/issues/91349?reload=1

Conda is still using the old zlib library, that's slow and mostly unmaintained.

Do you think you people could update conda to build with zlib-ng in compatibility mode?
It's a drop in replacement for zlib.

Regards.

### Installed packages

```shell
python=3.14 on windows
```

### Environment info

windows laptop

bench.py

```shell
import zlib
import time

# Load data
with open('access.log', 'rb') as f:
data = f.read()

size_mb = len(data) / 1024 / 1024
print(f"Data size: {size_mb:.2f} MB\n")

# Compress
start = time.perf_counter()
compressed = zlib.compress(data)
compress_time = time.perf_counter() - start

print(f"compress: {compress_time:.3f} seconds ({size_mb/compress_time:.2f} MB/s)")

# Decompress
start = time.perf_counter()
decompressed = zlib.decompress(compressed)
decompress_time = time.perf_counter() - start

print(f"decompress: {decompress_time:.3f} seconds ({size_mb/decompress_time:.2f} MB/s)")

# CRC32
start = time.perf_counter()
checksum = zlib.crc32(data)
crc_time = time.perf_counter() - start

print(f"crc32: {crc_time:.3f} seconds ({size_mb/crc_time:.2f} MB/s)")
```

Contributor guide

Open the contributing guide

Research direction

Start by reviewing the python-feedstock build configuration and dependency metadata for Windows Python 3.14, then compare how the official Python release selects zlib-ng. Confirm the compatibility-mode dependency is used for the conda build and rerun the issue's compression, decompression, and CRC32 benchmark to verify the performance difference.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
build-system, performance
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.