aio-libs / aio-libs/aiokafka

`test_read_write_serde_v2`[zstd] fails with cramjam 2.11.0

オープン
#1,124 コメント 0 件 リアクション 0 件 担当者 0 名 GitHub で見る
主要言語
Python
スター
1.4k
フォーク
269
平均マージ
1日 1時間
マージ済み PR(30日)
6

説明

**Describe the bug**
The test `test_read_write_serde_v2` fails when run with Zstandard compression on environments where `python-module-cramjam` version `2.11.0-rc4` or newer is installed. The test fails with an `AssertionError` because the actual CRC value from the compressed data does not match the hardcoded expected value.

**Failure Log Example**:
```
=========================== FAILURES ===========================
__________________ test_read_write_serde_v2[zstd] __________________

request = >
compression_type = 2, crc = 1714138923

@pytest.mark.parametrize("compression_type,crc", _test_serde_v2_params)
@pytest.mark.asyncio
async def test_read_write_serde_v2(
request, compression_type: int, crc: Optional[int]
):
...
if crc is not None:
> assert reader.crc == crc
E assert 1679657554 == 1714138923
E + where 1679657554 =
tests/record/test_default_records.py:68: AssertionError
```

**Expected behaviour**
The CRC check should pass regardless of the `cramjam` version used, or the test should be adapted to handle the version-dependent CRC values. The core functionality of compressing and decompressing data should remain compatible.

**Cause**
A [change](https://github.com/milesgranger/cramjam/commit/ce049ca79600dda7fa53cef7493d5fc996ba2a0a) in `cramjam` `2.11.0-rc4` alters the Zstandard compression output by including the original data size in the compressed stream's metadata. This modification causes the CRC to change from `1714138923` to `1679657554`. The current hardcoded CRC value in the test suite is no longer valid for newer `cramjam` versions.

**Proposed Solution**
I've created a patch that modifies the test to handle both CRC values based on the `cramjam` version. This maintains backward compatibility and ensures the test passes in all environments.

**Suggested change:**
Dynamically set the expected CRC based on the `cramjam` version.

```python
try:
_cramjam_version = tuple(map(int, cramjam.__version__.split('.')))
_zstd_crc = 1679657554 if _cramjam_version >= (2, 11, 0) else 1714138923
except (AttributeError, ValueError):
_zstd_crc = 1714138923
...
pytest.param(DefaultRecordBatch.CODEC_ZSTD, _zstd_crc, id="zstd"),
```

**Environment (please complete the following information):**
- aiokafka version: 0.12.0
- Kafka Broker version: Not applicable
- Python version: 3.12.11
- cramjam version: 2.11.0

**Reproducible example**
Run `pytest tests/record/test_default_records.py` in an environment `with cramjam>=2.11.0`. The test will fail as shown in the log above.

コントリビューションガイド

コントリビューションガイドを開く

評価

この issue はまだ評価されていません。

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。