intake / intake/python-snappy

pypy seg fault on larger byte arrays

Open
#36 4 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
490
Forks
104
PR merge metrics
No merged PRs in 30d

Description

This came up in a pykafka compression issue.

https://github.com/Parsely/pykafka/issues/508

So byte arrays above a certain size causes a seg fault.

```
(pypy) ➜ cat s.py
from uuid import uuid4
import snappy

payload = b''.join([uuid4().bytes for i in range(10)])

c = snappy.compress(payload)
assert snappy.decompress(c) == payload
(pypy) ➜ python s.py
[1] 4587 segmentation fault (core dumped) python s.py
(pypy) ➜ python
Python 2.7.10 (bbd45126bc69, Mar 18 2016, 21:35:08)
[PyPy 5.0.1 with GCC 4.8.4] on linux2
```

while

```
from uuid import uuid4
import snappy

payload = b''.join([uuid4().bytes for i in range(5)])

c = snappy.compress(payload)
assert snappy.decompress(c) == payload
```

passes

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by running the s.py reproduction from the issue with five and ten UUID byte arrays under PyPy 5.0.1, and compare the compression and decompression assertions. Then trace the python-snappy binding involved in those calls and use the pykafka issue for context. Done means larger byte arrays complete without a segmentation fault and the assertions still pass.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.