snappy import error
- Dominant language
- Python
- Stars
- 1.7k
- Forks
- 778
- Avg merge
- 2h 50m
- Merged PRs (30d)
- 3
Description
**What happened**:
I'm trying to install a python module named snappy inside a python environment where dask.distributed is already installed.
I got an import error with distributed module. I'm not sure this is a bug from distributed side, so here I am reporting the issue but quite asking for advice on how to handle this. There are obviously two different snappy module, and it seems to me the one imported from distributed is the one merely installed from a `pip install` instruction. Since snappy does seem to be mandatory for distributed could there be a workaround ? Maybe just an `try ... catch` around the snappy import ?
Thank you for your help
**What I expected to happen**:
I actually did not know about the snappy import from distributed, so I expected nothing to prevent me from using the snappy module from https://pypi.org/project/snappy/. But it looks like distributed import a snappy module if it can (if installed) to use its compression methods. This leads to thinking the snappy module distributed can rely on is the python-snappy project https://pypi.org/project/python-snappy/.
**Minimal Complete Verifiable Example**:
Everything's fine with no snappy module
```
root@0a8a5ef7b540:/home/ai4geo# python
Python 3.6.9 (default, Apr 18 2020, 01:56:04)
[GCC 8.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import snappy
Traceback (most recent call last):
File "", line 1, in
ModuleNotFoundError: No module named 'snappy'
>>> import distributed
>>>
```
After having installed snappy I no longer can import distributed
```
root@0a8a5ef7b540:/home/ai4geo# pip install snappy
root@0a8a5ef7b540:/home/ai4geo# python
Python 3.6.9 (default, Apr 18 2020, 01:56:04)
[GCC 8.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import distributed
Plink failed to import tkinter.
Traceback (most recent call last):
File "", line 1, in
File "/usr/local/lib/python3.6/dist-packages/distributed/__init__.py", line 3, in
from .actor import Actor, ActorFuture
File "/usr/local/lib/python3.6/dist-packages/distributed/actor.py", line 6, in
from .client import Future, default_client
File "/usr/local/lib/python3.6/dist-packages/distributed/client.py", line 42, in
from .batched import BatchedSend
File "/usr/local/lib/python3.6/dist-packages/distributed/batched.py", line 8, in
from .core import CommClosedError
File "/usr/local/lib/python3.6/dist-packages/distributed/core.py", line 17, in
from .comm import (
File "/usr/local/lib/python3.6/dist-packages/distributed/comm/__init__.py", line 25, in
_register_transports()
File "/usr/local/lib/python3.6/dist-packages/distributed/comm/__init__.py", line 16, in _register_transports
from . import inproc
File "/usr/local/lib/python3.6/dist-packages/distributed/comm/inproc.py", line 13, in
from ..protocol import nested_deserialize
File "/usr/local/lib/python3.6/dist-packages/distributed/protocol/__init__.py", line 4, in
from .compression import compressions, default_compression
File "/usr/local/lib/python3.6/dist-packages/distributed/protocol/compression.py", line 51, in
"compress": snappy.compress,
AttributeError: module 'snappy' has no attribute 'compress'
```
**Environment**:
- Distributed version: 2.14.0
- Python version: 3.6.9
- Operating System: ubuntu 18.04 bionic
- Install method (conda, pip, source): pip
Contributor guide
Assessment
This issue has not been assessed yet.