dask / dask/dask-yarn

Issue with deprecated format_bytes

Open
#161 3 comments 1 reaction 0 assignees View on GitHub
Dominant language
Python
Stars
69
Forks
40
PR merge metrics
No merged PRs in 30d

Description

I setup a new virtual environment and loaded Dask/Dask-Yarn and cannot import due to a deprecated package.
I searched for Dask-Yarn and format_bytes and only found the deprecation warning in older versions.
I tried both python3.9 and python3.11, I'll include steps for the python3.9 test.
I'm not sure what to do about this. Should I pin Dask and Dask-Yarn to specific versions?

Producing the issue:
```bash
python -c "from dask_yarn import YarnCluster"
Traceback (most recent call last):
File "", line 1, in
File ".../venv_py39_dask_yarn/lib/python3.9/site-packages/dask_yarn/__init__.py", line 6, in
from .core import YarnCluster
File ".../venv_py39_dask_yarn/lib/python3.9/site-packages/dask_yarn/core.py", line 16, in
from distributed.utils import (
ImportError: cannot import name 'format_bytes' from 'distributed.utils' (.../venv_py39_dask_yarn/lib/python3.9/site-packages/distributed/utils.py)
```

Possible fix is to import from `dask.utils` in the `dask_yarn/core.py` source instead:
```python
from distributed.utils import (
# format_bytes,
log_errors,
LoopRunner,
format_dashboard_link,
# parse_timedelta,
Log,
Logs,
)
from dask.utils import (
format_bytes,
parse_timedelta,
)
```
Setup Steps:
```bash
python3.9 -mvenv venv_py39_dask_yarn
cd venv_py39_dask_yarn
source bin/activate
pip install --upgrade pip
pip install dask[complete]
pip install dask-yarn
```

pip freeze:
```bash
$ pip freeze
bokeh==3.3.4
cffi==1.16.0
click==8.1.7
cloudpickle==3.0.0
contourpy==1.2.0
cryptography==42.0.5
dask==2024.2.1
dask-yarn==0.9
distributed==2024.2.1
fsspec==2024.2.0
grpcio==1.62.0
importlib-metadata==7.0.1
Jinja2==3.1.3
locket==1.0.0
lz4==4.3.3
MarkupSafe==2.1.5
msgpack==1.0.8
numpy==1.26.4
packaging==23.2
pandas==2.2.1
partd==1.4.1
pillow==10.2.0
protobuf==4.25.3
psutil==5.9.8
pyarrow==15.0.0
pyarrow-hotfix==0.6
pycparser==2.21
python-dateutil==2.9.0
pytz==2024.1
PyYAML==6.0.1
six==1.16.0
skein==0.8.2
sortedcontainers==2.4.0
tblib==3.0.0
toolz==0.12.1
tornado==6.4
tzdata==2024.1
urllib3==2.2.1
xyzservices==2023.10.1
zict==3.0.0
zipp==3.17.0
```

Python version verification
```bash
$ which python
.../venv_py39_dask_yarn/bin/python
$ python -V
Python 3.9.18
$ pip -V
pip 24.0 from .../venv_py39_dask_yarn/lib/python3.9/site-packages/pip (python 3.9)
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.