dask / dask/distributed

Don't import packages just for version compatibility check

Open
#5,723 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
1.7k
Forks
778
Avg merge
2h 50m
Merged PRs (30d)
3

Description

In https://github.com/dask/distributed/pull/5695#discussion_r794084335, we discovered that pandas was always being imported on Nanny workers because, to check version compatibility, all these modules are imported: https://github.com/dask/distributed/blob/9a4e0e2f097aa97a373fdd9b7c76f8d79ec746a9/distributed/versions.py#L15-L29

Instead of importing, we could use [`importlib.metadata.version`](https://docs.python.org/3/library/importlib.metadata.html#version) to get the distribution's version number without importing the package. This could speed up worker startup time and reduce memory footprint a little.

cc @crusaderky

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.