dask / dask/distributed

`Client.compute(df)` operates on larger task graphs than `df.compute()`

Open
#8,833 2 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
Python
Stars
1.7k
Forks
778
Avg merge
2h 50m
Merged PRs (30d)
3

Description

**Describe the issue**:

**Minimal Complete Verifiable Example**:

```python
from dask.distributed import Client
import dask

if __name__ == "__main__":
client = Client()

task_counts = []

def capture_task_counts(event):
_, msg = event
if not isinstance(msg, dict):
return
if msg.get("action", None) != "update_graph":
return
task_counts.append(msg["count"])
client.subscribe_topic("all", capture_task_counts)

df = dask.datasets.timeseries(
start="2000-01-01",
end="2001-01-01",
dtypes={"x": float, "y": float},
freq="10 s",
)
total = df.x.sum()

total.compute()
client.compute(total, sync=True)
assert task_counts[0] == task_counts[1], task_counts
```

raises `AssertionError: [367, 1466]`.

From what it looks like, `client.compute(...)` does not perform (all) task fusion.

**Anything else we need to know?**:

**Environment**:

- Dask version:
- Python version:
- Operating System:
- Install method (conda, pip, source):

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.