dask / dask/distributed

Clear `log_event` print buffer.

Open
#5,776 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
1.7k
Forks
778
Avg merge
2h 50m
Merged PRs (30d)
3

Description

When using `log_event` with `print` topic multiple times, the previous events are not cleared even the client is renewed.

**Minimal Complete Verifiable Example**:

```python
from distributed import get_client, LocalCluster, Client

def test(i: int):
client = get_client()
client.log_event("print", f"message-{i}")

def main(client):
for i in range(2):
print(f"iter {i}")
fut = client.submit(test, i)
fut.result()

if __name__ == "__main__":
with Client(scheduler_file="sched.json") as client:
main(client)
```

Save the above snippet into a file called `test.py`, start a cluster with 2 workers using CLI with the scheduler file named `sched.json`, then run the script multiple times. The outputs are duplicated for each successive run:

```
$ python ./test.py
iter 0
message-0
message-0
iter 1
message-1
message-1
message-1
$ python ./test.py
iter 0
message-0
message-0
message-0
message-0
iter 1
message-1
message-1
message-1
message-1
$ python ./test.py
iter 0
message-0
message-0
message-0
message-0
message-0
iter 1
message-1
message-1
message-1
message-1
message-1
```
The expected result is that it should print the same output for each run.

**Environment**:

- Dask version: 2022.01.1
- Python version: Python 3.8.10
- Operating System: Ubuntu 20.04
- Install method (conda, pip, source): pip

A non-related question, I looked into `log_event` for printing model training results for each iteration on the client side, is that a good choice in terms of performance?

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.