Re-submitting tasks can creates `Computation`s with no `stop`
- Dominant language
- Python
- Stars
- 1.7k
- Forks
- 778
- Avg merge
- 2h 50m
- Merged PRs (30d)
- 3
Description
`Computation`s are meant to track, high-level, the operations that ran for a given `compute`/`persist` call. They're currently just a container referencing multiple `TaskGroup`s. But the same `TaskGroup` could be used in multiple, separate `compute` calls. For example:
```python
a = da.ones(1000)
h = a[:50].persist()
wait(h)
a = a.persist()
```
In this case, there will be two `Computation`s for the two `persist` calls. However, the second `persist` call re-uses the same `TaskGroup`s as the first. So its `groups` will be empty. Since the `stop` time is defined as when the last group stopped, the second computation will have no stop.
A test:
```python
@gen_cluster(client=True)
async def test_computations_some_resubmitted_tasks(c, s, a, b):
futures = c.map(inc, list(range(4)))
await wait(futures)
futures2 = c.map(inc, list(range(6))) # NOTE: 4 tasks reused, 2 new
await wait(futures2)
assert len(s.computations) == 2
assert s.computations[0].stop > s.computations[0].start
assert s.computations[1].stop > s.computations[1].start
assert s.computations[1].start > s.computations[0].stop
```
Contributor guide
Assessment
This issue has not been assessed yet.