ipython / ipython/ipyparallel

Outstanding task on client but hub says completed when using broadcast view

Offen
#882 0 Kommentare 0 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen
bug
Vorherrschende Sprache
Jupyter Notebook
Sterne
2.6k
Forks
1k
PR-Merge-Kennzahlen
Keine gemergten PRs in 30 T.

Beschreibung

I want to `purge_everything ` or `purge_local_results ` but I'm getting `RuntimeError: Can't purge outstanding tasks: ...`
The reason I want to purge is that after a couple of tasks my engines memory seems to get full and slows down the process. I'm trying to figure out if it's a memory issue related to ipyparallel or something related to the tasks.

Here is a rough sketch of what I'm doing:

do some setup:
```
from ipyparallel import Client, Reference
rc = Client(client_args, client_kwargs)
view = rc.broadcast_view(view_args, view_kwargs)
view.is_coalescing = False
view.apply_sync( # do some setup stuff )
```
then:
```
ar = view.apply( # do the real work, and pickle it)
fvs = np.array(sum([pickle.loads(fv) for fv in ar.get()],[]))

view.client.purge_local_results('all') # this says there are outstanding tasks
view.client.queue_status() # this says all tasks are complete
```

Beitragsleitfaden

Beitragsleitfaden öffnen

Bewertung

Dieses Issue wurde noch nicht bewertet.

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.