bigscience-workshop / bigscience-workshop/petals

Error while hosting as provider

Open
#533 1 comment 1 reaction 0 assignees View on GitHub
Dominant language
Python
Stars
10.6k
Forks
650
PR merge metrics
No merged PRs in 30d

Description

### Description

```txt
Oct 31 07:53:33.251 [ERROR] [hivemind.moe.server.runtime.run:101] Caught 3736, attempting to recover
Traceback (most recent call last):
File "/opt/conda/lib/python3.10/site-packages/hivemind/moe/server/runtime.py", line 91, in run
outputs, batch_size = self.process_batch(pool, batch_index, *batch)
File "/opt/conda/lib/python3.10/site-packages/hivemind/moe/server/runtime.py", line 110, in process_batch
outputs = pool.process_func(*batch)
File "/opt/conda/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
File "/home/petals/src/petals/server/backend.py", line 232, in __call__
(hidden_states,) = self.backends[inference_info.uid].inference_step(hidden_states, hypo_ids, inference_info)
File "/opt/conda/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
File "/home/petals/src/petals/server/backend.py", line 119, in inference_step
with self.memory_cache.use_cache(
File "/opt/conda/lib/python3.10/contextlib.py", line 135, in __enter__
return next(self.gen)
File "/home/petals/src/petals/server/memory_cache.py", line 221, in use_cache
yield tuple(self._allocated_tensors[handle] for handle in handles)
File "/home/petals/src/petals/server/memory_cache.py", line 221, in
yield tuple(self._allocated_tensors[handle] for handle in handles)
KeyError: 3736
```

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.