michaelfeil / michaelfeil/infinity

get asyncio.exceptions.CancelledError error

Open
#317 4 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
2.9k
Forks
206
PR merge metrics
No merged PRs in 30d

Description

### System Info

Using CPU and kubernetes
CMD: infinity_emb v1 --model-name-or-path /mnt/model-storage/ms-marco-MiniLM-L-12-v2 --port 8452 --batch-size 32

### Information

- [ ] Docker
- [X] The CLI directly via pip

### Tasks

- [X] An officially supported command
- [ ] My own modifications

### Reproduction

Run the command on local computer works, but run on a kubernetes shows the following error
INFO: Application startup complete.
INFO: Uvicorn running on http://0.0.0.0:8452 (Press CTRL+C to quit)
ERROR: Traceback (most recent call last):
File "/usr/local/lib/python3.11/contextlib.py", line 222, in __aexit__
await self.gen.athrow(typ, value, traceback)
File "/usr/local/lib/python3.11/site-packages/infinity_emb/infinity_server.py", line 82, in lifespan
yield
File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 741, in lifespan
await receive()
File "/usr/local/lib/python3.11/site-packages/uvicorn/lifespan/on.py", line 137, in receive
return await self.receive_queue.get()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/asyncio/queues.py", line 158, in get
await getter
asyncio.exceptions.CancelledError

### Expected behavior

I see INFO: Uvicorn running on http://0.0.0.0:8452 (Press CTRL+C to quit)
so the server seems starts correctly? But the asyncio.exceptions.CancelledError cause the pod to crash immediately

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the provided infinity_emb command in Kubernetes and inspect infinity_emb/infinity_server.py around the lifespan function at line 82, comparing it with the local run. Done means the supported command keeps its pod running in Kubernetes without the reported CancelledError crash, with the observed behavior verified against the startup logs.

Written by the indexing model from the issue text.

Assessment

Tech stack
kubernetes, python
Domain
backend, infrastructure
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.