Lightning-AI / Lightning-AI/LitServe
`fast_queue=True` is silently ignored — the ZMQ transport is never enabled (and would be misconfigured if it were)
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 3.9k
- Forks
- 304
- Avg merge
- 3d 13h
- Merged PRs (30d)
- 6
Description
## Bug description
`LitServer(..., fast_queue=True)` is documented to switch the request/response
transport to ZeroMQ for high-throughput serving. It currently has no effect:
1. `LitServer.__init__` builds the transport configuration with the wrong
keyword name:
```python
# src/litserve/server.py
self.transport_config = TransportConfig(transport_config="zmq" if self.use_zmq else "mp")
```
`TransportConfig` has a field named `transport_type`, not
`transport_config`. Pydantic ignores the unknown keyword, so
`transport_type` always stays at its default `"mp"` — `fast_queue=True`
silently falls back to the multiprocessing-queue transport.
2. Even when `transport_type="zmq"` is set directly, the ZMQ transport is
wired with swapped addresses:
```python
# src/litserve/transport/factory.py
return ZMQTransport(config.frontend_address, config.backend_address)
```
`ZMQTransport.__init__(self, backend_address, frontend_address)` receives
the frontend address as `backend_address` and vice versa. Workers then
publish responses to the broker's frontend (XPUB) socket while consumers
subscribe on the backend (XSUB) socket, so responses are never delivered
and every request hangs until it times out.
## Reproduction
```python
from litserve.server import LitServer
from litserve.test_examples import SimpleLitAPI # or any LitAPI
server = LitServer(SimpleLitAPI(), fast_queue=True)
print(server.transport_config.transport_type) # expected: "zmq", actual: "mp"
```
And the address swap can be shown with:
```python
from litserve.transport.factory import TransportConfig, create_transport_from_config
config = TransportConfig(transport_type="zmq", num_consumers=1)
transport = create_transport_from_config(config)
assert transport.backend_address == config.backend_address # fails: swapped
```
## Expected behavior
- `fast_queue=True` should produce `transport_config.transport_type == "zmq"`.
- The produced `ZMQTransport` should connect producers to the broker backend
and consumers to the broker frontend, so responses actually reach the
requester.
## Environment
- LitServe: current `main` (commit 1624847)
- Python 3.12, Linux
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in src/litserve/server.py and src/litserve/transport/factory.py, then inspect TransportConfig and ZMQTransport to verify the configured transport type and address ordering. Run the issue's LitServer and create_transport_from_config reproductions; done means fast_queue=True selects zmq and the produced transport preserves the configured backend and frontend addresses.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend, distributed-systems
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 76/100