pathwaycom / pathwaycom/serviette
Bug: `type: litellm` LLM never routed through LiteLLM
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 5
- Forks
- 1
- PR merge metrics
- No merged PRs in 30d
Description
Note : Below is a AI generated human reviewed bug report for a human discovered bug, I encountered the bug and had to fix it, I asked IA to document it.
If the quality is bad feel free to say so (I'll learn), or reject.
I use the fix, it is proven to work, the code changes are sound and reviewed. We can discuss about the asserts in the code (and remove them) but the bug is real and the fix make the code work according to the documentation.
I open the pull request right after pushing the issue.
Symptom
Using the webui LLM dialog (/api/v1/rag) with a litellm-typed config:
llm:
type: litellm
model: "openrouter/moonshotai/kimi-k3"
api_key: "sk-or-..."
the request went to api.openai.com with the OpenRouter key and failed:
openai.AuthenticationError: Error code: 401 - Incorrect API key provided ...
at serviette/server/main.py:243 -> llm.complete(...)
at serviette/server/llm.py:122 -> client.chat.completions.create(...)
LiteLLM was never initialized on this path.
Root cause
build_llm in packages/serviette/serviette/server/llm.py treated litellm
as OpenAI-compatible:
_OPENAI_COMPATIBLE = {"openai", "litellm"}
def build_llm(config):
...
if config.type in _OPENAI_COMPATIBLE:
return OpenAIChat(config)
OpenAIChat._ensure_client builds AsyncOpenAI(api_key=..., **extra). With
this config extra is empty, so base_url defaults to
https://api.openai.com. The provider prefix in model
(openrouter/...) was passed verbatim to OpenAI, which knows neither the
model nor the key → 401.
The config itself is valid — the quickstart wizard
(quickstart/wizard.py:607) offers litellm as an LLM type, so this was a
server-side misrouting, not a config error.
A second, latent instance of the same bug existed in
server/reranker.py: LLMReranker._ensure_chat constructed OpenAIChat
directly, ignoring a litellm type inherited from the top-level llm
section.
Fix
server/llm.py— added aLiteLLMChatbackend that calls
litellm.acompletion(model=..., api_key=..., messages=...), so the
provider prefix inmodelselects the endpoint (OpenRouter here) and the
key is forwarded to that provider.build_llmnow dispatches:
mock→MockLLM,litellm→LiteLLMChat,openai→OpenAIChat.
Theopenaipath is unchanged.server/reranker.py—_ensure_chatnow usesbuild_llm(...)
instead of hardcodingOpenAIChat, so an LLM reranker honors the
configured type.tests/test_server.py— regression test
test_litellm_routes_through_litellm_not_openai_client: drives
/api/v1/ragwith alitellmLLM, patcheslitellm.acompletion, and
asserts the outgoing call carries the provider-prefixed model and the
API key, with nobase_url(i.e. no OpenAI client involved).
Verification
- 43 tests pass (
test_server.py,test_rag_quality.py,
test_frontend.py,test_qdrant_hybrid.py), including the new
regression test. - Manual check:
build_llm(LLMConfig(type="litellm", ...))returns
LiteLLMChat; mockedlitellm.acompletionreceives
model="openrouter/moonshotai/kimi-k3"and the configuredapi_key.
Note: embedder.py intentionally keeps its own
_OPENAI_COMPATIBLE = {"openai", "litellm"} — server-side embedders
genuinely map litellm onto the OpenAI-compatible async client; that path
is unrelated and untouched.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with packages/serviette/serviette/server/llm.py and the existing build_llm dispatch, then inspect server/reranker.py for its chat construction. Run tests/test_server.py, especially test_litellm_routes_through_litellm_not_openai_client. Done means the litellm configuration uses the intended provider path in both server flows while the existing OpenAI path remains unchanged.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- api, backend, testing
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Clearly specified
- Newbie friendliness
- 25/100