LLM analytics: support custom base URLs for OpenAI-compatible APIs
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 39.9k
- Forks
- 3.4k
- Avg merge
- 6h 51m
- Merged PRs (30d)
- 232
Description
Feature request
Is your feature request related to a problem?
When using LLM analytics features (evals, playground, "bring your own API key"),
the tooling only allows specified providers like OpenAI, Anthropic, OpenRouter etc.
We use Scaleway for both infrastructure and LLM inference. Their API is
OpenAI-compatible, but there's no way to configure PostHog to use our Scaleway
endpoint instead of openai.com.
Describe the solution you'd like
Allow users to configure a custom base URL alongside their API key in the LLM
analytics settings. This should apply to:
- Evals
- Playground
- Any "bring your own key" configuration
We can use different models via Scaleway, not only the open source OpenAI models, so how to pick the model is a bit unsure, if we don't want to just type the name.
Describe alternatives you've considered
I don't really know if there is a simple alternative.
Additional context
Many teams self-host models or use regional/private inference providers for cost,
latency, data residency, or compliance reasons. Any provider that implements the
OpenAI API spec (Scaleway, Azure OpenAI, Ollama, etc.) would benefit from this.
Supporting a configurable base URL would make PostHog's LLM tooling usable for
these teams without any changes to the API contract.
Debug info
We use the EU cloud version of PostHog.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
No files, tests, or entry points are named. Start by locating the LLM analytics settings and the Evals, Playground, and bring-your-own-key flows, then trace how providers, API keys, and models are selected. Done means users can configure a custom base URL and choose a model for OpenAI-compatible endpoints across the requested flows.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- ai, api
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 52/100