PostHog / PostHog/posthog

LLM analytics: support custom base URLs for OpenAI-compatible APIs

Open
#54,854 6 comments 4 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement
Dominant language
Python
Stars
39.9k
Forks
3.4k
Avg merge
6h 51m
Merged PRs (30d)
232

Description

Feature request

Is your feature request related to a problem?

When using LLM analytics features (evals, playground, "bring your own API key"),
the tooling only allows specified providers like OpenAI, Anthropic, OpenRouter etc.

We use Scaleway for both infrastructure and LLM inference. Their API is
OpenAI-compatible, but there's no way to configure PostHog to use our Scaleway
endpoint instead of openai.com.

Describe the solution you'd like

Allow users to configure a custom base URL alongside their API key in the LLM
analytics settings. This should apply to:

  • Evals
  • Playground
  • Any "bring your own key" configuration

We can use different models via Scaleway, not only the open source OpenAI models, so how to pick the model is a bit unsure, if we don't want to just type the name.

Describe alternatives you've considered

I don't really know if there is a simple alternative.

Additional context

Many teams self-host models or use regional/private inference providers for cost,
latency, data residency, or compliance reasons. Any provider that implements the
OpenAI API spec (Scaleway, Azure OpenAI, Ollama, etc.) would benefit from this.
Supporting a configurable base URL would make PostHog's LLM tooling usable for
these teams without any changes to the API contract.

Debug info
We use the EU cloud version of PostHog.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

No files, tests, or entry points are named. Start by locating the LLM analytics settings and the Evals, Playground, and bring-your-own-key flows, then trace how providers, API keys, and models are selected. Done means users can configure a custom base URL and choose a model for OpenAI-compatible endpoints across the requested flows.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
ai, api
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
52/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.