PostHog / PostHog/posthog

Feature request: prompt management - fetch all prompts at a label in one request

Open
#95,460 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement feature/llm-analytics team/ai-observability
Dominant language
Python
Stars
39.9k
Forks
3.4k
Avg merge
6h 51m
Merged PRs (30d)
232

Description

Is your feature request related to a problem?

An app that resolves prompts through a label at runtime has to make one request per prompt. GET /api/projects/:project_id/llm_prompts/name/:prompt_name/?label=production resolves a single prompt at the version its label points to. The collection endpoint, GET /api/projects/:project_id/llm_prompts/, returns every prompt in one request but ignores labels entirely and always serves each prompt's latest version. A team running a few dozen prompts released through a production label therefore has no way to load them in fewer than N requests, and at that fan-out they meet the documented API rate limit of 480 requests per minute and 4,800 per hour.

Passing the parameter anyway fails silently. LLMPromptListQuerySerializer (posthog/api/llm_prompt_serializers.py:167) accepts only search, created_by_id, order_by, and content, and DRF drops unrecognized query params rather than rejecting them. So ?label=production returns a 200 with latest versions, and nothing in the response says the label was ignored.

Describe the solution you'd like

Accept label on the collection endpoint, so that GET /api/projects/:project_id/llm_prompts/?label=production&content=full returns each prompt at the version that label currently points to.

Prompts that do not carry the label should be omitted, matching the 404 the per-name endpoint returns when a prompt has no such label. Falling back to the latest version instead would reproduce the silent-latest behavior described above, which is the failure the label exists to prevent.

Describe alternatives you've considered

Client-side resolution using all_labels. Each collection row already carries all_labels, a list of name and version pairs covering labels on any version. A caller can compare the wanted label's version against the row's own version and re-fetch only the prompts where the two disagree. This is the workaround available today, but it makes every caller reimplement label resolution, and it carries the same silent failure: a prompt with no such label shows no disagreement, so a naive sweep serves its latest version where ?label= would have returned a 404.

Caching harder. The SDKs cache each prompt for 300 seconds, so one long-lived client settles at N fetches per TTL. That helps a single process, but the floor still scales with process count for anyone running many independent backends without a shared cache, and it does not reduce the per-process request count at all.

Batch exports. Not applicable. Batch exports carry the events, persons, sessions, and hogql models, and prompts are not among them.

Additional context

From: https://us.posthog.com/project/2/support/tickets/70844

How it got here

Labels landed in #70862, fetch-by-label followed in #71509, and #72804 taught the prompt page about them. Each of those changed the per-name path or the UI. The collection endpoint was never brought along, so it is now the only prompt read path that cannot resolve a label.

Implementation notes
  • list builds on get_latest_prompts_queryset() (posthog/api/services/llm_prompt.py:112), which filters is_latest=True. A label filter cannot be a WHERE clause on top of that, since the whole point is returning versions that are not latest. It has to start from the label rows instead.
  • LLMPromptLabel.prompt is a direct FK to a version row, and _get_labeled_prompt_from_db (posthog/storage/llm_prompt_cache.py:123) already does this resolution for one name via select_related("prompt"). The bulk form is that query without the name filter, so it stays a single query.
  • _track_prompt_fetch (posthog/api/llm_prompt.py:193) fires only from get_by_name, so list emits no $llm_prompt_fetched today. A labeled collection fetch should emit one event per prompt returned, or usage counts drop discontinuously for anyone who migrates, including the AI observability usage report, which counts those events directly.
  • get_by_name reads through the prompt hypercache while list goes to Postgres. If this becomes an SDK-shaped hot path, it should read through the cache tier too.
  • The endpoint alone only helps callers willing to hand-roll HTTP next to prompts.get(). SDK support, in the shape of the feature flag poller that warms a whole cache in one call, is what would make it generally useful.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with LLMPromptListQuerySerializer in posthog/api/llm_prompt_serializers.py and the list path in posthog/api/llm_prompt.py. Read get_latest_prompts_queryset() and _get_labeled_prompt_from_db in posthog/api/services/llm_prompt.py and posthog/storage/llm_prompt_cache.py to understand the existing query and label resolution. Done means the collection endpoint resolves the requested label, omits unlabeled prompts, and tracks returned prompts as described.

Written by the indexing model from the issue text.

Assessment

Tech stack
postgresql, python
Domain
api, backend, databases
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.