PostHog / PostHog/posthog-python

Add ai_trace_sample_rate to sample AI observability events by trace id

Open
#965 0 comments 0 reactions 1 assignee View on GitHub

@jakesciotto is already working on this.

Since Sep 16, 2026.

enhancement team/llm-analytics
Dominant language
Python
Stars
61
Forks
82
Avg merge
1d 15h
Merged PRs (30d)
71

Description

Problem Statement

Teams whose AI observability volume is mostly $ai_* events have no supported way to keep a fraction of traces. The only path today is a hand-written before_send hook that hashes $ai_trace_id. Most teams do not write one. A hook that samples per event splits a trace, so the trace view shows partial trees.

Related: PostHog/posthog-js#4997 for the same option in posthog-node.

Solution Brainstorm

One client option, default off:

Client(api_key, ai_trace_sample_rate=0.1)

Env var POSTHOG_AI_TRACE_SAMPLE_RATE, same precedence rule as POSTHOG_CAPTURE_MODE (kwarg > env > default).

Behavior:

  1. Applies to events named $ai_* that carry $ai_trace_id.
  2. FNV-1a 32-bit hash of the trace id. Keep when hash / 2^32 < rate. Same hash in every SDK, so a mixed Python and Node stack makes one decision per trace.
  3. Runs in Client._enqueue before before_send, so a customer hook sees only kept events.
  4. Adds $ai_trace_sample_rate to kept events, so the server can scale totals later.
  5. Events with no trace id are kept. Rate unset or 1.0 means no sampling. Out-of-range values log a warning and are ignored.
flowchart LR
  W[AI wrappers and agent processors] --> C[_capture_ai_event]
  M[Manual capture of $ai_* events] --> E[Client._enqueue]
  C --> E
  E --> S{ai_trace_sample_rate}
  S -- drop --> X[Dropped]
  S -- keep --> B[before_send] --> Q[Queue]

Alternatives considered: a helper that returns a before_send function. Rejected because before_send takes one callable, so the helper does not compose with an existing hook.

Impact

  • Gives AI observability users a supported volume lever that keeps traces whole.
  • Removes the need for a custom hook that is easy to get wrong.

Blast radius

  • Default off. No behavior change for any existing caller.
  • Touches Client.__init__ and Client._enqueue only. Wrappers do not change.
  • Affects $ai_* events only. Other events never reach the sampler.
  • Kept $ai_* events gain one property when the option is set.
  • Public API grows by one option and one env var. references/public_api_snapshot.txt changes.

Not in this issue

Tail sampling that keeps errored traces. A per-call override. Server-side upweighting of cost dashboards.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.