PostHog / PostHog/ai-plugin

Honor a single umbrella opt-in (or documented alias) for LLMA Claude Code capture

Open
#150 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
84
Forks
15
Avg merge
14h 34m
Merged PRs (30d)
7

Description

Overview

Downstream tooling that layers its own analytics on top of this plugin's LLMA Claude Code capture currently has to ask operators to set two independent opt-in flags that mean, in practice, one decision. This request asks PostHog to let a single opt-in — or a documented, stable alias — enable the Claude Code capture, so downstream layers don't have to couple to POSTHOG_LLMA_CC_ENABLED by name and operators manage one switch instead of two.

Current behavior

The plugin gates its SessionEnd LLMA capture on POSTHOG_LLMA_CC_ENABLED=true (plus POSTHOG_API_KEY) — posthog_llma/config.py reads it directly. For the plugin on its own, that explicit, opt-in-by-default design is exactly right.

The friction appears one layer up. flow-skills, a Claude Code plugin, adds a second PostHog observability layer — session-friction $ai_evaluation events — that is only meaningful when this plugin's $ai_span capture is also on. Its own opt-in is USE_POSTHOG_AI_OBSERVABILITY=true. An operator who wants the full picture must set both flags, and setting only one is a silent, easy-to-miss misconfiguration: Layer 2 on / Layer 1 off produces friction events with no spans to anchor them to. flow-skills currently compensates by (a) pairing the two flags in its operator setup snippet and (b) emitting a fail-open drift warning when they disagree — but it cannot remove the coupling, because it does not own POSTHOG_LLMA_CC_ENABLED.

Proposal (any one of these resolves it)

  1. Honor an umbrella alias — read a documented synonym (e.g. POSTHOG_AI_OBSERVABILITY) as equivalent to POSTHOG_LLMA_CC_ENABLED, so a single operator-facing flag enables both this plugin and any downstream layer standardizing on that name; or
  2. Bless a stable opt-in name downstream tools can standardize on as "the PostHog AI-observability opt-in," so each ecosystem building on the plugin doesn't invent its own flag and force operators to keep two in sync.

Either way the goal is one switch, not N, for operators who run this plugin plus analytics layered on top of it.

Why it matters

The two-flag coupling is a recurring configuration footgun for anyone building on top of the plugin, and the failure mode (one flag set, the other not) is silent. A single documented opt-in lets downstream layers drop their compensating warnings and lets operators reason about one setting rather than a pair they must keep aligned by hand.


Filed on behalf of the flow-skills Claude Code plugin project (tracked there as the follow-up to flow-skills#447). Happy to open a PR if a specific alias name works for you.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reading posthog_llma/config.py, especially how the SessionEnd capture checks POSTHOG_LLMA_CC_ENABLED alongside POSTHOG_API_KEY. Review the surrounding configuration behavior, then clarify whether maintainers will accept an umbrella alias or bless a stable opt-in name. Done means one documented operator-facing switch can enable the intended capture without requiring downstream tools to couple to the existing variable name.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
cli, devtools
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.