PostHog / PostHog/ai-plugin

Skills reference bundled files that are not shipped with them (v1.1.56)

Open
#181 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
84
Forks
15
Avg merge
14h 34m
Merged PRs (30d)
7

Description

Several skills in the plugin point at references/*.md or docs/*.md paths that do not exist inside their own skill directory. Progressive disclosure breaks at that point — the body tells the model to read a file that is not there.

The plugin's own working-with-skills skill lists this exact failure as a quality gate:

Link hygiene — every relative link in the body (references/x.md, ./y.md) exists in the files manifest; every manifest file is reachable from the body. Dead links break progressive disclosure.

Version: 1.1.56 (installed from claude-plugins-official, confirmed latest via claude plugin update posthog@claude-plugins-official)

What is missing

Two different shapes. Three files exist in some skills but were not copied into others that reference them; two are not present anywhere in the plugin.

references/report-contract.md — missing in 11 skills

Present in: authoring-scouts, signals-scout-anomaly-detection

Missing in: exploring-scouts, signals-scout-apm, signals-scout-csp-violations, signals-scout-customer-analytics, signals-scout-error-tracking, signals-scout-health-checks, signals-scout-inbox-validation, signals-scout-insight-alerts, signals-scout-observability-gaps, signals-scout-replay-vision, signals-scout-session-replay

The eleven are the ones that most need it — they are the scouts whose output is supposed to follow the report contract.

references/dedupe-and-memory.md — missing in 1 skill

Present in authoring-scouts, referenced by exploring-scouts.

references/changing-distribution-after-launch.md — missing in 1 skill

Present in configuring-experiment-rollout, referenced by diagnosing-experiment-results.

references/primer.md — not present anywhere

Referenced by skills-store and working-with-skills. Both have an empty (or absent) references/ directory.

docs/support-mcp-tools.md — not present anywhere

Referenced by testing-mcp-tools-locally.

How to reproduce

From the installed plugin root:

cd ~/.claude/plugins/cache/claude-plugins-official/posthog/1.1.56/skills
for d in */; do
  grep -oE '(references|docs)/[A-Za-z0-9_./-]+\.md' "$d/SKILL.md" 2>/dev/null | sort -u | while read -r ref; do
    [ -e "$d/$ref" ] || echo "$d -> $ref"
  done
done

Not included in the counts above

Two paths that a naive scan flags are legitimate prose examples, not defects — they are excluded here deliberately:

  • references/x.md in working-with-skills, from the link-hygiene rule quoted at the top
  • references/rollback.md in signals-scout-skills-store, inside a sample memory entry describing another skill's dead link

Impact

15 skills are affected. Nothing crashes; the model simply cannot open the file it was told to read, and the guidance those files carry — the report contract in particular — silently does not reach it.

Found while auditing skill references across all installed plugins. Happy to send a PR that copies the three shared files into the skills that reference them, if that matches how you would rather fix it.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in the installed plugin’s skills directory and run the provided shell scan to reproduce missing references. Inspect the affected SKILL.md files, their files manifests, and the existing shared files in authoring-scouts and configuring-experiment-rollout. Done means every non-example relative reference resolves within its skill directory, every referenced file is packaged, and the two absent targets have an agreed source or replacement.

Written by the indexing model from the issue text.

Assessment

Tech stack
markdown, shell
Domain
documentation, tooling
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.