pymc-labs / pymc-labs/CausalPy
Improve agent discoverability of user-facing skills, and review graded-treatment routing in choosing-causalpy-methods
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 1.2k
- Forks
- 115
- Avg merge
- 6d 1h
- Merged PRs (30d)
- 11
Description
Summary
Two related gaps surfaced while an AI agent was authoring a new example notebook and had to choose a CausalPy method. Both are about the skills in causalpy/skills/ doing their job at the moment an agent needs them.
- Discovery. User-facing skills live in
causalpy/skills/(shipped to end users via pip / Decision AI Hub). Agents that rely on auto-discovery only scan.agents/skills/and its tool-specific twins, so they never seecausalpy/skills/. The only bridge is a prose "Skills Location" note partway downAGENTS.md, which is easy to skip. - Content.
choosing-causalpy-methodsappears to route "continuous / graded / dose-response treatment" to not a fit (reference/not_in_causalpy.md), but a graded intervention can often be represented in Piecewise ITS viastep()/ramp()terms. The no-fit rule looks too strict and can bounce work out of CausalPy that it can actually handle.
Motivating story
An agent set out to build an example around a graded, event-style treatment. It never found choosing-causalpy-methods, reasoned from general knowledge instead, concluded CausalPy had no quasi-experimental fit for a graded treatment, and nearly redirected the whole example to a different library. A maintainer intervened and pointed at Piecewise ITS, which is exactly the right tool and is described in the shipped skill. So the correct answer was in the repo the entire time; the agent just never reached it, and the one heuristic it might have leaned on would have pushed it the wrong way. Both failure modes are cheap to fix and will recur for any agent (or contributor) doing method selection.
Recommendation 1 — discoverability (keep files where they are)
The split is correct and should stay: user skills ship to end users; developer skills are for maintainers. This is a discovery problem, not a location problem.
- Add a loud, imperative pointer near the top of
AGENTS.md(andCLAUDE.mdif present): "Before answering any causal-inference or method-choice question, or authoring an example/notebook, readcausalpy/skills/choosing-causalpy-methods/SKILL.mdfirst." Directives get acted on; mid-file location notes get skimmed. Fold in the trigger keywords from the skill's owntriggers.md. - Reframe the dev/user divide explicitly as distribution, not discovery: authoring examples and docs is a maintainer task that requires the user-facing method skills, so dev agents must consult them.
- Optional cheapest auto-discovery bridge without moving or duplicating content: commit a one-line stub at
.agents/skills/choosing-causalpy-methods/SKILL.mdwhose body just says the real skill lives atcausalpy/skills/choosing-causalpy-methods/and must be read. Shows up in auto-discovery, avoids committed symlinks.
Recommendation 2 — review graded-treatment routing
- Revisit
not_in_causalpy.mdand the routing/disambiguation cards so a graded or continuously-dosed intervention at known times routes to Piecewise ITS (step()/ramp()terms, level + slope + cumulative effects) where the design supports it, rather than being classified as out of scope. - Clarify the genuine boundary: whole-domain dose-response curve estimation vs. graded interventions expressible as segmented terms. The former may be out of scope; the latter is not.
Recommendation 3 — route to pathmc when it is the better fit
When a causal question is genuinely outside CausalPy's quasi-experimental scope but is well suited to a structural-causal-model / DAG approach (e.g. pathmc), the skill should say so and point the user there, rather than only returning "not implemented in CausalPy". A no-fit outcome is far more useful when it names the right neighbouring tool. Keep this at the level of intent: the skill should recognise when a sibling library is the better home and route to it. Exact wording and the boundary between the two are for the skill authors to define.
Suggested acceptance criteria
- An agent working from a fresh checkout is reliably routed to
choosing-causalpy-methodsbefore selecting a method. choosing-causalpy-methodsgives a correct route for a graded intervention at known times (Piecewise ITS), with the no-fit boundary re-scoped to genuine dose-response-curve cases.- No-fit outcomes name a suitable neighbouring tool (such as
pathmc) when the problem is better served there, instead of ending at a bare "not implemented".
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Read AGENTS.md, CLAUDE.md if present, and causalpy/skills/choosing-causalpy-methods/SKILL.md with its triggers.md. Then review reference/not_in_causalpy.md and the routing cards, and consider the optional .agents/skills/choosing-causalpy-methods/SKILL.md bridge. Done means fresh-checkout agents find the skill, graded interventions route to Piecewise ITS, and genuine no-fit cases name an appropriate neighbouring tool.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- developer-experience, documentation
- Issue type
- Documentation
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 52/100