PostHog / PostHog/posthog

Self-driving: custom instructions field for the implementation agent

Open
#79,136 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
39.9k
Forks
3.4k
Avg merge
6h 51m
Merged PRs (30d)
232

Description

Filing via the API, so I can't apply the enhancement label myself — please tag per CONTRIBUTING.

Is your feature request related to a problem?

Please describe.

First, credit where it's due: the implementation agent clearly reads repo instruction files. Our PRs come back with the correct conventional-commit scopes from our own scope table, and the exact section structure of our .github/PULL_REQUEST_TEMPLATE.md, unprompted. That works well and we didn't have to configure anything.

The gap is a different class of rule — ones about how the PR is opened rather than how the code is written: PR lifecycle (draft vs ready), which labels to apply, ticket policy, which reviewers to request. Those don't belong in AGENTS.md, which is read by every agent that touches the repo and describes the codebase, not our PostHog integration.

We went looking for somewhere to put them and there isn't one:

  • scout-config-update writes enabled, emit, run_interval_minutes, run_cron_schedule, output_destinations.slack, structured_output_schema, network_access, model, auto_pause_exempt, tags. No prompt or instructions field, and skill_name is fixed.
  • That leaves editing a scout's skill, or scout-notes-create. Notes are the closest hook but the wrong shape: they steer what gets found, they're explicitly advisory, and they're scoped to the scout fleet rather than to a repo.
  • Scouts don't target a repo at all — each watches a PostHog product surface — so there's no object in the model a per-repo rule could attach to.

Describe the solution you'd like

A repo-level (or self-driving-project-level) custom instructions field passed to the implementation agent, alongside whatever it reads from the repo.

Even a plain free-text box would do it. Ours would say roughly: open as draft, add bot-review, put the Linear ID in the title, reply to review threads before pushing.

Describe alternatives you've considered

  • Putting it in AGENTS.md. We did, as a fallback, and the agent does read it. But it pollutes a file every other agent reads with instructions specific to one vendor's PR workflow, and it can't express things the agent has no control over anyway.
  • A general scout note. We left one. It's advisory and aimed at signal-finding, so it's a nudge, not configuration.
  • Enforcing after the fact in CI. What we do now — a workflow posts an advisory comment listing what a bot PR is missing. It works, but it's a per-PR correction loop for something that could be stated once.

Additional context

Several of the other requests we filed collapse into this one if it exists: PostHog/posthog#79134 (branch naming) and #79135 (draft + labels) are both "let the repo owner shape how the PR is opened". If a custom-instructions field could express those, they don't need separate settings.

Debug info

  • PostHog Cloud (US)

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by tracing the scout-config-update and scout-notes-create entry points, then compare them with the implementation agent's existing repo-instruction handling through AGENTS.md. Define where repo-level or self-driving-project instructions are stored and passed to the agent; done means the configured text is persisted and affects PR lifecycle actions such as draft status, labels, titles, and review replies.

Written by the indexing model from the issue text.

Assessment

Tech stack
github
Domain
developer-experience, tooling
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.