[Discussion] Shared `triage-agent` for issue/PR management, using Kubeflow-hosted infra
- Dominant language
- Jupyter Notebook
- Stars
- 199
- Forks
- 265
- Avg merge
- 1h 39m
- Merged PRs (30d)
- 1
Description
## Context
Andrey recently set up a GitHub Project board to track/triage SDK issues and PRs, which already helps — but there's room to take this further with actual AI-assisted triage rather than manual board hygiene.
At the [Aug 26 ml-experience WG call](https://docs.google.com/document/d/1jH2WAX2ePxOfI4JuiVK9nPlesDMiyg67xzLwhpR7wTQ/edit?tab=t.0#heading=h.rhnrsv1lefem) — building on the discussion from the [Kubeflow Trainer project board](https://github.com/orgs/kubeflow/projects/85/views/2) — we talked through the need for a workflow that automatically flags duplicate issues and already-open PRs before they pile up.
From @sharkyii we learned the KFP (Pipelines) has already been building something similar to this.
This is what I found, (@droctothorpe @alyssacgoins @modichika can explain more):
- [kubeflow/pipelines#13372](https://github.com/kubeflow/pipelines/pull/13372) — AI issue-quality analyzer (GitHub Models, gpt-4o-mini), posts a scope/context/complexity breakdown as a comment on new issues
- [kubeflow/pipelines#14035](https://github.com/kubeflow/pipelines/pull/14035) — follow-up migrating it to GitHub Agentic Workflows, adding per-user rate limits, AIC budget caps, deterministic title validation, and tighter permission scoping
## Idea
Rather than each subproject building its own version of this, we are exploring the option to have one shared triage agent (working name: `kubeflow/triage-agent`) that any Kubeflow subproject can eventually adopt.
This issue is to open up that discussion — get scope, features, and guardrails first.
Open questions to work through:
- **Scope** — duplicate detection (issues + open PRs) as the core, with template/quality checks as a possible extension?
- **Feature** — initial plan (open to discussion):
- Duplicate issue detection — flag likely duplicates against open/recently closed issues with a comment linking candidates. (ig GitHub already does it).
- Duplicate PR detection — flag PRs overlapping in scope/files with other open PRs
- Issue-quality check — flag missing repro steps, missing template sections, vague titles (similar to KFP's analyzer)
- Label suggestions — suggest `area/`, `kind/`, `good-first-issue` labels for a maintainer to confirm, not auto-apply
- Template compliance — flag issues/PRs that don't follow the required title/body format
- **Config per subproject** — label taxonomies, issue templates, and area ownership differ across Trainer, Pipelines, Katib, etc. Some config like `triage.md`, similar to `AGENTS.md`.
- **Guardrails** — no unnecessary write scopes and follows secure practice.
## Why this, and not an existing solution?
- **Cost of LLM** — running on our own infra means we're not paying per-request LLM costs. The only real constraint is aggregate request volume across subprojects, known load we can size for up front.
- **Kubeflow-specific context** — A shared agent configured per-repo (via the `triage.md`-style config above) can triage issues/PR much more efficiently.
- **We already have the infra** — [Kubeflow Cloud](https://github.com/kubeflow/community/pull/1022) exists as community-run infra, so the hardware is already there and we can just build on it.
Feel free to add your suggestions here
cc @andreyvelich @harshpopat23
Contributor guide
Research direction
Start by reviewing kubeflow/pipelines PRs #13372 and #14035, the GitHub Project board, and the Kubeflow Cloud infrastructure referenced in the issue. The immediate work is to turn the open questions into an agreed scope, per-project configuration model, and permission guardrails; done means the discussion has a concrete proposal rather than an implementation patch.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- github
- Domain
- ai, cloud, devtools
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100