kubeflow / kubeflow/community

[Discussion] Shared `triage-agent` for issue/PR management, using Kubeflow-hosted infra

Open
#1,028 9 comments 7 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
199
Forks
265
Avg merge
1h 39m
Merged PRs (30d)
1

Description

## Context

Andrey recently set up a GitHub Project board to track/triage SDK issues and PRs, which already helps — but there's room to take this further with actual AI-assisted triage rather than manual board hygiene.

At the [Aug 26 ml-experience WG call](https://docs.google.com/document/d/1jH2WAX2ePxOfI4JuiVK9nPlesDMiyg67xzLwhpR7wTQ/edit?tab=t.0#heading=h.rhnrsv1lefem) — building on the discussion from the [Kubeflow Trainer project board](https://github.com/orgs/kubeflow/projects/85/views/2) — we talked through the need for a workflow that automatically flags duplicate issues and already-open PRs before they pile up.

From @sharkyii we learned the KFP (Pipelines) has already been building something similar to this.

This is what I found, (@droctothorpe @alyssacgoins @modichika can explain more):
- [kubeflow/pipelines#13372](https://github.com/kubeflow/pipelines/pull/13372) — AI issue-quality analyzer (GitHub Models, gpt-4o-mini), posts a scope/context/complexity breakdown as a comment on new issues
- [kubeflow/pipelines#14035](https://github.com/kubeflow/pipelines/pull/14035) — follow-up migrating it to GitHub Agentic Workflows, adding per-user rate limits, AIC budget caps, deterministic title validation, and tighter permission scoping

## Idea

Rather than each subproject building its own version of this, we are exploring the option to have one shared triage agent (working name: `kubeflow/triage-agent`) that any Kubeflow subproject can eventually adopt.

This issue is to open up that discussion — get scope, features, and guardrails first.

Open questions to work through:
- **Scope** — duplicate detection (issues + open PRs) as the core, with template/quality checks as a possible extension?
- **Feature** — initial plan (open to discussion):
- Duplicate issue detection — flag likely duplicates against open/recently closed issues with a comment linking candidates. (ig GitHub already does it).
- Duplicate PR detection — flag PRs overlapping in scope/files with other open PRs
- Issue-quality check — flag missing repro steps, missing template sections, vague titles (similar to KFP's analyzer)
- Label suggestions — suggest `area/`, `kind/`, `good-first-issue` labels for a maintainer to confirm, not auto-apply
- Template compliance — flag issues/PRs that don't follow the required title/body format
- **Config per subproject** — label taxonomies, issue templates, and area ownership differ across Trainer, Pipelines, Katib, etc. Some config like `triage.md`, similar to `AGENTS.md`.
- **Guardrails** — no unnecessary write scopes and follows secure practice.

## Why this, and not an existing solution?

- **Cost of LLM** — running on our own infra means we're not paying per-request LLM costs. The only real constraint is aggregate request volume across subprojects, known load we can size for up front.
- **Kubeflow-specific context** — A shared agent configured per-repo (via the `triage.md`-style config above) can triage issues/PR much more efficiently.
- **We already have the infra** — [Kubeflow Cloud](https://github.com/kubeflow/community/pull/1022) exists as community-run infra, so the hardware is already there and we can just build on it.

Feel free to add your suggestions here

cc @andreyvelich @harshpopat23

Contributor guide

Open the contributing guide

Research direction

Start by reviewing kubeflow/pipelines PRs #13372 and #14035, the GitHub Project board, and the Kubeflow Cloud infrastructure referenced in the issue. The immediate work is to turn the open questions into an agreed scope, per-project configuration model, and permission guardrails; done means the discussion has a concrete proposal rather than an implementation patch.

Written by the indexing model from the issue text.

Assessment

Tech stack
github
Domain
ai, cloud, devtools
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.