Add ProjectTeam model, retire MaintainerStatus, and track upstream's `managed` team flag
- Dominant language
- Go
- Stars
- 9
- Forks
- 10
- Avg merge
- 16h 55m
- Merged PRs (30d)
- 6
Description
## Context
Issue #171 decided to wipe and regenerate the `lfx`-source identity observations. Working through the execution surfaced a bigger problem than the data reset: **the refill would reproduce the current blind spot**, because both project association and LFX enrichment read exactly one team out of each `maintainers.yaml` — the team named literally `project-maintainers`.
That requirement no longer exists for .project repos.[cncf/automation#602](https://github.com/cncf/automation/pull/602) ("feat: add managed flag to maintainer teams for flexible resource provisioning", merged Aug 2026) removed it:
- `Team` gained `Managed *bool` (`utilities/dot-project/types.go`), with `IsManaged()` returning **true when the field is nil** — absent means managed.
- `ExtractHandles` — the function that decides who upstream provisions CNCF resources to — now iterates **every team** and skips only `!IsManaged()`. Team name is no longer read at all.
- Validation changed from "a `project-maintainers` team is required" to "at least one managed team is required" / "at least one managed team must have members".
- The scaffold template now emits `name: "maintainers"` and ships commented-out `reviewers` / `emeritus` stanzas pre-set to `managed: false`. `volcano-sh/.project` is the live reference implementation.
So maintainer-d is pinned to a schema that .project retired. `grep -rn "managed" dotproject/ lfx/ model/ cmd/` returns no relevant hit — the concept is entirely absent here.
Production confirms the cost. A probe over stored `dot_project_sync_states.maintainers_file_body` found 34 distinct team names: `project-maintainers` 168 projects (the only one honoured), `reviewers` 55, `emeritus` 52, **`maintainers` 50**, 15 Tekton component teams at 10 each, `spiffe-maintainers` / `spire-maintainers` / `emeritus-maintainers` 3 each. The 50 `maintainers`-team projects and the Tekton component teams are unambiguous maintainers getting no association and no LFX check today.
Note `emeritus-maintainers` matches `*-maintainers`, so no name-suffix heuristic is safe. _`managed` is the correct gate._
**Intended outcome:** maintainer-d's notion of "who is a maintainer of this project" becomes the same set upstream provisions — _the union of managed teams_ — and every one of those people gets an LFX identity lookup. Since a full data reboot is on the table, this is also the moment to represent teams as first-class rows instead of collapsing them into a single boolean-gated parse, and to retire `MaintainerStatus` in favor of team membership.
## 1. Model change: `ProjectTeam` replaces the flat `MaintainerProject` join and `MaintainerStatus`
A project has many teams; a team has many maintainers. That structure doesn't technically exist today — `MaintainerProject` is a flat (maintainer, project) join carrying a hand-set `Status` enum (`Active`/`Emeritus`/`Retired`/`Archived`) that duplicates, and can drift from, what `maintainers.yaml` actually says.
maintainer-d does model projects and sub-projects, a sub-project is a project that has a parent_project_id, the CNCF has some projects that organise themselves this way and the implmentation cost to the model is very cheap at the storeage layer and resonable on the front-end webapp if rendering sub-projects separately is deemed useful.
```go
type ProjectTeam struct {
gorm.Model
ProjectID uint `gorm:"index;not null"`
Project Project `gorm:"foreignKey:ProjectID;constraint:OnDelete:CASCADE"`
Name string `gorm:"size:255;not null"` // upstream team name, informational only
Managed *bool `gorm:"default:true"` // nil == true, mirrors upstream Team.IsManaged()
Maintainers []Maintainer `gorm:"many2many:project_team_maintainers;joinForeignKey:ProjectTeamID;joinReferences:MaintainerID"`
}
type ProjectTeamMaintainer struct {
ProjectTeamID uint `gorm:"primaryKey;index"`
MaintainerID uint `gorm:"primaryKey;index"`
JoinedAt time.Time `gorm:"autoCreateTime"`
ProjectTeam ProjectTeam `gorm:"foreignKey:ProjectTeamID;constraint:OnDelete:CASCADE"`
Maintainer Maintainer `gorm:"foreignKey:MaintainerID;constraint:OnDelete:CASCADE"`
}
```
`MaintainerStatus` (`model/main.go`, the `Active`/`Emeritus`/`Retired`/`Archived` enum and its `IsValid`/`OrDefault` helpers) is **removed entirely**, along with `MaintainerProject.Status`. Decision: **do not** give `ProjectTeamMaintainer` a status override column either — every "who's a maintainer and what's their state" question is answered purely from team membership, derived as:
| Old `MaintainerStatus` | New derived meaning |
|---|---|
| `Active` | a `ProjectTeamMaintainer` row exists on a **managed** team |
| `Emeritus` | a `ProjectTeamMaintainer` row exists only on **unmanaged** team(s) |
| `Retired`/`Archived` | no `ProjectTeamMaintainer` row on this project at all |
This removes the manual status-override path that exists today (`handleMaintainerStatusUpdate`, `POST /api/maintainers/status`, `cmd/web-bff/main.go:3798`) — a human can currently mark someone Emeritus/Retired in maintainer-d independent of what's committed upstream. Going forward the only way to change someone's standing is a merged PR against the project's `.project` repo, which is deliberate: it pushes correction to the source of truth instead of letting maintainer-d and upstream disagree silently. **When and why a maintainer left is left to the git log of the relevant `.project` repo** — maintainer-d does not need to record it separately, so `ProjectTeamMaintainer` does not need a soft-delete/tombstone either; a departure is just a row not being present after the next sync.
**Frontend:** remove the maintainer status editing affordance entirely — `onBulkStatusChange` (`web/src/components/ProjectReconciliationCard.tsx:184,321,549,552`) and its caller in `web/src/app/projects/[id]/ProjectRouteClient.tsx:573-574` (`POST ${apiBaseUrl}/maintainers/status`). Whatever renders the status badge/column switches from reading `MaintainerProject.Status` to the derived managed/unmanaged/absent state above.
**Call sites that change semantics, not just plumbing** (`grep -rn "MaintainerStatus\|ActiveMaintainer\|EmeritusMaintainer\|RetiredMaintainer\|ArchivedMaintainer"` across non-test `.go` files for the full list):
- `db/store.go` / `db/store_impl.go` — `ListMaintainerProjectStatuses`, `GetMaintainerProjectStatuses`, `UpdateMaintainerProjectStatus(s)`, `GetMaintainerProjectStatus` (`:383-470`) are replaced by team-membership queries; `ListMaintainersActiveOnAnyProject` (`:412-428`, `Where("status = ?", model.ActiveMaintainer)`) becomes a join through `project_team_maintainers` filtered to managed teams.
- `cmd/web-bff/main.go` — `activeProjectMaintainerGitHubHandles` (`:1800`), `classifyIneligibleMaintainers` (`:6678`), `buildFossaInviteCandidates` (`:6711`), and the other `projectStatuses[...] != model.ActiveMaintainer` gates (`:5996,6047,6292`) that decide FOSSA invite eligibility all become "is on a managed team." The overall-status rollup at `:2691-2696` (`ArchivedMaintainer` default, preferring `Emeritus`/`Retired` across projects) becomes "managed on ≥1 project" / "unmanaged-only" / "on none."
- `cmd/sanitize/main.go:212-213` — `currentStatus != model.ActiveMaintainer` → `UpdateMaintainerProjectStatus(..., ActiveMaintainer)` becomes ensuring a `ProjectTeamMaintainer` row exists on the managed team the sync run found them in.
- `dotproject/auto_add.go:124` and `lfx/enricher.go:119` — both currently call `ParseProjectMaintainerEntries`; both are rewritten to walk parsed `ProjectTeam`s and upsert `ProjectTeamMaintainer` rows (association) / enumerate candidates (LFX) per managed team, rather than one flat handle set.
- `Project.Maintainers` / `Maintainer.Projects` many2many declarations in `model/main.go` and the `SetupJoinTable` calls in `db/connection.go:45,48` and `onboarding/test_helpers.go:24-25` move from `MaintainerProject` to being derived through `ProjectTeam` (either a real join or a computed accessor — implementer's call, but the many2many should not keep writing to a table nothing else reads).
`dotproject/discovery.go` changes accordingly: instead of returning a flat handle set or `MaintainerEntry` list, the parser returns one entry per team (`{Name string; Managed *bool; Members []MaintainerEntry}`), and callers (auto-add, LFX enricher, the `dot_project_maintainer_count` display) decide what to do with managed vs. unmanaged teams rather than the parser silently dropping unmanaged ones.
- Replace the exact-slug filter in `ParseProjectMaintainerEntries` (`:344-353`, including the `otherTeamNames` diagnostic that exists only to measure that filter's false negatives) with team-shaped output gated on `managed`. Team **name becomes informational**, not a gate.
- Restate the shape errors at `:383-393` in upstream's terms: "no managed team found" and "managed teams have no members".
- `ParseMaintainerHandles` (`:264-299`) currently reads every team *including unmanaged ones* and feeds the displayed `dot_project_maintainer_count`. That count should reflect managed-team membership only, so the page and the associated roster stop disagreeing (today they diverge in both directions).
- The `member.Kind != yaml.ScalarNode || ShortTag() != "!!str"` rejection at `:368-371` stays; only its message needs the team name removed.
Also confirm `SupportedSchemaVersion = "1.0.0"` (`discovery.go:20`, enforced at `:249`) against whatever #602 left in upstream `SCHEMA.md` — #602 did not appear to bump it, but this should be verified rather than assumed.
### Accepted consequence
Files predating #602 have no `managed:` field, so under `nil == managed` their `emeritus` (52 projects) and `reviewers` (55 projects) teams become associated maintainers (on an unmanaged team, so not "Active" in the old sense, but present).
This is not a maintainer-d bug — it is exactly who upstream provisions today, and the divergence is upstream data that needs migrating. Handle it by reporting, not by fudging the parser: add a read-only drift query listing projects with a team whose name matches `emeritus|alumni|reviewer|past|former` and **no** explicit `managed: false`, as the worklist for upstream fixes. Natural companion to [cncf/automation#673](https://github.com/cncf/automation/pull/673).
## 2. Update the docs that still assert the old requirement
The same drift cncf/automation#673 fixes upstream, mirrored here: `specifications/dot-project.md`, `specifications/auto-add-maintainer.md`, `specifications/lfx-integration.md`, `docs/dot-project-sync-walkthrough.md`. Plus the seed fixture `cmd/web-bff-seed/main.go:248`, which should grow a `managed: false` team so tests exercise the new path.
The `issue-draft-*` / `post-pr-157-*` collation files are historical records — leave them.
Do **not** touch the `project-maintainers.csv` references in `dotproject/auto_add.go:204,225,407` and `cmd/dot-project-sync/main.go:208-213`; that is the unrelated cncf/foundation CSV filename.
## 3. Raise the per-project lookup cap in step
Widening makes one `EnrichProject` call cover every managed team in the file — for Tekton-shaped files, 15 teams at once. `defaultMaxLookups = 100` (`cmd/dot-project-sync/main.go:316`) is a **per-call** cap checked against `summary.Attempted` (`lfx/enricher.go:66`), so it can truncate a roster mid-file. Size it from the pre-flight query below and set `LFX_MAX_LOOKUPS` rather than editing the constant.
## 4. Then reboot the data (#171 folds in here)
Since the model change replaces `MaintainerProject`/`MaintainerStatus` outright, #171's "wipe the `lfx` source" becomes a full reboot: drop and recreate the association tables, then let sync repopulate them under the new schema, rather than a soft-delete-and-refill of just the `lfx` observations.
1. Backup exists (`maintainerd-prod-pre-observation-cleanup.dump`).
2. Migrate: create `project_teams` / `project_team_maintainers`, drop `maintainer_projects` (and the `Status` column/enum), migrate `maintainer_identity_observations` rows for `source = 'lfx'` by soft-deleting them (`UPDATE ... SET deleted_at = now() WHERE source = 'lfx' AND deleted_at IS NULL`, 3,396 rows) since association provenance is changing underneath them.
3. Refill with **ordinary (non-enrich-all) `dot-project-sync` runs** — the per-project path is the one that reads `maintainers.yaml` and is now the one that populates `project_teams`/`project_team_maintainers`. Not `LFX_ENRICH_ALL_MAINTAINERS=true`: that branch (`lfx/enricher.go:145-169`) reads the `maintainers` table filtered to active-on-some-project, a different population that misses YAML handles with no maintainer row.
4. Optional sweep afterwards: one enrich-all pass for maintainers active on a project but named in no `.project` file (legacy-ref and CSV arrivals).
Operational constraints for step 3: the CronJob is suspended (`deploy/manifests/maintainerd-dot-project-sync-cronjob.yaml`), so runs are manual with a pasted token; each is bounded by `DOT_PROJECT_SYNC_TIMEOUT` (50m in the manifest, which overrides the enrich-all 1h bump); and LFX 401/429 are `FatalSyncError` (`lfx/client.go:47-70`) aborting the whole run. Expect several passes. `ListProjects` anti-starvation ordering (oldest `dot_project_last_synced_at` first) means unvisited projects come first.
## Pre-flight queries (read-only)
How far `managed:` has actually spread into stored bodies, versus how many still carry `project-maintainers`:
```sql
SELECT count(*) AS with_file,
count(*) FILTER (WHERE body ~ 'managed\s*:') AS declares_managed,
count(*) FILTER (WHERE body ~ '^\s*managed\s*:') AS declares_managed_uncommented,
count(*) FILTER (WHERE body ~ '-\s+name:\s*["'']?project-maintainers') AS has_project_maintainers
FROM (SELECT maintainers_file_body AS body
FROM dot_project_sync_states
WHERE maintainers_file_body IS NOT NULL) t;
```
Sizing for `LFX_MAX_LOOKUPS` — widest roster across all teams:
```sql
SELECT s.project_id,
(SELECT count(*) FROM regexp_matches(s.maintainers_file_body,
'^\s+-\s+\S+\s*$', 'gm')) AS member_lines
FROM dot_project_sync_states s
WHERE s.maintainers_file_body IS NOT NULL
ORDER BY 2 DESC
LIMIT 10;
```
Keep these file-based and run via `./scripts/psql-maintainerd.sh -f ` — inline SQL hits shell-escaping trouble.
## Critical files
- `model/main.go` — new `ProjectTeam`/`ProjectTeamMaintainer`; remove `MaintainerStatus` and `MaintainerProject`
- `dotproject/discovery.go:264-299` (`ParseMaintainerHandles`), `:313-400` (`ParseProjectMaintainerEntries`; filter at `:344`, errors at `:383-393`), `:20` / `:249` (schema version pin) — reworked to emit per-team results
- `dotproject/auto_add.go:124` and `lfx/enricher.go:119,66` — rewritten to walk teams and upsert `ProjectTeamMaintainer`
- `db/store.go` / `db/store_impl.go:103-122,383-470,890-960` — status accessors replaced by team-membership queries; soft-delete/upsert behavior on `maintainer_identity_observations` unchanged and re-verified
- `cmd/web-bff/main.go:1800,2691-2696,3798,5996,6047,6292,6678,6711` — FOSSA eligibility gates, overall-status rollup, and removal of `handleMaintainerStatusUpdate`
- `cmd/sanitize/main.go:212-213` — active-maintainer reconciliation against managed-team membership
- `db/connection.go:45,48`, `onboarding/test_helpers.go:24-25` — join-table setup moves off `MaintainerProject`
- `web/src/components/ProjectReconciliationCard.tsx:184,321,549,552` and `web/src/app/projects/[id]/ProjectRouteClient.tsx:573-574` — remove the status-edit UI and its `/api/maintainers/status` call
- `cmd/dot-project-sync/main.go:314-320` (`LFX_MAX_LOOKUPS`)
- Docs listed in step 2
## Verification
1. New fixtures in `dotproject/discovery_test.go` covering:
- a #602-style file (`maintainers` managed + `emeritus` / `reviewers` with `managed: false`) — only the managed team is associated; unmanaged teams parse but are marked unmanaged
- an explicit `managed: true`
- a pre-#602 file with no `managed:` anywhere — every team returned as managed, documenting the accepted consequence
- a file whose only teams are `managed: false` — "no managed team found"
2. `make test-package PKG=dotproject`, `PKG=lfx`, `PKG=db`, `PKG=onboarding`, then `make ci-local`.
3. Rehearse the full migration + reboot on `maintainerd_local` (restored from the prod dump). Confirm `dot_project_maintainer_count` drops for projects that declare unmanaged teams, rises for the 50 `maintainers`-team projects, and that FOSSA invite eligibility (`buildFossaInviteCandidates`) produces the same candidate set as before for a sample of unaffected projects.
4. On prod, track between refill passes:
```sql
SELECT match_status, count(*) FROM maintainer_identity_observations
WHERE source = 'lfx' AND deleted_at IS NULL GROUP BY 1 ORDER BY 2 DESC;
```
Baseline before the reboot: 1411/792 matched, 719/407 unmatched, 65 error, 2 legacy `ambiguous`. The `ambiguous` and `error` rows should not reappear; `chosen` / `duplicate` should appear for the first time (the #157 multi-profile path has never completed a production run).
5. Spot-check three project pages: a Tekton component project, one of the 50 `maintainers`-team projects, and volcano (the #602 reference implementation). Confirm the status column now shows managed/unmanaged/absent instead of a hand-editable value, and that there is no remaining status-edit control in the UI.
## Related
- #171 — wipe and regenerate `lfx` identity observations (superseded/folded in — see step 4)
- cncf/automation#602 — upstream `managed` flag
- cncf/automation#673 — upstream doc consistency fixes
🤖 Generated with [Claude Code](https://claude.com/claude-code) as an artefact of interaction with @RobertKielty
Claude Code has been instructed to use this issue as the prompt to imlement this change.
Reviewed and edtied by @RobertKielty post intiial generation
- Terminology change: Change "upstream" to ".project" to refer to the dot-project work that lives in https://github.com/cncf/automation/tree/main/utilities/dot-project
- Note that sub-project modelling in maintainer-d exists and could have been used to model teams but I think we should keep both abstractions. The CNCF has projects that use a sub-project structer but across the entirity of the landscape it is relatively rare. We could do with more input required from the CNCF Projects team on this topic.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.