elastic / elastic/ai-github-actions

[refactor-opportunist] Centralize workflow catalog metadata

Open
#1,612 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
11
Forks
16
Avg merge
22h 9m
Merged PRs (30d)
31

Description

## 🏗️ Refactor Proposal

**Summary:** Replace duplicated workflow-name lists in setup and dogfood scripts with one catalog that owns install recommendations, dogfood eligibility, and remediation metadata.

## Problem

Workflow metadata is currently encoded in multiple independently edited places. `scripts/quick-setup.sh` hardcodes the default and continuous-improvement workflow arrays, while `scripts/dogfood.sh` separately hardcodes excluded and remediation arrays. The same workflow is therefore represented in separate control paths, making additions and policy changes easy to apply incompletely. Recent churn confirms these are maintenance hotspots: the 60-day file-name histogram includes `Makefile` 6 times and many workflow artifacts, while the setup/dogfood scripts are the operational entry points that must be updated for every workflow change.

The repository already documents a split between consumer trigger sources and dogfood behavior (`gh-agent-workflows/DEVELOPING.md:37-43`), but does not have a single metadata boundary for that behavior.

## Proposed Approach

Make `scripts/workflow-catalog.tsv` the authoritative, reviewable metadata table. Keep workflow source files and consumer examples unchanged; have `quick-setup.sh` consume the catalog for recommended sets and have `dogfood.sh` consume it for dogfood eligibility and the transitional remediation flag. Add validation that every `gh-agent-workflows/*/example.yml` has exactly one catalog row, then migrate legacy remediation rows to per-workflow dogfood job fragments and remove the transitional column.

## Proof of Concept

I partially implemented this refactor on the setup/dogfood slice.

**Files changed:** `scripts/workflow-catalog.tsv`, `scripts/quick-setup.sh`, `scripts/dogfood.sh`, and `gh-agent-workflows/DEVELOPING.md`.

**Before → After:**

- Before: `quick-setup.sh` maintains two shell arrays; after: lines 114-138 load the catalog and derive both arrays.
- Before: `dogfood.sh` maintains `EXCLUDED_WORKFLOWS` and `REMEDIATION_WORKFLOWS`; after: lines 19-26 load catalog columns and lines 35-43 fail closed when a workflow has no metadata row.
- The catalog at `scripts/workflow-catalog.tsv:1-50` contains one row per workflow and explicit `quick_setup`, `continuous_improvement`, `dogfood`, and `remediation` flags.

**Verification:**

- `make lint` passed.
- `UV_CACHE_DIR=/tmp/gh-aw/agent/uv-cache make test` passed: 54 tests.
- `./scripts/dogfood.sh` completed successfully and generated the expected triggers.
- `bash -n scripts/dogfood.sh scripts/quick-setup.sh` passed.
- `git diff --check` passed.

## Incremental Rollout Plan

1. Adopt the catalog and validation for all workflow metadata (the proven slice).
2. Move each legacy remediation chain into a neighboring `dogfood-jobs.yml` fragment, retaining the catalog as the single eligibility source during migration.
3. Remove the remediation column and the remaining workflow-specific branches from `scripts/dogfood.sh`; add a small consistency check to CI.

## Risks and Mitigations

- **Catalog drift:** fail `scripts/dogfood.sh` when a workflow source has no row, and add a CI check for duplicate/stale rows.
- **Consumer behavior changes:** the catalog only controls setup recommendations and repository dogfood generation; consumer-facing `example.yml` files remain unchanged.
- **Partial migration complexity:** retain the remediation flag until all job fragments are migrated, then delete it in a separate change.

## Evidence

- `scripts/quick-setup.sh:114-145` now derives both recommended workflow sets from the catalog instead of hardcoded lists.
- `scripts/dogfood.sh:19-43,96-105` now derives dogfood policy from catalog metadata and validates missing entries.
- `gh-agent-workflows/DEVELOPING.md:232-234` documents the catalog as the shared metadata source.
- No open issue matched the exact `workflow-catalog.tsv` refactor; previously filed findings were checked and this proposal is distinct from them.

---
[What is this?](https://ela.st/github-ai-tools) | [From workflow: Trigger Refactor Opportunist](https://github.com/elastic/ai-github-actions/actions/runs/29752167389)

Give us feedback! React with 🚀 if perfect, 👍 if helpful, 👎 if not.

Contributor guide

Open the contributing guide

Research direction

Start with scripts/workflow-catalog.tsv and the referenced sections of scripts/quick-setup.sh and scripts/dogfood.sh, then read gh-agent-workflows/DEVELOPING.md:232-234. Run make lint, the repository test command, bash -n on both scripts, and ./scripts/dogfood.sh. Done means catalog metadata drives setup and dogfood behavior, every example workflow has exactly one row, and validation passes.

Written by the indexing model from the issue text.

Assessment

Tech stack
github-actions, shell
Domain
ci-cd, devops, tooling
Issue type
Refactor
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
52/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.