/orchestration cleanup: API-only blueprints, canonical docs links, content-owned categories
- Dominant language
- Astro
- Stars
- 155
- Forks
- 177
- Avg merge
- 1d 20m
- Merged PRs (30d)
- 192
Description
### Why
Team flagged "trash code" on the orchestration pages, with the blueprint handling as his example. An audit on 2026-09-14 confirmed the issues below against the repo, git history (`803af4739`, `ac97b6e91`), the live API and rendered pages.
### Files involved
| Role | Path |
|---|---|
| Per-tool template | `src/pages/orchestration/[slug].astro` |
| Catalog page | `src/pages/orchestration/index.astro` |
| Blueprint section | `src/components/orchestrate/OrchPattern.astro` |
| Schema | `src/schemas/orchestration.ts` |
| Content | `src/contents/orchestration/*.yaml` (38 files) |
| Existing utils to reuse | `src/utils/fetch.ts` (`$fetchApiCachedOptional`), `src/utils/plugins/taskUrls.ts` (`buildTaskUrls`), `src/utils/blueprints/*.ts` + `*.test.ts` as the pattern |
| Consumer to protect | `src/pages/blueprints/[id].astro:198` (regex-parses the YAML `tool:` block) |
---
### PR 1: blueprints come only from the API
**Problem.** `gemini.yaml` item 3 ("Multimodal extraction") has `blueprintPlaceholder: true` + inline `code:` + `plugins:`. The blueprint now exists in the API as `gemini-multimodal-receipt-extraction` and the inline copy has drifted (model names, trigger credentials, payload). The page also does its blueprint fetching inline with hand-rolled try/catch (`[slug].astro:118-149`) even though the repo pattern is a util in `src/utils/blueprints/` using `$fetchApiCachedOptional`. Commit `803af4739` ("remove fallback for blueprint and trust API") removed inline code from 20 files in May but left `code`/`plugins` optional in the schema, which let this come back.
**Tasks**
- [ ] `src/contents/orchestration/gemini.yaml`: on item 3 set `blueprintId: "gemini-multimodal-receipt-extraction"`, delete `blueprintPlaceholder`, `plugins`, `code`.
- [ ] `src/schemas/orchestration.ts`: in `blueprintSchema` remove `code`, `plugins`, `flowDiagram`, `blueprintPlaceholder`; make `blueprintId` required. Delete `flowNodeSchema` and `flowDiagramSchema` (no consumers, 0 YAML uses).
- [ ] New `src/utils/blueprints/orchestrationBlueprints.ts`:
- `resolveBlueprintRenders(items)`: for each item fetch `/blueprints/${id}/versions/latest` and `/blueprints/${id}/versions/latest/graph` with `$fetchApiCachedOptional`, return `{ ...item, render: { source, graph } | undefined }`. Keep the URLs byte-identical (fixtures depend on them).
- `fetchToolBlueprintCount(toolName)`: `/blueprints/versions/latest?q=${encodeURIComponent(toolName)}&size=1`, return `total ?? 0`.
- [ ] New `src/utils/blueprints/orchestrationBlueprints.test.ts` mirroring `src/utils/plugins/taskUrls.test.ts` (`vi.mock("~/utils/fetch")`): happy path, graph 422 keeps source, full failure returns item without `render`.
- [ ] `[slug].astro`: replace lines 118-149 with the two calls. Remove the `$fetchApiCached` import if nothing else uses it.
- [ ] `OrchPattern.astro`: remove `code`, `plugins`, `blueprintPlaceholder` from `BlueprintItem`; simplify lines 116-124 to `bp.render?.source ? : `. Keep the fallback block (only degradation path if the API 404s). Props unchanged.
**Acceptance**
- `/orchestration/gemini` tab 3 shows the API flow, a topology, and "Open blueprint" -> `/blueprints/gemini-multimodal-receipt-extraction`.
- `grep -rn "code: |" src/contents/orchestration` returns nothing.
- `npm run test:unit` passes with the new test.
---
### PR 2: plugin docs link resolves to the canonical page
**Problem.** `[slug].astro:158-164` derives the docs URL from the class name. For DuckDB, Postgres, Snowflake, SQL Server it yields `/plugins/plugin-jdbc` and for Fabric `/plugins/plugin-microsoft`. All five 301 to `/plugins` in production. Canonical names are `plugin-jdbc-duckdb`, `plugin-jdbc-postgres`, `plugin-jdbc-snowflake`, `plugin-jdbc-sqlserver`, `plugin-microsoft-fabric`.
**Tasks**
- [ ] Import `buildTaskUrls` from `~/utils/plugins/taskUrls`.
- [ ] `const pluginDocsUrl = (await buildTaskUrls([tool.pluginClass]))[tool.pluginClass] ?? `.
- [ ] Keep the derivation as fallback so an API failure still produces a link.
**Acceptance**
- Rendered `docs-card` href on `/orchestration/duckdb` is `/plugins/plugin-jdbc-duckdb`; same check for postgres, snowflake, sql-server, microsoft-fabric.
- Unchanged hrefs on terraform, gemini, netbox (already correct).
---
### PR 3: categories in content, dead fields removed
**Problem.** `index.astro:13-51` `CATEGORY_MAP` is missing `gemini`, so the index shows an "Other" card and an "Other" filter tab. The `vs` collection stores `category` in the schema instead. Several schema fields are dead: `blueprints.lead` (never rendered), `items[].href` (only read when `blueprintId` is absent), `LOGO_MAP.fortune500` (no YAML uses it), `data-name`/`data-color` on carousel items (script only reads cards).
**Tasks**
- [ ] Schema: add `category: z.enum(["Infrastructure","Cloud","Data","Business","AI"])` under `tool`. **Put it after `pluginClass`**, see the regex warning below.
- [ ] 38 YAML files: add `category:` under `tool:` using the current `CATEGORY_MAP` values; `gemini: AI`.
- [ ] `index.astro`: delete `CATEGORY_MAP`, use `entry.data.tool.category`. Keep `CATEGORY_ORDER`.
- [ ] Schema + YAML: remove `blueprints.lead` and `items[].href`.
- [ ] `[slug].astro`: remove `Fortune500Logo` import and the `fortune500` entry in `LOGO_MAP`.
- [ ] `index.astro:159-160`: remove `data-name` and `data-color` from `.carousel-item`.
**Acceptance**
- Index shows exactly the tabs All tools, Infrastructure, Cloud, Data, Business, AI. No "Other".
- Gemini card carries `data-category="AI"`.
- Blueprint pages still show "Orchestration" backlinks (see warning).
---
### Warnings
- **Regex consumer.** `src/pages/blueprints/[id].astro:198-215` reads each orchestration YAML with `/^tool:\s*\n\s+name:\s*"([^"]+)"\s*\n\s+pluginClass:\s*"([^"]+)"/m`. `name` and `pluginClass` must stay on consecutive lines, quoted. Any new key goes after `pluginClass`. Check `/blueprints/terraform-drift-detection` still lists the Terraform orchestration link after PR 3.
- **Fixtures.** `tests/fixtures/api/` contains recorded responses keyed by URL. Do not change the request URLs in PR 1.
- **Incremental build.** API data is not in the page cache key (`[slug].astro:86-98`). Schema edits change the module graph and re-render all 38 pages once. That is expected.
- **Do not touch.** `OrchPattern` styles and tab script, the index inline script (`ClientRouter` is on in prod, the `window.__orchTimer` guards are needed), `fetchPluginIcon` inline SVGs (the hero recolors them), the blog keyword heuristic, any copy.
### Verification
```bash
npm run test:unit
npx astro dev --port 4399
npm run test:visual
```
Check `/orchestration`, `/orchestration/gemini`, `/orchestration/duckdb`, `/orchestration/terraform` and `/blueprints/terraform-drift-detection`. Visual snapshots may need updating for the Gemini page (new topology on tab 3) and the index (no "Other" tab).
### Open decisions
1. Keep `items[].description`? It is only shown in the API-failure fallback. Default: keep.
2. Merge PR 2 and PR 3 into one? Default: yes, both are mechanical.
Contributor guide
Research direction
Start with the listed orchestration pages, schema, content YAML, and existing utilities under src/utils/blueprints/ and src/utils/plugins/taskUrls.ts. Run the unit tests first, then verify the specified orchestration and blueprint pages with the Astro dev server and visual tests. Done means API-backed blueprint renders, canonical plugin links, content-owned categories without an Other tab, and passing verification.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- typescript
- Domain
- api, documentation, frontend
- Issue type
- Refactor
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 48/100