microsoft / microsoft/winappCli
[Feature]: Publish sample indexes upstream (WinUI Gallery + Community Toolkit) so find-ui stops scraping
- Dominant language
- C#
- Stars
- 1.3k
- Forks
- 80
- Avg merge
- 3d 6h
- Merged PRs (30d)
- 51
Description
### Is your feature request related to a problem? Please describe.
`winapp find-ui` (#681) builds its corpus by scraping three upstream repos. The cost of that scraping is wildly uneven, and it comes down entirely to **what each repo publishes**:
| Source | Publishes | Fetcher size |
|---|---|---|
| `microsoft/microsoft-ui-reactor` | [`reactor-search-index.json`](https://github.com/microsoft/microsoft-ui-reactor/blob/main/samples/ReactorGallery/reactor-search-index.json) — metadata **and sample code** | **7 KB** (`ReactorFetcher.cs`) |
| `microsoft/WinUI-Gallery` | `ControlInfoData.json` — metadata only | **43 KB** (`GalleryFetcher.cs`) |
| `CommunityToolkit/Windows` | nothing published | **54 KB** (`ToolkitFetcher.cs`) |
Reactor's fetcher is ~6x smaller for the same job, purely because the Reactor team publishes a purpose-built index. Gallery and Toolkit require us to reverse-engineer repo layout, split samples out of XAML pages, clean code-behind, fold `#if` branches, re-balance tags, and truncate — roughly **1,450 lines of parsing** that exists only to reconstruct data the upstream repos already have.
This is not just a size problem. It produces **wrong output**:
- **9 of 321 Gallery scenarios (~3%) emit structurally malformed XAML** that fails to parse. Verified against both the current corpus and the original `winui-search` snapshot in `microsoft/win-dev-skills` — identical results, so this is inherent to scraping, not a porting bug:
```
xamlresources-3, appbarbutton-5, swipecontrol-4, swipecontrol-5, listview-8,
infobadge-2, selectorbar-3, themeshadow-1, richtextblock-3
```
Example — `gallery-appbarbutton-5`, a short, **non-truncated** sample:
```xml
```
- Every upstream layout change breaks us. `CacheVersion.cs` is on version **18**, and the bump history is almost entirely upstream churn (e.g. `"17" — WinUI-Gallery moved + reformatted its samples`).
- Toolkit control names are maintained by hand in a **34-entry `SampleOverrides` table**, so a newly added Toolkit sample gets a *guessed* name until someone notices.
Since `find-ui` is aimed at AI agents, a snippet that looks authoritative but doesn't compile or parse is worse than no result at all.
This also isn't a winapp-only problem: every WinUI agent/tool that wants sample data re-scrapes these same repos and independently re-derives these same bugs.
### Describe the solution you'd like
Submit PRs to the two upstream repos so each publishes a machine-readable sample index, the way `microsoft-ui-reactor` already does.
**1. `CommunityToolkit/Windows` — lowest effort, biggest win.**
The Toolkit **already generates exactly the index we need**. `ToolkitSampleMetadataGenerator` (in `CommunityToolkit/Tooling-Windows-Submodule`) stitches together:
- YAML frontmatter on every `components/*/samples/*.md` (`title`, `description`, `keywords`, `category`, `subcategory`)
- `> [!SAMPLE XSample]` markers linking docs to sample classes
- `[ToolkitSample(id:, title:, description:)]` attributes on the sample classes
…into `ToolkitSampleMetadata` at build time, with source-generator diagnostics when the frontmatter is malformed. It just never leaves the compiler — it only materializes inside the gallery app's binary.
**Ask:** also emit that metadata as a JSON artifact (repo file, release asset, or docs-site output). No new data modeling and no new validation — the assembly step already exists and is already enforced.
**2. `microsoft/WinUI-Gallery` — additive schema change.**
`WinUIGallery/SampleSupport/Data/ControlInfoData.json` already carries `UniqueId`, `Title`, `Subtitle`, `Description`, `ApiNamespace`, `RelatedControls`, and `Docs[]`. What's missing is the sample code, which lives in separate `Samples/{UniqueId}/{SampleDefinition}` `.txt` bundles.
**Ask:** inline those bundle contents (`--- header` / `--- xaml` / `--- c#`) into `ControlInfoData.json`, or emit a sibling index that does. The repo already maintains `ControlInfoDataSchema.json`, so there's a natural place to version the addition.
**Outcome.** Either change makes that source Reactor-shaped: its fetcher collapses from ~40-50 KB to ~100 lines, and the malformed-output class of bug disappears **by construction**, because nothing is being guessed.
### Additional context
- Blocked on nobody — this is follow-up work after #681 merges. #681 should ship the local mitigations (validate snippets at the corpus boundary; parse `[ToolkitSample(...)]` instead of stripping it) independently of upstream timelines.
- Suggested order: open the Toolkit issue/PR first — their data is already assembled and validated, so it's the shortest path to a proof point that helps the Gallery conversation.
- Once an upstream index exists, `GalleryFetcher.cs` / `ToolkitFetcher.cs` and most of `CacheVersion.cs`'s bump history can be deleted outright.
- Prior art within this codebase: `ReactorFetcher.cs` + `DATA_SOURCES.md` in `microsoft/win-dev-skills`, which states it directly — *"The Reactor team owns a purpose-built search index we consume directly, so there is no scraping or per-file cleaning."*
## Phases
- #808 — Phase 0: define the shared sample index contract
- #809 — Phase 1: WinUI-Gallery publishes a sample index (and we delete `GalleryFetcher`)
- #810 — Phase 2: CommunityToolkit publishes a sample index (and we delete `ToolkitFetcher`)
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.