alandtse / alandtse/devbench

devbench: surface + extend CS config-menu control for QA (discoverability, describe, section nav)

Abierto
#37 2 comentarios 1 reacción 0 asignados Ver en GitHub
Lenguaje dominante
C++
Estrellas
10
Forks
4
Merge medio
1 d 10 h
PR fusionados (30 d)
17

Descripción

## Background

devbench *can* drive the Community Shaders ImGui config menu — `menu {action:invoke, name:"CommunityShaders"}` toggles it (CS registers itself; it shows under `menu {action:list}` → `.registered`). This is the right tool for QA-ing the CS menu (e.g. verifying i18n string coverage, exercising every `DrawSettings` section).

## Cost: wasted MCP round-trips before real work

Opening the CS menu took **four** MCP calls to get right because the verb/param shape isn't guessable, and each wrong call is a full game round-trip:

1. `menu {action:open, name:"CommunityShaders"}` -> `400` (open is engine-menus-only)
2. `menu {action:invoke, menu:"CommunityShaders"}` -> `400` (wrong param name)
3. `menu {action:invoke, name:"CommunityShaders"}` -> works

For an agent driving the bench, every failed call is wasted latency and tokens before any real testing happens. The fix is to make the *first* guess work: a discoverable `openshaders.menu` tool and/or a one-line doc + a clearer error that names the exact working call. Lowering this first-call cost is the main ask.

## Problem — it's under-discoverable / under-powered

While QA-testing i18n menu-wrap, the menu-open path was not obvious and cost real time (fell back to blocked keyboard injection first):

1. **No `openshaders.*` entry.** Consumers look under the `openshaders.*` namespace for CS-specific controls (`openshaders.capture/feature/settings/shadercache` exist). There is **no `openshaders.menu`**, so the natural guess 404s. The CS menu is instead toggled through the *generic* built-in `menu` tool via `invoke` — non-obvious.
2. **`open` vs `invoke` is subtle.** The `menu` tool's description says *"open/close: menu to show/hide (e.g. TweenMenu)"* (engine RE::UI menus) and *"invoke: a registered menu"*. `open name:CommunityShaders` returns 400 (it's not an engine menu); only `invoke` works. The param is `name`, not `menu` — discovered only via the 400 error text.
3. **`describe` is empty for registered menus.** `menu {action:describe, name:"CommunityShaders"}` returns `{}`. There's no way to read the menu's section/tab tree or the visible label/value text, so QA must screenshot + eyeball instead of asserting on text.
4. **`invoke` only toggles visibility.** No way to navigate to a specific tab/section (Advanced, Upscaling, Light Limit Fix, …), so a script can't exercise each `DrawSettings` section programmatically — exactly what i18n / UI-regression QA needs.

## Suggestions

- **Docs:** add to `devbench-usage.md` (and the bridge README): *"Toggle the CS config menu: `menu {action:invoke, name:\"CommunityShaders\"}`."}* One line removes most of the friction.
- **Alias:** register an `openshaders.menu` tool (`open` / `close` / `toggle`, optional `section`) so it's discoverable in the namespace people actually search. It can delegate to the same registration.
- **`describe`:** make `menu describe name:CommunityShaders` return the menu descriptor — the section/tab list and, ideally, each rendered label + current value. This enables text-level assertions (e.g. "every visible string is non-empty / present in en.json") without screenshots.
- **Navigation:** let `invoke`/`openshaders.menu` take a `section`/`tab` arg to open the menu *and* select a section, so QA can sweep all `DrawSettings` sections (the i18n coverage use case).

## Why it matters

The fork's QA gate is SE+VR runtime smoke. For UI/i18n PRs, the only way to verify menu rendering today is screenshot + manual read; the menu can't be driven section-by-section, and its text can't be inspected. The above turns "open it and squint" into scriptable, assertable menu QA.

## Repro / evidence

- `menu {action:list}` → `registered: ["CommunityShaders", "devbench.selftest"]`
- `menu {action:open, name:"CommunityShaders"}` → `400 Bad Request`
- `menu {action:invoke, menu:"CommunityShaders"}` → `400 "action 'invoke' requires a 'name'"`
- `menu {action:invoke, name:"CommunityShaders"}` → `{"op":"toggle","queued":true}` ✅ (menu opens)
- `menu {action:describe, name:"CommunityShaders"}` → `{}` (no descriptor)

---

*Originally filed at alandtse/open-shaders#134 — moved here as this is a devbench API issue. Requested by @alandtse.*

Guía de contribución

No hay ninguna guía de contribución indexada para este repositorio

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.