alandtse / alandtse/devbench

devbench: surface + extend CS config-menu control for QA (discoverability, describe, section nav)

Ouverte
#37 2 commentaires 1 réaction 0 personnes assignées Voir sur GitHub
Langage dominant
C++
Étoiles
10
Forks
4
Merge moyen
1 j 10 h
PR mergées (30 j)
17

Description

## Background

devbench *can* drive the Community Shaders ImGui config menu — `menu {action:invoke, name:"CommunityShaders"}` toggles it (CS registers itself; it shows under `menu {action:list}` → `.registered`). This is the right tool for QA-ing the CS menu (e.g. verifying i18n string coverage, exercising every `DrawSettings` section).

## Cost: wasted MCP round-trips before real work

Opening the CS menu took **four** MCP calls to get right because the verb/param shape isn't guessable, and each wrong call is a full game round-trip:

1. `menu {action:open, name:"CommunityShaders"}` -> `400` (open is engine-menus-only)
2. `menu {action:invoke, menu:"CommunityShaders"}` -> `400` (wrong param name)
3. `menu {action:invoke, name:"CommunityShaders"}` -> works

For an agent driving the bench, every failed call is wasted latency and tokens before any real testing happens. The fix is to make the *first* guess work: a discoverable `openshaders.menu` tool and/or a one-line doc + a clearer error that names the exact working call. Lowering this first-call cost is the main ask.

## Problem — it's under-discoverable / under-powered

While QA-testing i18n menu-wrap, the menu-open path was not obvious and cost real time (fell back to blocked keyboard injection first):

1. **No `openshaders.*` entry.** Consumers look under the `openshaders.*` namespace for CS-specific controls (`openshaders.capture/feature/settings/shadercache` exist). There is **no `openshaders.menu`**, so the natural guess 404s. The CS menu is instead toggled through the *generic* built-in `menu` tool via `invoke` — non-obvious.
2. **`open` vs `invoke` is subtle.** The `menu` tool's description says *"open/close: menu to show/hide (e.g. TweenMenu)"* (engine RE::UI menus) and *"invoke: a registered menu"*. `open name:CommunityShaders` returns 400 (it's not an engine menu); only `invoke` works. The param is `name`, not `menu` — discovered only via the 400 error text.
3. **`describe` is empty for registered menus.** `menu {action:describe, name:"CommunityShaders"}` returns `{}`. There's no way to read the menu's section/tab tree or the visible label/value text, so QA must screenshot + eyeball instead of asserting on text.
4. **`invoke` only toggles visibility.** No way to navigate to a specific tab/section (Advanced, Upscaling, Light Limit Fix, …), so a script can't exercise each `DrawSettings` section programmatically — exactly what i18n / UI-regression QA needs.

## Suggestions

- **Docs:** add to `devbench-usage.md` (and the bridge README): *"Toggle the CS config menu: `menu {action:invoke, name:\"CommunityShaders\"}`."}* One line removes most of the friction.
- **Alias:** register an `openshaders.menu` tool (`open` / `close` / `toggle`, optional `section`) so it's discoverable in the namespace people actually search. It can delegate to the same registration.
- **`describe`:** make `menu describe name:CommunityShaders` return the menu descriptor — the section/tab list and, ideally, each rendered label + current value. This enables text-level assertions (e.g. "every visible string is non-empty / present in en.json") without screenshots.
- **Navigation:** let `invoke`/`openshaders.menu` take a `section`/`tab` arg to open the menu *and* select a section, so QA can sweep all `DrawSettings` sections (the i18n coverage use case).

## Why it matters

The fork's QA gate is SE+VR runtime smoke. For UI/i18n PRs, the only way to verify menu rendering today is screenshot + manual read; the menu can't be driven section-by-section, and its text can't be inspected. The above turns "open it and squint" into scriptable, assertable menu QA.

## Repro / evidence

- `menu {action:list}` → `registered: ["CommunityShaders", "devbench.selftest"]`
- `menu {action:open, name:"CommunityShaders"}` → `400 Bad Request`
- `menu {action:invoke, menu:"CommunityShaders"}` → `400 "action 'invoke' requires a 'name'"`
- `menu {action:invoke, name:"CommunityShaders"}` → `{"op":"toggle","queued":true}` ✅ (menu opens)
- `menu {action:describe, name:"CommunityShaders"}` → `{}` (no descriptor)

---

*Originally filed at alandtse/open-shaders#134 — moved here as this is a devbench API issue. Requested by @alandtse.*

Guide de contribution

Aucun guide de contribution indexé pour ce dépôt

Évaluation

Cette issue n'a pas encore été évaluée.

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.